Overwater target identification and tracking method and system based on multiband electromagnetic fusion perception technology

Abstract

The invention relates to the technical field of unmanned aerial vehicle target identification, in particular to an overwater target identification and tracking method and system based on a multiband electromagnetic fusion sensing technology. The infrared high-definition camera and the event type camera are combined for image fusion, the target recognition module is used for preliminary target detection, the target candidate area is confirmed, the misjudgment rate can be reduced, the system can effectively work under different illumination and meteorological conditions by adopting the multispectral camera, and the accuracy of the system is improved. The target identification and tracking capability is improved, the laser ranging technology is used in cooperation, accurate position data is provided for the target, the system can update the target state in real time, the tracking reliability is enhanced, and in addition, the multi-spectral analysis is carried out on the target, so that the tracking accuracy is improved. And spectral features output by the multispectral camera are analyzed in combination with the authenticity judgment module, so that a real target and a false signal can be effectively distinguished, and the safety and credibility of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) target recognition, and more particularly to a method and system for water target recognition and tracking based on multi-band electromagnetic fusion sensing technology. Background Art

[0002] With the rapid development of UAV technology, its applications in military, security, agriculture, environmental monitoring and other fields are becoming increasingly widespread. However, in the process of target recognition and tracking, traditional single-light-source sensors are difficult to meet the requirements of efficient and accurate recognition and tracking due to the influence of environmental changes (such as illumination, weather, etc.). Existing technologies mainly rely on a single type of sensor (such as an RGB camera or an infrared camera) for target recognition, and such a method is prone to problems such as lost tracking and misrecognition in complex environments.

[0003] To solve the above problems, multi-light-source fusion technology has been proposed in recent years. By combining different types of sensors and utilizing the complementarity of different information such as infrared and hyperspectral, the recognition accuracy and tracking ability of targets are improved. However, the current UAV target recognition and tracking system based on multi-light-source fusion still has the following defects:

[0004] 1. Complex data processing and insufficient real-time performance;

[0005] 2. Insufficient ability to distinguish between true and false targets, and low credibility of recognition results;

[0006] 3. Lack of an effective target position and status update mechanism, which affects the tracking accuracy and stability.

[0007] Therefore, adopting a method and system for water target recognition and tracking based on multi-band electromagnetic fusion sensing technology can effectively improve the robustness and real-time performance of target recognition, and at the same time improve the accuracy of target discrimination, which has important application value. Summary of the Invention

[0008] The purpose of the present invention is to provide a method and system for water target recognition and tracking based on multi-band electromagnetic fusion sensing technology to solve the problems of complex data processing, insufficient real-time performance, insufficient ability to distinguish between true and false targets, low credibility of recognition results, lack of an effective target position and status update mechanism, and affecting the tracking accuracy and stability proposed in the above background art.

[0009] To achieve the above purpose, the present invention provides a method for water target recognition and tracking based on multi-band electromagnetic fusion sensing technology, including the following steps:

[0010] S1. Startup preparation: Start a UAV equipped with an infrared high-definition camera, an event-based camera, and a multi-spectral camera to perform water area scanning;

[0011] S2. Acquisition and Processing of Target Images: Use an infrared high-definition camera and an event-based camera to obtain real-time water surface target images, and perform image fusion processing through an image fusion module to generate high-quality target images;

[0012] S3. Target Recognition and Detection: Transmit the processed image to the target recognition module for preliminary target detection and confirm the target candidate area;

[0013] S4. Target Authenticity Discrimination: Use a multi-spectral camera to deeply analyze the target, and at the same time use the authenticity discrimination module to combine spectral features to discriminate the authenticity of the target;

[0014] S5. Coordinate Calculation: Activate the laser rangefinder to obtain the accurate distance between the target and the UAV, and calculate the GPS coordinates of the target;

[0015] S6. Update Position and Movement Trajectory: Use the position and trajectory update module to real-time update the position and movement trajectory of the target according to the target detection results and distance data, and generate a target movement log;

[0016] S7. Data Feedback: Real-time feedback the target recognition and tracking data to the ground control terminal through the data transmission module.

[0017] As a further improvement of this technical solution, the specific operation method for startup preparation in step S1 is:

[0018] S11. Check the working status of each device and sensor of the UAV and ensure sufficient power;

[0019] S12. Install and connect the infrared high-definition camera, event-based camera and multi-spectral camera to ensure their normal operation;

[0020] S13. Start the UAV and perform self-check;

[0021] S14. Select the water area scanning mode on the control interface of the ground control terminal and set the navigation parameters, including altitude and speed;

[0022] S15. Take off and scan the predetermined area according to the set route.

[0023] As a further improvement of this technical solution, the specific operation method for target image acquisition and processing in step S2 is:

[0024] S21. Start real-time image acquisition and use the infrared high-definition camera to obtain the thermal imaging data around the water surface target;

[0025] S22. Use the event-based camera to capture high-contrast image data according to the movement or change of the target;

[0026] S23. Transmit the two sets of acquired data to the image fusion module;

[0027] S24. In the image fusion module, apply the weighted average method to merge the two sets of image data to generate a high-quality fused image.

[0028] As a further improvement of this technical solution, the specific operation method of target recognition and detection in step S3 is:

[0029] S31. Transmit the high-quality fused image to the target recognition module in real time through the built-in data processing channel;

[0030] S32. Load the target detection algorithm in the target recognition module, and use the algorithm to analyze the fused image, identify possible targets, and determine the candidate regions of the targets;

[0031] S33. Generate the border and feature information of the candidate regions according to the recognition results.

[0032] As a further improvement of this technical solution, the specific operation method of target authenticity discrimination in step S4 is:

[0033] S41. Send a control command to the multispectral camera through the ground control terminal to start multi-band data acquisition for the candidate target regions;

[0034] S42. Transmit the acquired multispectral data to the authenticity discrimination module;

[0035] S43. Apply the spectral feature analysis algorithm in the authenticity discrimination module to extract and analyze the spectral features of the target, match the spectral features of real and false targets recorded in the predefined feature library with the currently acquired data, and output the authenticity discrimination result of the target.

[0036] As a further improvement of this technical solution, the specific operation method of coordinate calculation in step S5 is:

[0037] S51. Start the laser rangefinder, set the ranging mode, emit and receive laser pulses to the target, and calculate the target distance in real time;

[0038] S52. Use the triangulation method to calculate the GPS coordinates of the target based on the GPS coordinates of the UAV and the measured target distance.

[0039] As a further improvement of this technical solution, the specific operation method of updating the position and movement trajectory in step S6 is:

[0040] S61. Input the target recognition and distance measurement results into the position and trajectory update module;

[0041] S62. Dynamically update the target position using algorithms such as the Kalman filter, considering historical trajectory data and current observations;

[0042] S63. Generate the motion trajectory of the target and regularly record the motion state of the target to generate a motion log.

[0043] As a further improvement of this technical solution, the specific operation method of data feedback in step S7 is as follows:

[0044] S71. Confirm the integrity of the target recognition, tracking, and motion log data, and send a data transmission request to the data transmission module;

[0045] S72. Real-time send the recognition and tracking data to the ground control terminal through the data transmission module.

[0046] The present invention also provides a water target recognition and tracking system based on multi-band electromagnetic fusion perception technology for implementing the above-mentioned water target recognition and tracking method based on multi-band electromagnetic fusion perception technology, including: an unmanned aerial vehicle (UAV), a fusion pod, and a ground control terminal. The fusion pod is installed on the UAV. The ground control terminal is used to control the navigation of the UAV. Inside the fusion pod, an infrared high-definition camera, an event camera, a laser rangefinder, and a multispectral camera are integrated. Inside the UAV, a data storage module, an image fusion module, a target recognition module, a authenticity discrimination module, a position trajectory update module, and a data transmission module are integrated;

[0047] The data storage module is used to store the data collected by the infrared high-definition camera, the event camera, the multispectral camera, and the laser rangefinder, and is also used to store the data processed by the image fusion module, the target recognition module, the authenticity discrimination module, the position trajectory update module, and the data transmission module;

[0048] The image fusion module is used to perform fusion processing on the target images obtained in real time by the infrared high-definition camera and the event camera;

[0049] The target recognition module is used to perform preliminary target detection on the target image after being fused by the image fusion module and confirm the target candidate area;

[0050] The authenticity discrimination module is used to discriminate the authenticity of the target by combining the spectral features output by the multispectral camera;

[0051] The position trajectory update module is used to real-time update the position and motion trajectory of the target according to the target detection result and distance data, and generate a target motion log;

[0052] The data transmission module is used to real-time feedback the target recognition and tracking data to the ground control terminal.

[0053] As a further improvement of the technical solution, the image fusion module, the target recognition module, the authenticity discrimination module, the position trajectory update module and the data transmission module are all bidirectionally electrically connected to the data storage module. The output ends of the infrared high-definition camera, the event camera, the laser rangefinder and the multispectral camera are all electrically connected to the input end of the data storage module. The data transmission module and the ground control terminal maintain real-time data intercommunication through the Internet.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] 1. In the present invention, by combining the infrared high-definition camera and the event camera for image fusion, and using the target recognition module to perform preliminary target detection on the target image after image fusion processing to confirm the target candidate area, the target recognition accuracy is improved and the misjudgment rate is reduced in a complex environment.

[0056] 2. In the present invention, through the application of the multispectral camera, the system can effectively work under different lighting and meteorological conditions, improving the recognition and tracking ability of water targets. At the same time, combined with the laser ranging technology, accurate position data is provided for the target, enabling the system to update the target status in real time, thereby enhancing the reliability of tracking.

[0057] 3. In the present invention, through the multispectral analysis of the target and the analysis of the spectral characteristics output by the multispectral camera by the authenticity discrimination module, real targets and false signals are effectively distinguished, improving the safety and credibility of the system. Description of the Drawings

[0058] Figure 1 It is a schematic diagram of the overall steps of the method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention.

[0059] Figure 2 It is a schematic diagram of the first step of the method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention.

[0060] Figure 3 It is a schematic diagram of the second step of the method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention.

[0061] Figure 4 It is a schematic diagram of the third step of the method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention.

[0062] Figure 5 It is a schematic diagram of the fourth step of the method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention.

[0063] Figure 6Schematic diagram of the fifth step of the method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention.

[0064] Figure 7 Schematic diagram of the sixth step of the method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention.

[0065] Figure 8 Schematic diagram of the seventh step of the method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention.

[0066] Figure 9 Block diagram of the system for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology of the present invention. Specific implementation mode

[0067] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0068] In a specific embodiment, as Figure 1 shown, the present invention provides a method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology, including the following steps:

[0069] The first step, as Figure 2 shown, start preparation: Start a drone equipped with an infrared high-definition camera, an event camera, and a multispectral camera to scan the water area.

[0070] 1. Check the working status of each device and sensor of the drone and ensure that the battery is fully charged.

[0071] Battery check: Turn on the power management system of the drone and check the remaining battery power. If the battery power is lower than the set safety threshold (for example, 30%), charging or battery replacement is required.

[0072] Device status check: Enter the self-check interface of the drone and check the status of the following devices one by one:

[0073] Motor - Start the motor and observe whether it is running normally. Check if there are any abnormal sounds or vibrations in the motor.

[0074] Sensor - Conduct individual tests on the infrared high-definition camera, event camera, and multispectral camera to ensure that the image acquisition function is normal. The sampling rate of each sensor can be calculated using the following formula:

[0075]

[0076] GPS module - Ensure that the GPS module can successfully receive satellite signals, and the position accuracy can be evaluated by signal strength and the number of satellites (for example, at least 4 satellite signals are acquired to ensure 3D positioning).

[0077] Check the data link - Ensure that the data link between the sensor and the control system is normal. The data transmission can be manually tested to verify that the delay is within the allowable range (such as less than 100 ms).

[0078] 2. Install and connect the infrared high-definition camera, event camera, and multispectral camera to ensure they can work properly.

[0079] Install the sensors: Fix the infrared high-definition camera, event camera, and multispectral camera on the pod according to the installation manual provided by the manufacturer and ensure they are stable.

[0080] Connect the power supply and data cable:

[0081] Power connection - Connect the power cable of each camera to the power management system of the drone. Usually, voltage matching needs to be noted.

[0082] Data connection - Use the corresponding interfaces (such as USB, HDMI, or dedicated interfaces) to connect the data cables of the cameras to the computing unit of the drone.

[0083] Start the sensors: Turn on the power of each camera and ensure that the indicator lights are on normally, showing the working status of the device.

[0084] Conduct functional tests: View the real-time video stream of each camera in the control system to confirm whether it works properly and can capture high-quality images. For the infrared high-definition camera, pixel intensity maps can be used for analysis to confirm whether the set sensitivity standard can be achieved.

[0085] 3. Start the drone and conduct a self-check.

[0086] Drone startup: Press the startup button on the drone control interface, and the system will prompt for a self-check.

[0087] Self-check process:

[0088] Hardware self-check - The system automatically checks the status of the motors, sensors, GPS module, and all electronic devices. Ensure that all devices report normal.

[0089] Software self-check - Check whether the version of the control system is the latest and load the relevant flight control programs and algorithms.

[0090] Self-check result feedback:

[0091] If the self - test passes, the system will display "Self - test successful" and be ready to enter the flight mode.

[0092] If problems are found, the system will prompt specific fault information and recommend maintenance or replacement.

[0093] Boot adjustment - According to the self - test results, make corresponding adjustments (such as replenishing power, fixing faults, etc.) until all devices are in a normal state before proceeding to the next step.

[0094] 4. Select the water area scanning mode on the control interface of the ground control terminal and set the navigation parameters, including altitude and speed.

[0095] Select the operation mode: In the main menu of the ground control terminal, select "Water area scanning mode". The system will switch to this mode and display relevant options.

[0096] Set the flight altitude:

[0097] The user enters the desired flight altitude (e.g., 200 meters) on the interface, and the system will automatically adjust according to the flight state and safety parameters.

[0098] Altitude limit formula: H 允许 = H 最大 - height of water surface obstacles - safety margin.

[0099] Ensure that the entered altitude does not exceed the set maximum altitude (e.g., 500 meters) and the acceptable obstacle height.

[0100] Set the flight speed: Select an appropriate forward speed (e.g., 8 m / s), and the system will adjust the speed according to the acquisition ability of the sensor.

[0101] Relationship formula between speed and image acquisition: V×T≤D 最大采集距 .

[0102] Where V is the set speed, T is the image acquisition interval, and D 最大采集距 is the maximum recognition distance that can be maintained at this speed.

[0103] 5. Take off and scan the predetermined area according to the set route.

[0104] Take - off preparation: Ensure that the drone is in a ready - to - fly state and all parameters have been successfully set.

[0105] Send the take - off command: Click the "Take - off" button on the ground control terminal, and the system will start the take - off procedure. The drone will automatically take off vertically to the set flight altitude.

[0106] Route Planning and Startup: The system automatically generates the optimal route based on the set predefined area, and can calculate the shortest path through the Dijkstra algorithm.

[0107] Start the drone for autonomous navigation, and at the same time, monitor the position and altitude in real time during flight to ensure stability.

[0108] Scanning Execution: During the flight of the drone, adjust the active state of the sensors, and achieve full coverage scanning of the target water area according to the set flight speed and altitude. Each sensor records the data collected in real time to the internal storage or forwards it to the cloud to ensure that the data is always available.

[0109] Second step, as Figure 3 shown, Target Image Acquisition and Processing: Use an infrared high-definition camera and an event-based camera to obtain the water surface target image in real time, and perform image fusion processing through the image fusion module to generate a high-quality target image.

[0110] 1. Start real-time image acquisition, and use an infrared high-definition camera to obtain the thermal imaging data around the water surface target.

[0111] Start the infrared high-definition camera: Execute the start command on the ground control terminal to activate the infrared high-definition camera. The device should be in a working state and start collecting thermal imaging data.

[0112] Set parameters: Set the corresponding parameters according to the environmental conditions:

[0113] Sensitivity - Adjust the sensitivity of the infrared sensor to optimize the thermal imaging effect, especially in low-temperature or high-contrast situations.

[0114] Frame Rate - Set an appropriate frame rate (such as 30Hz) to ensure that enough thermal image data is collected per second.

[0115] Real-time Data Acquisition: Use the infrared high-definition camera to continuously monitor the water surface target area and generate thermal imaging maps in real time. The following formula can be used to describe the thermal image collected at each time point:

[0116] I t = K·A(t) + B.

[0117] Where, I t is the thermal imaging value at time t, K is the gain, A(t) is the thermal radiation characteristic of the target, and B is the influence of the background temperature.

[0118] Data Saving and Processing: Temporarily store the obtained thermal imaging data in the memory for subsequent transmission and processing.

[0119] 2. Use an event-based camera to capture high-contrast image data based on the movement or change of the target.

[0120] Start the event-based camera: Issue a command on the ground control terminal to start the event-based camera in order to capture the movement or change of the water surface target in real time.

[0121] Set event detection parameters:

[0122] Sensitivity - Adjust the sensitivity parameter to ensure that the movement of the water surface target can still be captured under changing environmental conditions.

[0123] Trigger Threshold - Set the contrast threshold for triggering. For example, when the brightness change in the scene exceeds the set 5%, trigger the acquisition.

[0124] Capture high-contrast images: When the movement of the water surface target or the change of the scene is detected, the event-based camera quickly captures high-contrast images, generating a series of clear instantaneous images.

[0125] Data caching: Cache the captured high-contrast image data to ensure that these data can be quickly transmitted to the subsequent processing module during image merging.

[0126] 3. Transmit the two sets of obtained data to the image fusion module.

[0127] Establish a data transmission channel: Ensure that the output data of the infrared high-definition camera and the event-based camera can be transmitted through a data information channel (such as USB, CAN, Wi-Fi, etc.).

[0128] Prepare the data format: Convert the two sets of image data (infrared images and event-based images) to ensure that they can be accepted by the image fusion module. This usually requires the following steps:

[0129] Ensure that the data types of the two sets of images are the same (for example, both are 8-bit or 16-bit grayscale images).

[0130] Unify the image resolution (such as 640x480 or 1280x720).

[0131] Data transmission: Transmit the two sets of data stored in the memory to the image fusion module in real time through the data channel. The following formula can be used to calculate the data transmission rate:

[0132]

[0133] Where R is the transmission rate (bps), D is the total amount of data (bytes), and T is the transmission time required (seconds).

[0134] Confirmation of receipt: After the image fusion module receives the data, it triggers a confirmation response to ensure the integrity and accuracy of the data.

[0135] 4. In the image fusion module, the weighted average method is applied to merge two sets of image data to generate a high-quality fused image.

[0136] Initialization of the image fusion algorithm: Initialize the weighted average method processing module in the image fusion module to prepare for the fusion process.

[0137] Setting weight parameters: According to the image features (such as the clarity and importance of infrared images and event-based images), set the weight parameters of the images. Assuming the weighting parameters are w1 and w2 respectively, usually w1 + w2 = 1.

[0138] Execute the weighted average method: Perform a weighted average calculation for each pixel point, and merge the infrared image I IR and the event-based image I E :

[0139] I fused (x,y) = w1·I IR (x,y) + w2·I E (x,y).

[0140] Among them, I fused (x,y) is the value of the fused image at the point (x,y).

[0141] Generate the fused image: By performing this calculation on the entire image, generate a high-quality fused image, and store the image in the processed intermediate data or directly output it to the target recognition module.

[0142] Quality assessment: Perform quality assessment on the fused image (such as using SSIM or PSNR metrics) to ensure that it meets the requirements of subsequent processing in terms of clarity and information richness.

[0143] The third step, as Figure 4 shown, target recognition and detection: Transmit the processed image to the target recognition module for preliminary target detection and confirm the target candidate area.

[0144] 1. Transmit the high-quality fused image in real time to the target recognition module through the built-in data processing channel.

[0145] Prepare the data transmission channel: Ensure that the data channel between the image fusion module and the target recognition module is active and efficient. Common transmission channels may include high-speed serial ports, Ethernet, or Wi-Fi connections.

[0146] Data Formatting: Before sending the fused image, format the image appropriately to ensure the use of standard image formats (such as JPEG, PNG, or a specific RAW format) so that the target recognition module can correctly parse the image content.

[0147] Real-time Data Transmission: Use encoding and compression algorithms to ensure efficient data transmission during the process, avoiding delays and packet losses. The following transmission rate formula can be referred to:

[0148] R a =R t ×E.

[0149] Where, R a is the actual transmission rate, R t is the theoretical maximum transmission rate, and E represents the proportion of valid data in the total transmission volume.

[0150] Confirmation of Receipt: After receiving the image, the target recognition module should promptly return a confirmation signal indicating that the data is complete and error-free and is ready for analysis and processing.

[0151] 2. Load the target detection algorithm in the target recognition module and use the algorithm to analyze the fused image, identify possible targets, and determine the candidate regions of the targets.

[0152] Loading the Target Detection Algorithm: Start the target recognition module and load the pre-selected target detection algorithm. Deep learning-based methods (such as Convolutional Neural Network CNN) or traditional non-depth methods (such as Haar feature classifier) can be used.

[0153] Preprocessing the Fused Image: Before applying the target detection algorithm, perform image preprocessing, such as image normalization, size adjustment, denoising, etc. This can be done through the following formula:

[0154]

[0155] Where, I p =is the processed image, I is the original image, μ is the mean of the image, and σ is the standard deviation of the image.

[0156] Performing Target Recognition: Use the loaded target detection algorithm to analyze the fused image and identify the possible existing targets. In the target detection network (such as YOLO, Faster R-CNN, etc.), the algorithm will automatically generate a large number of candidate target regions and classify and score them according to features.

[0157] Determine candidate regions: Based on the output of the recognition algorithm, extract possible target candidate regions from the image and generate region proposals. These regions can usually be represented by bounding boxes, which are represented in the form of rectangular boxes in the detection algorithm as: B(x, y, w, h), where (x, y) are the coordinates of the upper left corner of the bounding box, w is the width, and h is the height.

[0158] 3. Generate the bounding box and feature information of the candidate region according to the recognition result.

[0159] Generate the bounding box of the candidate region: Use the coordinate and size information output by the object detection algorithm to generate a bounding box (bounding box) for each candidate region and mark it in the fused image. The change in the gray value (or color value) can help determine whether dynamic adjustment of the bounding box is required.

[0160] Extract feature information: For each generated candidate region, extract its feature information, which may include:

[0161] Color histogram: It can be obtained by calculating the color distribution of the pixels within the candidate region.

[0162] Texture feature: Use a texture filter (such as a Gabor filter) to extract the texture information of the candidate region.

[0163] Shape feature: Use a shape recognition algorithm to extract shape feature information (such as Hu invariant moments).

[0164] Combine feature information: Combine the above feature information together to form a feature vector for each candidate target, denoted as: (F = C, T, S), where C represents the color feature, T represents the texture feature, and S represents the shape feature.

[0165] Return the recognition result: Send the bounding box and feature information to the authenticity discrimination module for further analysis and decision-making.

[0166] Step 4, as Figure 5 shown, target authenticity discrimination: Use a multispectral camera to conduct in-depth analysis of the target, and at the same time use the authenticity discrimination module to combine spectral features to discriminate the authenticity of the target.

[0167] 1. Send a control command to the multispectral camera through the ground control terminal to start multi-band data acquisition for the candidate target region.

[0168] Start the multispectral camera: Send a command on the ground control terminal to start the multispectral camera for data acquisition. This device should ensure that it is in the "data acquisition" mode to receive commands and start working.

[0169] Configure acquisition parameters: Set the relevant parameters of the multispectral camera, including:

[0170] Band selection - According to the characteristics of the target to be monitored, select appropriate spectral bands, such as multiple bands of visible light, near-infrared, and short-wave infrared.

[0171] Image resolution - Set the image resolution to ensure high-quality data acquisition, such as 640x480 or 1280x720.

[0172] Start data acquisition: The multispectral camera performs real-time data acquisition on the candidate target area according to the preset bands, forming multi-band spectral image data.

[0173] Data caching: Cache the real-time acquired multispectral data into the transmission module for subsequent transmission.

[0174] 2. Transmit the acquired multispectral data to the authenticity discrimination module.

[0175] Establish a data transmission channel: Ensure that the data channel between the multispectral camera and the authenticity discrimination module is unobstructed. Usually, high-speed serial ports, Wi-Fi, or other reliable transmission methods are used.

[0176] Format the multispectral data: Before transmission, format the multispectral data into a standard format (such as TIFF or HDF5) to ensure that the data of each band can be correctly parsed by the authenticity discrimination module.

[0177] Data transmission: Transmit the multispectral image data stored in the memory to the authenticity discrimination module in real-time through the data channel. Use compression and encoding technologies to improve transmission efficiency and reduce transmission time.

[0178] Confirm receipt: After receiving the data, the authenticity discrimination module immediately returns a confirmation signal, indicating that the multispectral data has been completely received and is ready for the next step of processing.

[0179] 3. Apply spectral feature analysis algorithms in the authenticity discrimination module to extract and analyze the spectral features of the target. According to the spectral features of real and false targets recorded in the predefined feature library, match them with the currently acquired data and output the authenticity discrimination result of the target.

[0180] Load spectral feature analysis algorithms: Load spectral feature analysis algorithms in the authenticity discrimination module. Commonly used algorithms include Spectral Angle Mapper (SAM), Support Vector Machine (SVM), etc. The selection of the algorithm should match the characteristics of the target.

[0181] Extract spectral features: Perform spectral feature extraction on the received multispectral data, usually including:

[0182] Spectral reflectance: Calculate the spectral reflectance for each band, which is used to analyze the spectral characteristics of the target.

[0183] Average spectrum: Calculate the average spectrum of all pixels within the candidate target area.

[0184] Match with the feature library: Match the extracted spectral features with the spectral features of real targets and false targets stored in the predefined feature library.

[0185] Output the true / false discrimination result: Determine the authenticity of the target based on the matching result and its similarity, and output the discrimination result.

[0186] Step 5: As Figure 6 shown, coordinate calculation: Activate the laser rangefinder to obtain the accurate distance between the target and the UAV, and calculate the GPS coordinates of the target.

[0187] 1. Start the laser rangefinder, set the ranging mode, emit a laser pulse towards the target and receive it, and calculate the target distance in real time.

[0188] Select the ranging mode: Select a suitable ranging mode according to the application requirements, such as single-shot ranging mode or continuous ranging mode. Configure relevant parameters on the control interface, such as ranging accuracy, data acquisition frequency, etc.

[0189] Emit a laser pulse: The laser rangefinder emits a laser pulse and irradiates the target directionally. The laser pulse is reflected after hitting the target object and returns to the data receiver of the rangefinder.

[0190] Receive the reflected pulse: The laser rangefinder receives the reflected laser pulse and records the return time t. This time is used to calculate the target distance.

[0191] Calculate the target distance: Use the laser wave speed (usually the speed of light c) and the return time t to calculate the distance D from the target to the laser rangefinder. The formula is: where D represents the target distance, c is the speed of light approximately 3×10 8 m / s, and the division by 2 here is because the laser pulse needs to travel a round-trip distance.

[0192] 2. Use the triangulation method to calculate the GPS coordinates of the target based on the GPS coordinates of the UAV and the measured target distance.

[0193] Obtain the GPS coordinates of the UAV: Obtain the current GPS coordinates of the UAV from the navigation system of the UAV, denoted as (X u , Y u , H u ), where X u , Y u represents the position of the UAV in the plane coordinate system, and Hu is the height coordinate.

[0194] Set the triangulation reference point: Assume the position of the rangefinder on the UAV is (X m , Y m , H m ), usually X m = X u and Y m = Y u , and the height H m may be the set height of the rangefinder.

[0195] Use triangulation to calculate the target coordinates: According to the measured target distance D and the height of the UAV, use triangulation to calculate the GPS coordinates (X t , Y t , H t ) of the target, where H t can be estimated by knowing the incident and reflection angles of the light (usually it can be assumed to be on the same horizontal plane to reduce errors). Assume the observation height difference of the rangefinder is ΔH, which can be expressed as: H t = H m - ΔH.

[0196] Calculate the horizontal coordinates: According to the range D, use trigonometric functions to calculate the coordinates of the target on the horizontal plane:

[0197] X t = X m + D·cos(θ), Y t = Y m + D·sin(θ).

[0198] Among them, θ is the horizontal viewing angle between the UAV and the target, and cos(θ) and sin(θ) respectively represent the ratios of the horizontal distances to the target.

[0199] Output the GPS coordinates of the target: Convert the calculated target coordinates (X t , Y t , H t ) into the standard GPS coordinate format for further processing or visualization in the system. The output result will include the longitude, latitude and height information of the target.

[0200] Step 6, as Figure 7 shown, update the position and motion trajectory: Use the position trajectory update module to update the position and motion trajectory of the target in real time according to the target detection result and distance data, and generate a target motion log.

[0201] 1. Input the target recognition and distance measurement results into the position trajectory update module.

[0202] Obtain the target recognition result: Extract the recognition information of the target at the current moment from the target recognition module, including the ID, features, and known state parameters (such as position, speed, etc.) of the target.

[0203] Collect distance measurement data: Obtain the measured target distance D and the current UAV's GPS coordinates (X u , Y u , H u ) from a laser rangefinder or other sensors.

[0204] Data integration: Integrate the target recognition result with the distance measurement result to form an updated data packet.

[0205] Send to the position trajectory update module: Input the sorted updated data packet into the position trajectory update module. This can be achieved through a software interface or real-time data stream operation.

[0206] 2. Use algorithms such as the Kalman filter to dynamically update the target position, considering historical trajectory data and current observations.

[0207] Initialize the Kalman filter: Define the initial parameters of the Kalman filter, including the initial state estimate and the initial error covariance matrix. The initial state estimate usually depends on the previous target state.

[0208] Prediction step: Before each update, predict the target state according to the motion model. Use the state transition matrix and control input for state prediction.

[0209] Update step: Receive new observations and update the state estimate through the Kalman gain.

[0210] Tracking state update: Store the updated state in the historical trajectory data for subsequent analysis and recording.

[0211] 3. Generate the target's motion trajectory and regularly record the target's motion state to generate a motion log.

[0212] Record motion state information: After each position update, extract the current target's state information (including position, speed, acceleration, etc.) and record it in the motion state list.

[0213] Motion trajectory generation: Generate the target's motion trajectory based on the historical state data of the evading target. The motion trajectory is usually represented by a planar coordinate diagram and can form a polyline trajectory by connecting historical position points.

[0214] Regularly generate a motion log: Generate a motion log at regular intervals (such as every second or every 5 seconds). The log content includes a timestamp, target id, position coordinates, speed, etc., and can be optionally converted to a text format or database record.

[0215] Storage and backup function: Store the generated motion logs and motion trajectories in the data storage module to ensure data security and enable convenient access and analysis at any time.

[0216] Step 7: As Figure 8 shown, data feedback: Real-time feedback of target recognition and tracking data to the ground control terminal through the data transmission module.

[0217] 1. Confirm the integrity of target recognition, tracking, and motion log data, and send a data transmission request to the data transmission module.

[0218] Data integrity check: First, the system needs to perform integrity and validity checks on target recognition, tracking data, and motion logs. This includes:

[0219] Integrity of target recognition data - Verify whether each recognized target has corresponding status information (such as ID, features, distance).

[0220] Integrity of tracking data - Ensure that each updated status is recorded and the timestamps are in the correct order.

[0221] Integrity of motion logs - Confirm whether the motion status records include necessary parameters such as timestamps, target positions, speeds, and accelerations.

[0222] Use a simple algorithm to calculate data integrity:

[0223] Integrity check = (Number of recognized targets == Expected number of targets) ∧ (Number of tracking data == Number of timestamps).

[0224] If the integrity check fails, an alarm should be triggered and a log should be recorded.

[0225] Send a data transmission request: If the integrity check of the above data passes, send a data transmission request to the data transmission module.

[0226] 2. Real-time send the recognition and tracking data to the ground control terminal through the data transmission module.

[0227] Establish a data connection: The data transmission module first establishes a wireless communication connection with the ground control terminal.

[0228] Confirm connection success: Confirm the feedback received from the ground control terminal to ensure a clear communication channel. This confirmation can include the version of the data transmission protocol and reliability verification.

[0229] Data packaging: Organize the recognition and tracking data into a suitable format, which usually includes hexadecimal encoding of the data, adding a data header and a data tail.

[0230] Real-time transmission of data: The data packet is sent to the ground control terminal through the wireless transmission module with minimal delay. The UDP or TCP protocol can be used for data transmission, and the most suitable protocol is selected to meet the real-time requirements.

[0231] Confirm the reception result: After the data transmission is completed, verify the feedback from the ground control terminal to confirm whether the data is correctly received and processed. If the data reception fails, the data needs to be resent, and the number of transmissions and error information are recorded.

[0232] As Figure 9 shown, based on the above embodiment, the present invention further provides an underwater target recognition and tracking system based on multi-band electromagnetic fusion sensing technology for implementing the above-mentioned underwater target recognition and tracking method based on multi-band electromagnetic fusion sensing technology, including an unmanned aerial vehicle (UAV), a fusion pod, and a ground control terminal.

[0233] The UAV is controlled by the ground control terminal. At the same time, the fusion pod is installed on the UAV, and the fusion pod is internally integrated with an infrared high-definition camera, an event camera, a laser rangefinder, and a multispectral camera. At the same time, the UAV is also internally integrated with a data storage module, an image fusion module, a target recognition module, a authenticity discrimination module, a position and trajectory update module, and a data transmission module.

[0234] During use, the water area is scanned through the infrared high-definition camera, event camera, and multispectral camera inside the fusion pod. First, the infrared high-definition camera and event camera are used to obtain the water surface target image in real time, and the image fusion module is used for image fusion processing to generate a high-quality target image. Then, the fused image is transmitted to the target recognition module for preliminary target detection and confirmation of the target candidate area. Next, the multispectral camera is used to deeply analyze the target and output spectral features. At the same time, the authenticity discrimination module is used to combine the spectral features to discriminate the authenticity of the target. Then, the laser rangefinder is used to obtain the accurate distance between the target and the UAV, and the GPS coordinates of the target are calculated. Next, the position and trajectory update module is used to update the position and movement trajectory of the target in real time according to the target detection result and distance data, and generate a target movement log. Finally, the target recognition and tracking data are real-time fed back to the ground control terminal through the data transmission module, so as to ensure that the ground control terminal can better control the flight of the UAV by combining information.

[0235] The basic principles, main features and advantages of the present invention have been shown and described above. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. An underwater target recognition and tracking method based on multi-band electromagnetic fusion perception technology, characterized in that It includes the following steps: S1. Startup Preparation: Start a drone equipped with an infrared high-definition camera, an event camera, and a multispectral camera to conduct a water area scan; S2. Target Image Acquisition and Processing: Use the infrared high-definition camera and the event camera to acquire water surface target images in real time, and perform image fusion processing through an image fusion module to generate high-quality target images; S3. Target Recognition and Detection: Transmit the processed image to the target recognition module for preliminary target detection and confirm the target candidate area; S4. Target Authenticity Discrimination: Use the multispectral camera to conduct in-depth analysis of the target, and at the same time use the authenticity discrimination module to combine spectral features to discriminate the authenticity of the target; S5. Coordinate Calculation: Activate the laser rangefinder to obtain the accurate distance between the target and the drone, and calculate the GPS coordinates of the target; S6. Update Position and Movement Trajectory: Use the position and trajectory update module to update the position and movement trajectory of the target in real time according to the target detection results and distance data, and generate a target movement log; S7. Data Feedback: Real-time feedback the target recognition and tracking data to the ground control terminal through the data transmission module.

2. The method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology according to claim 1, wherein, The specific operation method for startup preparation in step S1 is as follows: S11. Check the working status of each device and sensor of the drone and ensure sufficient power; S12. Install and connect the infrared high-definition camera, the event camera, and the multispectral camera to ensure their normal operation; S13. Start the drone for self-check; S14. Select the water area scan mode on the control interface of the ground control terminal and set the navigation parameters, including altitude and speed; S15. Take off and scan the predetermined area according to the set route.

3. The method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology according to claim 1, characterized in that The specific operation method for target image acquisition and processing in step S2 is as follows: S21. Start real-time image acquisition, and use the infrared high-definition camera to obtain the thermal imaging data around the water surface target; S22. Use the event camera to capture high-contrast image data according to the movement or change of the target; S23. Transmit the two sets of obtained data to the image fusion module; S24. In the image fusion module, apply the weighted average method to merge the two sets of image data to generate a high-quality fused image.

4. The method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology according to claim 1, characterized in that, The specific operation method for target recognition and detection in step S3 is as follows: S31. Transmit the high-quality fused image in real time through the built-in data processing channel to the target recognition module; S32. Load the target detection algorithm in the target recognition module, and use the algorithm to analyze the fused image to identify possible targets and determine the candidate area of the target; S33. Generate the border and feature information of the candidate area according to the recognition result.

5. The method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology according to claim 1, wherein The specific operation method for target authenticity discrimination in step S4 is as follows: S41. Send a control command to the multispectral camera through the ground control terminal to start multi-band data acquisition for the candidate target area; S42. Transmit the collected multispectral data to the authenticity discrimination module; S43. Apply the spectral feature analysis algorithm in the authenticity discrimination module to extract and analyze the spectral features of the target. Match the spectral features of the real target and the false target recorded in the predefined feature library with the currently acquired data, and output the authenticity discrimination result of the target.

6. The method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology according to claim 5, wherein The specific operation method of coordinate calculation in step S5 is as follows: S51. Start the laser rangefinder, set the ranging mode, emit and receive laser pulses to the target, and calculate the target distance in real time. S52. Use the triangulation method to calculate the GPS coordinates of the target based on the GPS coordinates of the UAV and the measured target distance.

7. The method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology according to claim 1, characterized in that The specific operation method of updating the position and motion trajectory in step S6 is as follows: S61. Input the target recognition and distance measurement results into the position and trajectory update module. S62. Use algorithms such as the Kalman filter to dynamically update the target position, considering the historical trajectory data and the current observation values. S63. Generate the motion trajectory of the target and regularly record the motion state of the target to generate a motion log.

8. The method for identifying and tracking water targets based on multi-band electromagnetic fusion perception technology according to claim 7, characterized in that The specific operation method of data feedback in step S7 is as follows: S71. Confirm the integrity of the target recognition, tracking, and motion log data, and send a data transmission request to the data transmission module. S72. Real-time send the recognition and tracking data to the ground control terminal through the data transmission module.

9. An on-water target recognition and tracking system based on multi-band electromagnetic fusion perception technology is used to implement the on-water target recognition and tracking method based on multi-band electromagnetic fusion perception technology according to any one of claims 1-8, and is characterized in that, Including: A UAV, a fusion pod, and a ground control terminal. The fusion pod is installed on the UAV. The ground control terminal is used to control the navigation of the UAV. The fusion pod internally integrates an infrared high-definition camera, an event camera, a laser rangefinder, and a multispectral camera. The UAV internally integrates a data storage module, an image fusion module, a target recognition module, an authenticity discrimination module, a position and trajectory update module, and a data transmission module. The data storage module is used to store the data collected by the infrared high-definition camera, the event camera, the multispectral camera, and the laser rangefinder, and is also used to store the data processed by the image fusion module, the target recognition module, the authenticity discrimination module, the position and trajectory update module, and the data transmission module. The image fusion module is used to perform fusion processing on the target images obtained in real time by the infrared high-definition camera and the event camera. The target recognition module is used to perform preliminary target detection on the target image after being fused by the image fusion module and confirm the target candidate area. The authenticity discrimination module is used to discriminate the authenticity of the target by combining the spectral features output by the multispectral camera. The position and trajectory update module is used to update the position and motion trajectory of the target in real time according to the target detection result and the distance data, and generate a target motion log. The data transmission module is used to feedback the target recognition and tracking data to the ground control terminal in real time.

10. The water target recognition and tracking system based on multi-band electromagnetic fusion perception technology according to claim 9, characterized in that, The image fusion module, target recognition module, authenticity discrimination module, position and trajectory update module, and data transmission module are all bidirectionally electrically connected to the data storage module. The output ends of the infrared high-definition camera, event camera, laser rangefinder, and multispectral camera are all electrically connected to the input end of the data storage module. The data transmission module and the ground control terminal maintain real-time data intercommunication through the Internet.

Images