Intelligent optics and radar cooperative aerial target tracking method and device

Abstract

The invention discloses an intelligent optics and radar cooperative aerial target tracking method and device. The method comprises the following steps: acquiring a visible light image and an SAR image; respectively performing target detection tracking on the visible light image and the SAR image to obtain a first target detection tracking result and a second target detection tracking result; carrying out consistency judgment, and when a consistency judgment result does not meet a first preset condition, respectively carrying out target detection tracking mapping on the visible light image and the SAR image to obtain a corresponding next frame target tracking result; distributing a tracking confidence coefficient to the next frame of target tracking result, and when the tracking confidence coefficient does not meet a second preset condition, fusing the first target detection tracking result and the second target detection tracking result to obtain a fused detection tracking result; and adjusting the fusion detection tracking result to obtain a final target tracking result. According to the invention, the precision, robustness and adaptability of aerial target tracking can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence security technology, and particularly to an air target tracking method, device, storage medium and electronic device that combines intelligent optics and radar. Background Art

[0002] Air target tracking technology is widely used in military reconnaissance, unmanned aerial vehicle (UAV) navigation, aerospace monitoring and other fields. With the rapid development of aviation technology and diverse requirements, how to achieve efficient and accurate target tracking has become an important technical challenge.

[0003] Traditional target tracking relies on a single sensor, such as a visible light camera or a synthetic aperture radar (SAR). Each of these two sensors has its own advantages and limitations. Visible light imaging technology can provide clear target images and is suitable for daytime or well-lit environments, but its performance drops significantly in low visibility, night or adverse weather conditions. In contrast, SAR images can penetrate obstacles such as clouds, rain and snow and have strong adaptability in complex environments, but the image resolution is low and the target features are often not as clear as visible light images. Summary of the Invention

[0004] Embodiments of this application provide an air target tracking method, device, storage medium and electronic device that combines intelligent optics and radar, which can improve the accuracy, robustness and adaptability of air target tracking.

[0005] Embodiments of this application provide an air target tracking method that combines intelligent optics and radar, including: Obtain visible light images and SAR images; Perform target detection and tracking on the visible light images to obtain a first target detection and tracking result, and perform target detection and tracking on the SAR images to obtain a second target detection and tracking result; Perform a consistency determination on the first target detection and tracking result and the second target detection and tracking result. When the consistency determination result does not meet the first preset condition, perform target detection and tracking mapping on the visible light images and the SAR images respectively to obtain corresponding next-frame target tracking results; Assign tracking confidence to the next-frame target tracking results. When the tracking confidence does not meet the second preset condition, fuse the first target detection and tracking result and the second target detection and tracking result to obtain a fused detection and tracking result; Adjust the fused detection and tracking result to obtain the final target tracking result.

[0006] Further, for the above-mentioned method for tracking airborne targets by collaborative intelligent optics and radar, wherein, performing target detection and tracking on the SAR image to obtain a second target detection and tracking result includes: Inputting the SAR image into a trained deep neural network model based on multi-scale dilated convolution and pyramid module to obtain a second target detection and tracking result.

[0007] Further, for the above-mentioned method for tracking airborne targets by collaborative intelligent optics and radar, wherein the training process of the deep neural network model based on multi-scale dilated convolution and pyramid module includes: Obtaining a training sample set; wherein the training sample set includes a first type of training sample set and a second type of training sample set, the first type of training sample set includes samples with non-scale overlapping targets and the target center not at the center position of the 9-grid, and the second type of training sample set includes samples with scale overlapping targets and the target center located at the middle position of the 9-grid; Inputting the sample data in the training sample set into a deep neural network model based on multi-scale dilated convolution and pyramid module to obtain a prediction box; Calculating the overlap degree between the prediction box and the ground truth box, calculating a loss function based on the overlap degree, and iteratively training the deep neural network model based on multi-scale dilated convolution and pyramid module through the loss function until the overlap degree satisfies being greater than the preset overlap degree value and then stopping the training.

[0008] Further, for the above-mentioned method for tracking airborne targets by collaborative intelligent optics and radar, wherein the first preset condition is: the first target detection and tracking result and the second target detection and tracking result track the same target, and the target is in the same scene; Performing consistency determination on the first target detection and tracking result and the second target detection and tracking result includes: Calculating the overlap degree between the target in the first target tracking result and the target in the second target detection and tracking result, and if the overlap degree is greater than the overlap degree threshold, determining that the target in the first target tracking result and the target in the second target detection and tracking result are the same target; Determining whether the background of the target in the first target tracking result and the background of the target in the second target detection and tracking result are the same scene through a feature matching algorithm.

[0009] Further, for the above-mentioned method for tracking airborne targets by collaborative intelligent optics and radar, wherein before the step of adjusting the fusion detection and tracking result, it includes: Calculating the target center coordinates in the next frame image according to the target center coordinates in the current frame image in the fusion detection and tracking result: Compare whether the target center coordinates in the current frame image match the predicted target center coordinates in the next frame image.

[0010] Further, in the above-mentioned intelligent optical and radar collaborative air target tracking method, where the adjustment of the fusion detection and tracking results includes: Horizontally adjust the fusion detection and tracking results through the first formula, and the first formula is:

[0011] Where, represents the adjustment amount of the target in the horizontal direction, v is the flight speed of the target, T is the frame interval, is the horizontal angle between the target center points in the visible light image and the SAR image; Vertically adjust the fusion detection and tracking results through the second formula, and the second formula is:

[0012] Where, Δr represents the adjustment amount of the target in the vertical direction, r and r′ are the target vertical coordinates in the current frame and the next frame of the SAR image respectively, is the pitch angle between the visible light image and the SAR image.

[0013] Further, in the above-mentioned intelligent optical and radar collaborative air target tracking method, before the steps of performing target detection and tracking on the visible light image to obtain the first target detection and tracking result and performing target detection and tracking on the SAR image to obtain the second target detection and tracking result, it includes: Register the visible light image and the SAR image; Calculate the registration error between the target coordinates of the registered visible light image and the target coordinates in the SAR image. When the registration error is less than the preset error threshold, the registration is successful.

[0014] Further, in the above-mentioned intelligent optical and radar collaborative air target tracking method, where the second preset condition is: Meet one of the following preset conditions: The visible light tracking confidence is greater than the set value; The SAR tracking confidence is greater than the set value; The product of the visible light tracking confidence and the SAR tracking confidence is greater than the set value; The reliability of the first target detection and tracking is greater than the reliability of the second target detection and tracking; The product of the visible light tracking confidence and the SAR tracking confidence is greater than the product of the reliability of the first target detection and tracking and the reliability of the second target detection and tracking.

[0015] The embodiment of the present application also provides an air target tracking device that collaborates intelligent optics and radar, including: An acquisition module, configured to acquire visible light images and SAR images; A target detection and tracking module, configured to perform target detection and tracking on the visible light image to obtain a first target detection and tracking result, and perform target detection and tracking on the SAR image to obtain a second target detection and tracking result; A tracking mapping module, configured to perform a consistency determination on the first target detection and tracking result and the second target detection and tracking result. When the consistency determination result does not meet the first preset condition, perform target detection and tracking mapping on the visible light image and the SAR image respectively to obtain corresponding next-frame target tracking results; A tracking fusion module, configured to assign a tracking confidence level to the next-frame target tracking results. When the tracking confidence level does not meet the second preset condition, fuse the first target detection and tracking result and the second target detection and tracking result to obtain a fused detection and tracking result; An adjustment module, configured to adjust the fused detection and tracking result to obtain a final target tracking result.

[0016] The embodiment of the present application also provides a computer-readable storage medium, in which multiple instructions are stored, and the instructions are suitable for being loaded by a processor to execute any one of the above-mentioned air target tracking methods that collaborate intelligent optics and radar.

[0017] The embodiment of the present application also provides an electronic device, including a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used for the steps in any one of the above-mentioned air target tracking methods that collaborate intelligent optics and radar.

[0018] The air target tracking method, device, storage medium and electronic device provided by the present application. Through the collaborative processing of visible light images and SAR images, the present application can make full use of the advantages of both to solve the limitations that may exist in a single image source. Visible light images are suitable for clear weather and daytime environments, while SAR images have strong adaptability in bad weather, at night or under low visibility conditions, enhancing the target tracking ability of the system in various complex environments. The present application can dynamically adjust the output of the target tracking result according to the change of the target position between the current frame image and the next frame image. Through the calculation and adjustment of the coordinate change law, it can effectively predict the movement trajectory of the target and ensure the accurate tracking of the target in a dynamic scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The following will, in conjunction with the accompanying drawings, clearly and completely describe the technical solutions in the embodiments of the present application through a detailed description of the specific embodiments of the present application, and it will be obvious that the technical solutions and other beneficial effects of the present application will become apparent.

[0020] Figure 1 It is a flowchart of the method for tracking airborne targets by intelligent optical and radar collaboration provided in the embodiment of the present application.

[0021] Figure 2 It is another flowchart of the method for tracking airborne targets by intelligent optical and radar collaboration provided in the embodiment of the present application.

[0022] Figure 3 It is a flowchart of the training process of the deep neural network model based on multi-scale dilated convolution and pyramid module provided in the embodiment of the present application.

[0023] Figure 4 It is a flowchart of the consistency determination provided in the embodiment of the present application.

[0024] Figure 5 It is a flowchart of the tracking failure determination provided in the embodiment of the present application.

[0025] Figure 6 It is a flowchart of adjusting the fused detection and tracking results provided in the embodiment of the present application.

[0026] Figure 7 It is a schematic structural diagram of the device for tracking airborne targets by intelligent optical and radar collaboration provided in the embodiment of the present application.

[0027] Figure 8 It is a schematic structural diagram of the electronic device provided in the embodiment of the present application.

[0028] Figure 9 It is another schematic structural diagram of the electronic device provided in the embodiment of the present application. Specific Embodiments

[0029] The following will, in conjunction with the accompanying drawings in the embodiments of the present application, clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present application.

[0030] Traditional target tracking methods mainly rely on single image processing techniques, such as template matching, optical flow method, Kalman filtering, etc., to locate and track targets. These methods can achieve certain tracking effects in specific scenarios, but usually face some problems. First, the tracking method based on template matching depends on the stable features of the target in the image and does not have good adaptability to situations such as target rotation, scale change, or occlusion. Second, the optical flow method realizes tracking by estimating the pixel motion of the target, but this method is prone to failure in complex environments or low-contrast images. Although classical methods such as Kalman filtering have good real-time performance and accuracy in static backgrounds, they rely on the target motion model and are prone to misjudgment in complex dynamic environments and cannot effectively handle the simultaneous tracking of multiple targets. In addition, due to the low resolution of traditional radar image processing methods, it is difficult to provide target positioning with the same accuracy as visible light images and often can only rely on rough target position information, which limits the application of radar data in precise tracking. Although traditional target tracking technologies can achieve basic functions under some conditions, with the complexity of application scenarios and the continuous improvement of requirements for accuracy and robustness, traditional methods have exposed many deficiencies. First, the dependence on a single sensor makes the system prone to failure in harsh environments (such as low visibility, night, rain and snow weather, etc.), so it cannot guarantee continuous tracking capabilities in all weather conditions and all scenarios. Second, traditional image processing methods such as template matching and optical flow method are easily affected by factors such as occlusion and illumination change in dynamic changes and complex backgrounds, resulting in the failure of target recognition and tracking. Moreover, the independent processing of traditional radar and visible light images, lacking effective fusion, leads to deficiencies in image quality and target recognition accuracy.

[0031] To solve the above problems, embodiments of the present application provide an intelligent optical and radar collaborative air target tracking method, device, storage medium, and electronic device. An intelligent optical and radar collaborative air target tracking device provided by embodiments of the present application can be integrated in an electronic device, and the electronic device can be a device such as a terminal, a server, etc. Among them, the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a microprocessing box, or other devices, etc.

[0032] Please refer to Figure 1 And Figure 2 , Figure 1 is a flowchart of the intelligent optical and radar collaborative air target tracking method provided by embodiments of the present application. Figure 2 is another flowchart of the intelligent optical and radar collaborative air target tracking method provided by embodiments of the present application, which is applied to an electronic device. The intelligent optical and radar collaborative air target tracking method includes the following steps: S1, obtain visible light images and SAR images.

[0033] Among them, the SAR image is an image generated by a Synthetic Aperture Radar system.

[0034] Specifically, the embodiments of the present application use an intelligent optical camera and an SAR radar (SAR processing unit) to process visible light images and SAR images respectively. The intelligent optical camera is responsible for extracting target features from the visible light images, while the SAR radar processes the target information in the SAR images. The SAR image can provide additional target information in low visibility environments, especially at night or under adverse weather conditions. Therefore, by comparing and fusing the detection results of these two image sources, the robustness and accuracy of target tracking can be effectively improved.

[0035] For example, at night or under cloud cover, the SAR image can clearly reflect the position and shape of the target, while the visible light image may fail to detect the target due to low contrast or insufficient lighting. In this case, by comparing and fusing the information of the two, it can be ensured that the tracking of the target is not affected under different conditions.

[0036] Through this multi-source information fusion method, the stability of target tracking can be effectively improved, especially in complex scenarios and adverse conditions, ensuring the accuracy and real-time nature of the tracking results.

[0037] Before step S2, the following steps are also included: A1, register the visible light image and the SAR image; A2, calculate the registration error between the target coordinates of the registered visible light image and the target coordinates in the SAR image. When the registration error is less than the preset error threshold, the registration is successful.

[0038] Before performing target detection and tracking, it is necessary to ensure proper registration between the visible light image and the SAR image. Image registration is the process of aligning two images from different sources (in this solution, the visible light image and the SAR image). The purpose of registration is to ensure that the position of the target is consistent in the two images, so that subsequent target detection and tracking can be accurately matched.

[0039] In the registration process, common techniques include feature-based registration (such as SIFT, SURF) and region-based registration (such as the mutual information method). For SAR images and visible light images, considering their characteristics and different imaging methods, it is usually necessary to select an appropriate registration algorithm to ensure the registration accuracy.

[0040] As an example, assume that when performing image registration, the coordinates of a certain target in the SAR image are , and the coordinates of the target in the visible light image are . Through the registration algorithm, the two coordinates should be aligned as much as possible to form a consistent target position for subsequent tracking. The registration error is usually measured by the following formula:

[0041] where is the registration error, representing the sum of the squares of the differences in the target positions between the visible light image and the SAR image. The goal is to minimize this error to ensure that the positions of the targets in the registered images are as consistent as possible.

[0042] Through image registration, the spatial positions of the targets can be accurately aligned in the visible light image and the SAR image, ensuring that subsequent target detection and tracking algorithms can correctly identify and track the targets. The smaller the registration error, the higher the accuracy of target tracking.

[0043] Assume that the initial coordinates of the targets in the visible light image and the SAR image are respectively: The target coordinates in the visible light image:

[0044] The target coordinates in the SAR image:

[0045] Through image registration, the error of the target in the two images is:

[0046] The difference in the registered target coordinates is 5 pixels, and this error is acceptable in practical applications and can ensure the accuracy of subsequent target tracking.

[0047] S2. Perform target detection and tracking on the visible light image to obtain the first target detection and tracking result, and perform target detection and tracking on the SAR image to obtain the second target detection and tracking result.

[0048] In one embodiment, the process of performing target detection and tracking on the visible light image to obtain the first target detection and tracking result can be implemented by a neural network.

[0049] In one embodiment, before the step of performing target detection and tracking on the SAR image, it further includes: performing processing such as background suppression, low-pass filtering, bilateral filtering, mean filtering, and histogram equalization on the SAR image.

[0050] In one embodiment, the process of performing target detection and tracking on the SAR image to obtain the second target detection and tracking result specifically includes: inputting the SAR image into a trained deep neural network model based on multi-scale dilated convolution and pyramid module to obtain the second target detection and tracking result.

[0051] Among them, the deep neural network can be a deformable deep neural network.

[0052] Figure 3 The figure is a flowchart of the training process of the deep neural network model based on multi-scale dilated convolution and pyramid module provided by the embodiments of the present application. As Figure 3 shown, the training process of the deep neural network model based on multi-scale dilated convolution and pyramid module includes: B1. Obtain a training sample set; among them, the training sample set includes a first type of training sample set and a second type of training sample set. The first type of training sample set includes samples with non-scale-overlapping targets and the target centers not at the center positions of the 9 grids. The second type of training sample set includes samples with scale-overlapping targets and the target centers located at the middle positions of the 9 grids.

[0053] To achieve efficient object detection, two different training sample sets are required during the training process. The division of the first type of training sample set and the second type of training sample set helps to improve the robustness of object detection in different scenarios, especially when the relative positions and sizes of the targets change greatly.

[0054] B2. Input the sample data in the training sample set into the deep neural network model based on multi-scale dilated convolution and pyramid module to obtain predicted bounding boxes.

[0055] B3. Calculate the overlap degree between the predicted bounding boxes and the ground truth bounding boxes, calculate the loss function based on the overlap degree, and iteratively train the deep neural network model based on multi-scale dilated convolution and pyramid module through the loss function until the overlap degree meets the requirement of being greater than the preset overlap degree value, then stop the training.

[0056] In the neural network model, using dilated convolution can expand the receptive field without increasing the computational complexity, and the pyramid module enhances the model's perception ability of targets at different scales by processing multi-scale features. To quantify the matching degree between the object detection result and the actual object, the following formula can be used to calculate the overlap degree between the target bounding box and the ground truth bounding box, that is, the intersection over union (IoU):

[0057] Among them, A represents the predicted bounding box, B represents the ground truth bounding box, A∩B is the intersection area between the predicted bounding box and the ground truth bounding box, and A∪B is the union area between the predicted bounding box and the ground truth bounding box. This metric is used to evaluate the performance of the object detection model. The higher the IoU value, the higher the overlap degree between the detected target bounding box and the ground truth bounding box, indicating that the object detection ability of the model is stronger.

[0058] Specifically, when the IoU between the target detection result and the true position exceeds a certain threshold, it is considered that the target has been successfully detected. This threshold can be adjusted according to experimental requirements. For example, setting the IoU greater than 0.5 is considered a successful detection. If the IoU value is lower than the threshold, the model will continue to optimize the detection strategy or adjust the target tracking mechanism of the current image frame to ensure the accuracy of the final tracking result.

[0059] For example, when dealing with low-contrast targets in SAR images, dilated convolution helps expand the receptive field and effectively capture background information far from the target, while the pyramid module can extract features at multiple scales, enabling the network to maintain a high detection accuracy even when the target size changes. During training, the non-scale-overlapping samples in the first type of training sample set train the network's ability in the case of uneven target distribution, and the scale-overlapping samples in the second type of training sample set help the network better handle the interference and overlap between targets.

[0060] S3. Perform a consistency determination on the first target detection and tracking result and the second target detection and tracking result. When the consistency determination result does not meet the first preset condition, perform target detection and tracking mapping on the visible light image and the SAR image respectively to obtain the corresponding next-frame target tracking result.

[0061] Among them, the first preset condition is that the first target detection and tracking result and the second target detection and tracking result track the same target, and the target is in the same scene.

[0062] This condition is to ensure that when fusing multi-source information, only when the targets in the visible light image and the SAR image are consistent and the scenes match, can the tracking result be further processed and output. If the targets are inconsistent or the scenes do not match, the next fusion and adjustment process (steps S4 and S5) needs to be executed to ensure the accuracy and stability of the tracking.

[0063] It should be noted that when the consistency determination result meets the first preset condition, the first target detection and tracking result of the visible light image is used as the target tracking result. When the consistency determination result does not meet the first preset condition, steps S4 and S5 are continued to be executed.

[0064] Figure 4 This is the flowchart of the consistency determination provided by the embodiment of the present application. As Figure 4 shown, performing a consistency determination on the first target detection and tracking result and the second target detection and tracking result includes the following steps: S31. Calculate the overlap degree between the target in the first target tracking result and the target in the second target detection and tracking result. If the overlap degree is greater than the overlap degree threshold, determine that the target in the first target tracking result and the target in the second target detection and tracking result are the same target.

[0065] Tracking target consistency determination: To determine whether the targets in two image sources (visible light and SAR) are the same target, a matching strategy for spatial position and appearance features can be used. This can be done by comparing the center position, size, shape, etc. of the targets. Generally, the overlap degree of the target positions is used to judge whether they are the same target. Specifically, the overlap degree of the target in two different image sources can be calculated by the following formula to further judge whether the targets are consistent:

[0066] where A and B represent the target bounding boxes in the visible light image and the SAR image respectively, and represent the start and end positions of the target bounding box, is the overlap ratio between the target bounding boxes. By calculating the overlap ratio, it can be judged whether the two target bounding boxes are the same target. If the overlap ratio reaches a certain threshold (e.g., above 0.5), they are considered to be the same target.

[0067] S32. Determine whether the background of the target in the first target tracking result and the background of the target in the second target detection and tracking result are the same scene through a feature matching algorithm.

[0068] In addition to target consistency, scene consistency is also one of the preset conditions. Visible light images and SAR images may be affected by different environmental factors, such as lighting, weather, etc. These factors may cause differences in the performance of the same target in different images. Therefore, scene consistency is not only a matter of position matching, but also involves the background of the image and the relative relationship of the target. In practical applications, a feature matching algorithm (such as the SIFT or ORB algorithm) is used to assist in judging whether the scenes are consistent. For each frame of image, the feature descriptors of the target can be calculated and compared. If the target feature matching degrees in the two images are relatively high, it is considered that they are in the same scene.

[0069] As an example: Suppose in a certain scene, the target detection results of the SAR image and the visible light image show a high overlap and are located at the same position, and the appearance features of the target have a high matching degree in the two images. After the above overlap degree calculation and scene consistency determination, we can confirm that they are the same target and in the same scene, and then use the target detection result in the visible light image as the output. If the overlap degree between the two is lower than the set threshold, or the appearance features do not match, the target tracking fusion process continues to ensure that the final output target detection result has high reliability.

[0070] S4. Assign a tracking confidence level to the target tracking result of the next frame. When the tracking confidence level does not meet the second preset condition, fuse the first target detection and tracking result with the second target detection and tracking result to obtain a fused detection and tracking result.

[0071] The tracking confidence level includes a visible light tracking confidence level and a SAR tracking confidence level. The second preset condition is to meet one of the following preset conditions: (1) The visible light tracking confidence level is greater than the set value; (2) The SAR tracking confidence level is greater than the set value; (3) The product of the visible light tracking confidence level and the SAR tracking confidence level is greater than the set value; (4) The reliability of the first target detection and tracking is greater than the reliability of the second target detection and tracking; (5) The product of the visible light tracking confidence level and the SAR tracking confidence level is greater than the product of the reliability of the first target detection and tracking and the reliability of the second target detection and tracking.

[0072] When the tracking confidence level does not meet any of the above preset conditions (1)-(5), it means that the second preset condition is not met, and continue to execute step S5. When the tracking confidence level meets one of the above preset conditions (1)-(5), it means that the second preset condition is met. When the second preset condition is met, determine whether all the targets in the visible light image and the SAR image have failed to be tracked. If not all have failed to be tracked, repeat steps S3 - S5; if all have failed to be tracked, do not output the target tracking result of the current frame image.

[0073] Figure 5 This is the flowchart of the tracking failure determination provided by the embodiment of the present application. Among them, the determination of tracking failure is based on two key situations: 1. All targets have failed to be tracked: In the current frame image and the subsequent frame or the next frame image, all target tracking fails, resulting in no valid target detection results. This situation usually occurs when the target loses visibility, the target features are blurred, or the image quality is poor.

[0074] 2. There are no moving targets or only one target has been successfully tracked: If there are no moving targets in the detection and tracking results, or only one target has been successfully tracked, the system should also prompt tracking failure. This indicates that the tracking process cannot effectively identify the target, or fails to track a sufficient number of targets in a multi-target scenario, affecting subsequent target recognition and analysis.

[0075] The process of prompting tracking failure can be broken down into the following steps: 1. Stop output when all targets fail: When the tracking results of all targets in the current frame image cannot be determined, the target tracking results of the current frame will stop being output. This means that the targets in the current frame image are either lost or cannot be recognized, and no meaningful tracking information can be provided.

[0076] 2. Prompt failure when there are no moving targets or a single target: If there are no moving targets in the current frame image, or only one target is successfully tracked, this situation will be recognized as a tracking failure and feedback will be given in a timely manner. At this time, the target tracking results in the current frame image will also stop being output, and specific prompt information will be given.

[0077] 3. Specific information prompt: When a tracking failure occurs, error information will be prompted to the user. For example, prompt messages such as "Target tracking failed" or "No valid target detected" may be displayed so that the operator can timely understand the operating status and take corresponding measures.

[0078] S5. Adjust the fusion detection and tracking results to obtain the final target tracking results.

[0079] Figure 6 This is a flowchart for adjusting the fusion detection and tracking results provided by an embodiment of this application. It is determined whether the target between the current frame image and the next frame image can continue to be tracked according to the change law of the target center coordinates. When the speed of the target flight is known, the center coordinates of the target in the next frame image can be predicted according to the motion law of the target, and the detection and tracking results are updated according to this coordinate. If the change of the target center coordinates conforms to the preset coordinate change law, continue to track (that is, use the current fusion detection and tracking results as the target tracking results), otherwise use the adjusted detection and tracking results as the target tracking results.

[0080] In one embodiment, before the step of adjusting the fusion detection and tracking results, it includes: Calculate the center coordinates of the target in the next frame image according to the center coordinates of the target in the current frame image in the fusion detection and tracking results: Compare whether the center coordinates of the target in the current frame image match the predicted center coordinates of the target in the next frame image.

[0081] According to the physical law of target flight, the change of the center coordinates of the target between two frame images can be described by the following formula:

[0082]

[0083] Among them, c and r respectively represent the abscissa and ordinate of the target in the current frame image, v is the flight speed of the target, θ is the pitch angle of the target flight, T is the frame interval, and c′ and r′ respectively represent the abscissa and ordinate of the target in the next frame image.

[0084] The formula for the change in abscissa (c′) describes the displacement of the target in the horizontal direction in the image plane. The change in the abscissa of the target is closely related to the flight speed, pitch angle, and frame interval. The flight speed is proportional to the frame interval, and the pitch angle affects the displacement in the horizontal direction through the sine function. The formula for the change in ordinate (r′) describes the displacement of the target in the vertical direction in the image plane. The change in the ordinate of the target is related to the flight speed, pitch angle, and frame interval, where the flight speed is proportional to the frame interval, and the pitch angle affects the displacement in the vertical direction through the cosine function.

[0085] Based on the known speed and flight angle of the target, predict the exact position of the target in the next frame image. Through this prediction, precise target tracking can be performed between multiple frame images to ensure the consistency and accuracy of the tracking results.

[0086] In the fused detection and tracking results, first, the predicted position (c′, r′) of the target in the next frame image needs to be calculated using the above formula. Then, compare whether the target center coordinates (c, r) in the current frame image match the predicted coordinates (c′, r′) of the next frame image. If the target center coordinates in the current frame and the next frame image conform to the above coordinate change rule, continue to execute the subsequent tracking steps to ensure the continuity of tracking. Otherwise, it indicates that the motion trajectory of the target is abnormal, possibly due to changes in target speed, image quality problems, or tracking errors. At this time, the adjusted detection and tracking results will be used as the subsequent output results.

[0087] As an example: Suppose in the current frame image, the center coordinates of the target are c = 100 and r = 200, the flight speed v = 50 m / s, the pitch angle θ = 30°, and the frame interval T = 0.1 second. Then, the center coordinates of the target in the next frame image can be calculated according to the following steps: Calculate the change in abscissa:

[0088] Calculate the change in ordinate:

[0089] According to the calculation, the expected position of the target in the next frame image is c′ = 102.5 and r′ = 202.165. If the center coordinates of the target in the current frame image match this expected position, the system continues to track; if not, the tracking results are adjusted and output.

[0090] Through this method, it is possible to predict the position of the target in the next frame of image based on the flight speed and motion trajectory of the target, which provides a theoretical basis and algorithm support for the continuous tracking of aerial targets. By fusing visible light images and SAR images and combining the calculation of coordinate change rules, the accuracy and stability of target tracking can be effectively improved, especially in high-speed motion or complex scenarios.

[0091] In the method for tracking aerial targets through the cooperation of intelligent optics and radar, the adjustment of target tracking includes horizontal adjustment and pitch adjustment. These two adjustment methods are accurately calculated based on the flight speed of the target, the frame interval, and the relative position of the target in visible light images and SAR images. Through these adjustments, the accurate matching of the target between different image sources can be ensured, thereby improving the accuracy and stability of tracking.

[0092] Specifically, in step S5, the fusion detection and tracking results are adjusted, including: S51, horizontally adjust the fusion detection and tracking results through the first formula, and the first formula is:

[0093] where, represents the adjustment amount of the target in the horizontal direction, v is the flight speed of the target, T is the frame interval, is the horizontal angle between the center points of the target in the visible light image and the SAR image.

[0094] The purpose of horizontal adjustment is to correct the position deviation of the target in the visible light image and the SAR image. Assume that the position of the target in the current frame of image is (c, r), and the predicted position of the target in the next frame of image is (c′, r′), where c and c′ are the abscissas of the target in the current frame and the next frame of image respectively, and r and r′ are the ordinates of the target. When performing horizontal adjustment, the difference in the center point of the target between the two images is closely related to the flight speed of the target, the frame interval, and the horizontal angle between the images. In this way, the position that the target should be adjusted in the next frame of image can be calculated according to the motion characteristics of the target to ensure its correct position in the horizontal direction.

[0095] S52, vertically adjust the fusion detection and tracking results through the second formula, and the second formula is:

[0096] where, Δr represents the adjustment amount of the target in the vertical direction, r and r′ are the ordinates of the target in the current frame and the next frame of the SAR image respectively, is the pitch angle between the visible light image and the SAR image, It represents the vertical coordinate difference of the target in the SAR image, indicating the offset of the target center point on the y-axis between the current frame and the next frame image.

[0097] Pitch adjustment is to correct the deviation of the target in the longitudinal direction (i.e., the y-axis direction). This adjustment also depends on the flight speed of the target, the frame interval, and the relative pitch angle of the target in the visible light image and the SAR image. The purpose of this formula is to adjust the position of the target in the longitudinal direction according to its flight path. Through pitch adjustment, it can be ensured that the position of the target in the longitudinal direction is consistent in the two images, thus achieving accurate target tracking.

[0098] The goal of these two adjustment methods is to improve the tracking accuracy, especially when the target flight speed is relatively fast or the trajectory changes greatly. For example, in aviation or satellite remote sensing applications, the target flight speed is usually high, and the angular difference between the image sources (visible light image and SAR image) may be large. If effective horizontal and pitch adjustments are not made, it may lead to the accumulation of target position errors, thus affecting the subsequent tracking results.

[0099] Take a practical example. Suppose the flight speed of a certain target is v = 200 m / s, the frame interval T = 0.2 s, and the horizontal angle = 10°. The deviation of the target in the horizontal direction can be calculated through the horizontal adjustment formula:

[0100] Similarly, suppose the vertical coordinate difference of the target in the SAR image is = 55 m, the pitch angle = 20°. According to the pitch adjustment formula, the deviation correction of the target in the longitudinal direction can be calculated:

[0101] Through these adjustments, the position of the target in different images can be corrected according to its actual motion state, thus ensuring the accurate tracking of the target in each frame of the image.

[0102] Horizontal adjustment and pitch adjustment ensure the position consistency of the target in the visible light image and the SAR image by accurately calculating the flight speed of the target, the frame interval, and the angular difference between the images. This adjustment mechanism can effectively correct the errors caused by the change of the target flight state or the angular difference between the image sources, improving the accuracy and stability of target tracking, especially in complex air target tracking tasks.

[0103] The embodiments of this application significantly improve the accuracy, robustness, and adaptability of airborne target tracking through multi-modal data fusion, the application of deep learning models, precise coordinate adjustment, and a perfect failure detection mechanism. It can not only maintain efficient tracking in complex environments but also has strong dynamic adjustment and real-time feedback capabilities, adapting to different scenarios and task requirements, with broad application prospects. Specifically as follows: Multi-modal data fusion: By jointly processing visible light images and SAR images, the advantages of both can be fully utilized to address the limitations that may exist in a single image source. Visible light images are suitable for clear weather and daytime environments, while SAR images have strong adaptability in adverse weather, at night, or under low visibility conditions, enhancing the system's target tracking ability in various complex environments.

[0104] Precise target detection and tracking: Using a deep neural network (such as a neural network based on multi-scale dilated convolution and pyramid modules) for target detection and tracking can improve the accuracy of target detection. Especially in a multi-target environment, the system can more flexibly and precisely identify and track multiple targets. The deformable convolutional network (DCN) enhances the model's adaptability to targets of different shapes and scales, further improving the accuracy of target tracking.

[0105] Intelligent background suppression and noise removal: Through preprocessing steps (such as background suppression, low-pass filtering, bilateral filtering, etc.), the noise in the image is effectively removed, enhancing the distinguishability of target features. This not only enhances the stability of target detection but also avoids the interference of image noise on subsequent tracking and analysis.

[0106] Dynamic adjustment and precise output: The system can dynamically adjust the output of the target tracking result according to the change in the target position between the current frame image and the next frame image. Through the calculation and adjustment of the coordinate change law, the motion trajectory of the target can be effectively predicted to ensure accurate tracking of the target in a dynamic scene.

[0107] Tracking failure detection and feedback mechanism: The system designs a perfect tracking failure detection mechanism that can give a failure prompt in a timely manner when the target is lost or cannot be effectively tracked and stop the incorrect output. This not only reduces the risk of false detection and false output but also improves the reliability. For possible abnormal situations during target tracking (such as no target or only one target tracking successfully), clear failure feedback can also be given to avoid incorrect data processing.

[0108] Efficient target registration: The registration process between visible light images and SAR images ensures the consistency of the target in two different image sources, providing accurate input for subsequent target detection and tracking. Through precise registration algorithms, the error caused by image differences can be minimized to ensure accurate identification and tracking of the target from different perspectives.

[0109] According to the method described in the above embodiments, this embodiment will be further described from the perspective of an air target tracking device that collaborates intelligent optics and radar. The air target tracking device that collaborates intelligent optics and radar can be specifically implemented as an independent entity, or integrated in an electronic device, which can be a device such as a terminal, a server, etc. Among them, the terminal can include a tablet computer, a notebook computer, a personal computer (PC), a microprocessing box, or other devices, etc.

[0110] Please refer to Figure 7 , Figure 7 which specifically describes the air target tracking device that collaborates intelligent optics and radar provided by the embodiments of the present application, and is applied to an electronic device. The air target tracking device that collaborates intelligent optics and radar may include: An acquisition module, configured to acquire a visible light image and a SAR image; A target detection and tracking module, configured to perform target detection and tracking on the visible light image to obtain a first target detection and tracking result, and perform target detection and tracking on the SAR image to obtain a second target detection and tracking result; A tracking mapping module, configured to perform a consistency determination on the first target detection and tracking result and the second target detection and tracking result. When the consistency determination result does not meet the first preset condition, perform target detection and tracking mapping on the visible light image and the SAR image respectively to obtain corresponding next-frame target tracking results; A tracking fusion module, configured to assign a tracking confidence level to the next-frame target tracking results. When the tracking confidence level does not meet the second preset condition, fuse the first target detection and tracking result and the second target detection and tracking result to obtain a fused detection and tracking result; An adjustment module, configured to adjust the fused detection and tracking result to obtain a final target tracking result.

[0111] Specifically in implementation, the above-mentioned various modules and / or units can be implemented as independent entities, or can be combined arbitrarily to be implemented as the same or several entities. The specific implementation of the above-mentioned various modules and / or units can refer to the method embodiments described above, and the specific beneficial effects that can be achieved can also refer to the beneficial effects in the method embodiments described above, which will not be elaborated here.

[0112] In addition, the embodiments of the present application further provide an electronic device, which can be a device such as a computer, a tablet computer, etc. As Figure 8 shown, the electronic device 400 includes a processor 401 and a memory 402. Among them, the processor 401 is electrically connected to the memory 402.

[0113] The processor 401 is the control center of the electronic device 400, connecting various parts of the entire electronic device through various interfaces and circuits. By running or loading application programs stored in the memory 402 and invoking the data stored in the memory 402, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole.

[0114] In this embodiment, the processor 401 in the electronic device 400 will load the instructions corresponding to the processes of one or more application programs into the memory 402 according to the following steps, and the processor 401 will run the application programs stored in the memory 402 to achieve various functions: Obtain visible light images and SAR images; Perform target detection and tracking on the visible light image to obtain a first target detection and tracking result, and perform target detection and tracking on the SAR image to obtain a second target detection and tracking result; Perform a consistency determination on the first target detection and tracking result and the second target detection and tracking result. When the consistency determination result does not meet the first preset condition, perform target detection and tracking mapping on the visible light image and the SAR image respectively to obtain the corresponding next-frame target tracking result; Assign a tracking confidence level to the next-frame target tracking result. When the tracking confidence level does not meet the second preset condition, fuse the first target detection and tracking result and the second target detection and tracking result to obtain a fused detection and tracking result; Adjust the fused detection and tracking result to obtain the final target tracking result. This electronic device can implement the steps in any of the embodiments of the intelligent optical and radar collaborative air target tracking method provided in this application embodiment. Therefore, it can achieve the beneficial effects that any of the intelligent optical and radar collaborative air target tracking methods provided in this invention embodiment can achieve. For details, please refer to the previous embodiments and will not be elaborated here.

[0115] Figure 9 Shows the specific structural block diagram of the electronic device provided in this invention embodiment. This electronic device can be used to implement the intelligent optical and radar collaborative air target tracking method provided in the above embodiment. This electronic device 500 can be a device such as a terminal or a server. Among them, the terminal can include a tablet computer, a notebook computer, a personal computer (PC), a microprocessing box, or other devices, etc.

[0116] The RF circuit 510 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices. The RF circuit 510 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, subscriber identity module (SIM) cards, memories, and so on. The RF circuit 510 can communicate with various networks such as the Internet, intranets, wireless networks or communicate with other devices through a wireless network. The above-mentioned wireless networks may include cellular phone networks, wireless local area networks or metropolitan area networks. The above-mentioned wireless networks can use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, and may even include those protocols that have not yet been developed currently.

[0117] The memory 520 can be used to store software programs and modules, such as the corresponding program instructions / modules in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, that is, realizes functions such as taking pictures with the front camera, processing the captured images, and switching the display colors of the display content on the display screen. The memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 520 may further include a memory remotely disposed relative to the processor 580, and these remote memories can be connected to the electronic device 500 through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0118] The input unit 530 can be used to receive input digital or character information, and generate a keyboard and a mouse related to user settings and function controls. The display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, and these graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 540 may include a display panel 541. Optionally, the display panel 541 can be configured in the form of an LCD (Liquid Crystal Display) or an OLED (Organic Light-Emitting Diode).

[0119] The audio circuit 560, speaker 561, and microphone 562 can provide an audio interface between the user and the electronic device 500. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, and the speaker 561 converts it into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and then converted into audio data. After the audio data is output to the processor 580 for processing, it is sent to another terminal, for example, through the RF circuit 510, or the audio data is output to the memory 520 for further processing. The audio circuit 560 may further include an earphone jack to provide communication between the peripheral earphone and the electronic device 500.

[0120] The electronic device 500 can help the user receive requests, send information, etc. through the transmission module 570 (such as a Wi-Fi module), and it provides the user with wireless broadband Internet access. Although the transmission module 570 is shown in the figure, it can be understood that it does not belong to the essential components of the electronic device 500 and can be completely omitted within the scope of not changing the essence of the invention according to needs.

[0121] The processor 580 is the control center of the electronic device 500, connecting various parts of the entire mobile phone through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 520, and by invoking the data stored in the memory 520, it executes various functions of the electronic device 500 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580 either.

[0122] The electronic device 500 also includes a power supply 590 (such as a battery) for supplying power to each component. In some embodiments, the power supply can be logically connected to the processor 580 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 590 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0123] Although not shown, the electronic device 500 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be elaborated here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, where one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: Obtain visible light images and SAR images; Perform target detection and tracking on the visible light image to obtain a first target detection and tracking result, and perform target detection and tracking on the SAR image to obtain a second target detection and tracking result; Perform a consistency determination on the first target detection and tracking result and the second target detection and tracking result. When the consistency determination result does not meet the first preset condition, perform target detection and tracking mapping on the visible light image and the SAR image respectively to obtain corresponding next-frame target tracking results; Assign a tracking confidence level to the next-frame target tracking result. When the tracking confidence level does not meet the second preset condition, fuse the first target detection and tracking result and the second target detection and tracking result to obtain a fused detection and tracking result; Adjust the fusion detection and tracking results to obtain the final target tracking results. In specific implementation, each of the above modules can be implemented as an independent entity, or can be combined arbitrarily and implemented as the same or several entities. For the specific implementation of each of the above modules, reference can be made to the foregoing method embodiments, which will not be elaborated herein.

[0124] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling relevant hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. For this purpose, an embodiment of the present invention provides a storage medium in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps of any one of the embodiments of the intelligent optical and radar collaborative air target tracking method provided by the embodiments of the present invention.

[0125] Among them, the computer-readable storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc.

[0126] Since the instructions stored in the storage medium can execute the steps of any one of the embodiments of the intelligent optical and radar collaborative air target tracking method provided by the embodiments of the present invention, the beneficial effects that can be achieved by any of the intelligent optical and radar collaborative air target tracking methods provided by the embodiments of the present invention can be realized. For details, see the foregoing embodiments, which will not be elaborated herein.

[0127] The above has introduced in detail an intelligent optical and radar collaborative air target tracking method, device, storage medium, and electronic device provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for tracking aerial targets by intelligent optics and radar collaboration, characterized in that: The method comprises: Acquire visible light images and SAR images; Performing target detection and tracking on the visible light image to obtain a first target detection and tracking result, and performing target detection and tracking on the SAR image to obtain a second target detection and tracking result; Performing consistency determination on the first target detection and tracking result and the second target detection and tracking result, and when the consistency determination result does not meet a first preset condition, performing target detection and tracking mapping on the visible light image and the SAR image respectively to obtain a corresponding next frame target tracking result; Assigning a tracking confidence to the next frame target tracking result, and when the tracking confidence does not meet a second preset condition, fusing the first target detection and tracking result with the second target detection and tracking result to obtain a fused detection and tracking result; The fusion detection and tracking result is adjusted to obtain a final target tracking result.

2. The method for tracking aerial targets by intelligent optics and radar collaboration according to claim 1, characterized in that: The performing target detection and tracking on the SAR image to obtain a second target detection and tracking result includes: The SAR image is input into a trained deep neural network model based on multi-scale hole convolution and pyramid module to obtain a second target detection and tracking result.

3. The method for tracking aerial targets by intelligent optics and radar collaboration according to claim 2, characterized in that: The training process of the deep neural network model based on multi-scale dilated convolution and pyramid module includes: Acquire a training sample set; wherein the training sample set includes a first type of training sample set and a second type of training sample set, the first type of training sample set includes samples in which the targets have no scale overlap and the target center is not at the center of the 9-grid, and the second type of training sample set includes samples in which the targets have scale overlap and the target center is located in the middle of the 9-grid; Inputting the sample data in the training sample set into a deep neural network model based on multi-scale hole convolution and pyramid module to obtain a prediction box; The overlap between the predicted box and the real box is calculated, and a loss function is calculated based on the overlap. The deep neural network model based on the multi-scale hole convolution and pyramid module is iteratively trained by the loss function until the overlap satisfies a preset overlap value and stops the training.

4. The method for tracking aerial targets by intelligent optics and radar collaboration according to claim 1, characterized in that: The first preset condition is that the first target detection and tracking result and the second target detection and tracking result track the same target, and the targets are in the same scene; The performing consistency determination on the first target detection and tracking result and the second target detection and tracking result includes: Calculating the degree of overlap between the target in the first target tracking result and the target in the second target detection and tracking result, and if the degree of overlap is greater than an overlap threshold, determining that the target in the first target tracking result and the target in the second target detection and tracking result are the same target; A feature matching algorithm is used to determine whether the background of the target in the first target tracking result and the background of the target in the second target detection and tracking result are the same scene.

5. The method for tracking aerial targets by intelligent optics and radar collaboration according to claim 1, characterized in that: Before the step of adjusting the fusion detection and tracking result, the method includes: According to the target center coordinates in the current frame image in the fusion detection and tracking results, calculate the target center coordinates in the next frame image: Compare the target center coordinates in the current frame image with the predicted target center coordinates in the next frame image to see if they match.

6. The method for tracking aerial targets by intelligent optics and radar collaboration according to claim 5, characterized in that: The adjusting the fusion detection and tracking result includes: The fusion detection and tracking result is horizontally adjusted by a first formula, and the first formula is: in, It represents the adjustment amount of the target in the horizontal direction, v is the flight speed of the target, T is the frame interval, is the horizontal angle between the target center points in the visible light image and the SAR image; The fusion detection and tracking result is pitch-adjusted by a second formula, and the second formula is: Among them, Δr represents the adjustment amount of the target in the vertical direction, r and r′ are the vertical coordinates of the target in the current frame and the next frame of the SAR image, respectively. is the pitch angle between the visible light image and the SAR image.

7. The method for tracking aerial targets by intelligent optics and radar collaboration according to claim 1, characterized in that: Before the steps of performing target detection and tracking on the visible light image to obtain a first target detection and tracking result and performing target detection and tracking on the SAR image to obtain a second target detection and tracking result, the method includes: Registering the visible light image with the SAR image; The registration error between the target coordinates of the registered visible light image and the target coordinates in the SAR image is calculated, and when the registration error is less than a preset error threshold, the registration is successful.

8. The method for tracking aerial targets by intelligent optics and radar collaboration according to claim 1, characterized in that: The second preset condition is: One of the following prerequisites is met: The visible light tracking confidence is greater than the set value; SAR tracking confidence is greater than the set value; The visible light tracking confidence multiplied by the SAR tracking confidence is greater than the set value; The reliability of the first target detection and tracking is greater than the reliability of the second target detection and tracking; The product of the visible light tracking confidence and the SAR tracking confidence is greater than the product of the reliability of the first target detection and tracking and the reliability of the second target detection and tracking.

9. An intelligent optical and radar coordinated aerial target tracking device, characterized in that: include: An acquisition module, used for acquiring visible light images and SAR images; A target detection and tracking module is used to perform target detection and tracking on the visible light image to obtain a first target detection and tracking result, and to perform target detection and tracking on the SAR image to obtain a second target detection and tracking result; A tracking and mapping module, configured to perform consistency determination on the first target detection and tracking result and the second target detection and tracking result, and when the consistency determination result does not meet a first preset condition, perform target detection, tracking and mapping on the visible light image and the SAR image respectively to obtain a corresponding next frame target tracking result; A tracking fusion module, configured to assign a tracking confidence to the target tracking result of the next frame, and when the tracking confidence does not meet a second preset condition, fuse the first target detection and tracking result with the second target detection and tracking result to obtain a fused detection and tracking result; The adjustment module is used to adjust the fusion detection and tracking result to obtain the final target tracking result.

10. An electronic device, characterized in that: It includes a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to execute the steps in the intelligent optical and radar collaborative aerial target tracking method described in any one of claims 1 to 8.

Images