Continuous video frame SAR vehicle target tracking method based on deep learning

Abstract

The invention discloses a continuous video frame SAR vehicle target tracking method based on deep learning, and the method comprises the steps: firstly, based on a YOLOv8 frame, embedding an SE attention module behind a CSPLayer of a backbone network, replacing a CIoU loss function with an EIoU loss function, and obtaining an SAR target perception enhancement network; secondly, integrating the bidirectional feature pyramid network to a detection head level of an SAR target perception enhancement network, and then inputting the SAR image into the SAR target perception enhancement network integrated with the bidirectional feature pyramid network to obtain a detection frame of all vehicle targets in the SAR image; then, the ByteTrack multi-target tracking framework is optimized; and finally, inputting the detection frames of all vehicle targets in the SAR image into the optimized ByteTrack multi-target tracking framework to obtain a tracking result. According to the method, a cooperative solution of'feature enhancement-detection optimization-tracking robustness' is provided for SAR image target tracking, and the continuous tracking capability of a vehicle target in a complex environment is remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing and deep learning, and in particular to a deep learning-based method for tracking SAR vehicles in continuous video frames. Background Technology

[0002] Synthetic Aperture Radar (SAR), as an active microwave imaging sensor, possesses all-weather, all-time operational capabilities and plays a crucial role in military reconnaissance, disaster monitoring, and other fields. Especially in complex battlefield environments, SAR vehicle target tracking technology can penetrate clouds and smoke to achieve covert reconnaissance, providing critical information support for situational awareness. However, SAR images inherently suffer from strong speckle noise, blurred target outlines, and severe background clutter interference, leading to problems such as target loss and trajectory breakage in traditional tracking methods based on feature matching or motion models.

[0003] In recent years, deep learning technology has brought new breakthroughs to SAR image interpretation. Among them, YOLOv8, as a representative of single-stage detection algorithms, significantly improves the recognition accuracy of dense small targets while maintaining high detection speed through adaptive feature extraction modules and dynamic label allocation strategies. Meanwhile, the ByteTrack algorithm innovatively utilizes the correlation between detection box similarity and motion trajectory to construct a data association matrix, effectively solving the multi-target tracking problem in complex scenes. However, when directly transferring models trained in the optical domain to the SAR domain, the feature distribution shift caused by differences in imaging mechanisms leads to a sharp decline in the generalization performance of the tracking algorithm.

[0004] This paper proposes a novel solution for vehicle target tracking in SAR images, integrating the YOLOv8 detection framework with the ByteTrack association mechanism. By constructing an SE attention-guided feature enhancement network, the scattering characteristic response of the SAR target is dynamically calibrated along the channel dimension, strengthening the channel weight difference between the vehicle target and background clutter. Simultaneously, a motion trajectory predictor is constructed using Doppler parameters, leveraging prior motion information from the SAR platform to establish a target motion state estimation model. This method adaptively focuses on the target's strong scattering region through the channel attention mechanism of the SE module, suppressing the impact of speckle noise and sidelobe interference on feature extraction. Furthermore, it optimizes the target trajectory prediction accuracy based on the geometric characteristics of SAR imaging, ultimately achieving highly robust continuous tracking of vehicle targets in complex electromagnetic environments. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies mentioned in the background art by providing a deep learning-based continuous video frame SAR vehicle target tracking method.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A deep learning-based continuous video frame SAR vehicle target tracking method includes the following steps:

[0008] Step 1) Based on the YOLOv8 framework, an SE attention module is embedded after the CSPrayer in the backbone network to obtain a YOLOv8 framework with an embedded SE attention module; the SE attention module is used to perform channel recalibration to enhance the scattering features of vehicle targets in SAR images.

[0009] Step 2) Replace the CIoU loss function of the YOLOv8 framework with the embedded SE attention module with the EIoU loss function to obtain the SAR target perception enhancement network; the EIoU loss function is used to optimize the bounding box regression of vehicle targets in SAR images to reduce localization errors in complex backgrounds;

[0010] Step 3) Integrate the bidirectional feature pyramid network into the detection head level of the SAR target perception enhancement network. The bidirectional feature pyramid network is used to improve the detection robustness of dense vehicle targets through cross-level feature interaction.

[0011] Step 4) Input the SAR image into the SAR target perception enhancement network that integrates a bidirectional feature pyramid network to obtain the detection boxes of all vehicle targets in the SAR image;

[0012] Step 5) Based on the trajectory-feature dual-drive association mechanism, the dynamic weighted matching of Kalman filter predicted trajectory and ReID appearance features is used to solve the ID switching problem caused by occlusion in SAR image, and the ByteTrack multi-target tracking framework is optimized to obtain the optimized ByteTrack multi-target tracking framework.

[0013] Step 6) Input the detection bounding boxes of all vehicle targets in the SAR image into the optimized ByteTrack multi-target tracking framework to obtain the tracking results.

[0014] As a further optimization of the deep learning-based continuous video frame SAR vehicle target tracking method of the present invention, in step 1), when embedding the SE attention module after the CSPlayer of the backbone network, the importance of channel features is captured by global average pooling, and the channel weights are dynamically adjusted by using the Sigmoid activation function to enhance the high-frequency feature response of the vehicle scattering point and suppress background clutter interference.

[0015] As a further optimization of the deep learning-based continuous video frame SAR vehicle target tracking method of the present invention, in step 3), when integrating the bidirectional feature pyramid network into the detection head level of the SAR target perception enhancement network, the features of layers P3-P5 are fused through bidirectional cross-scale connections, and the adaptive weighting of features of different resolutions is achieved by using learnable weights to enhance the semantic information transmission of dense small targets.

[0016] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0017] This invention utilizes deep learning methods to construct a SAR target perception enhancement network and design a multi-scale tracking fusion architecture. A comprehensive tracking performance verification system is also established. Experimental results show that an 89.1% MOTA (Moment of Response Time) index is achieved. This method provides a collaborative solution for SAR image target tracking, encompassing feature enhancement, detection optimization, and tracking robustness, significantly improving the continuous tracking capability of vehicle targets in complex environments. Attached Figure Description

[0018] Figure 1 This is a comparative illustration of vehicle targets in a SAR scene from two different angles.

[0019] Figure 2 Diagram of SAR target perception enhancement network architecture;

[0020] Figure 3 This is a schematic diagram of the EIoU loss function.

[0021] Figure 4 This is a framework diagram of a bidirectional feature pyramid network;

[0022] Figure 5 This is a flowchart of the experiment. Detailed Implementation

[0023] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings:

[0024] This invention can be implemented in many different forms and should not be considered limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully express the scope of the invention to those skilled in the art. In the drawings, components are enlarged for clarity.

[0025] This invention discloses a deep learning-based continuous video frame SAR vehicle target tracking method, comprising the following steps:

[0026] Step 1) Based on the YOLOv8 framework, an SE attention module is embedded after the CSPrayer in the backbone network to obtain a YOLOv8 framework with an embedded SE attention module; the SE attention module is used to perform channel recalibration to enhance the scattering features of vehicle targets in SAR images.

[0027] Step 2) Replace the CIoU loss function of the YOLOv8 framework with the embedded SE attention module with the EIoU loss function to obtain the SAR target perception enhancement network; the EIoU loss function is used to optimize the bounding box regression of vehicle targets in SAR images to reduce localization errors in complex backgrounds;

[0028] Step 3) Integrate the bidirectional feature pyramid network into the detection head level of the SAR target perception enhancement network. The bidirectional feature pyramid network is used to improve the detection robustness of dense vehicle targets through cross-level feature interaction.

[0029] Step 4) Input the SAR image into the SAR target perception enhancement network that integrates a bidirectional feature pyramid network to obtain the detection boxes of all vehicle targets in the SAR image;

[0030] Step 5) Based on the trajectory-feature dual-drive association mechanism, the dynamic weighted matching of Kalman filter predicted trajectory and ReID appearance features is used to solve the ID switching problem caused by occlusion in SAR image, and the ByteTrack multi-target tracking framework is optimized to obtain the optimized ByteTrack multi-target tracking framework.

[0031] Step 6) Input the detection bounding boxes of all vehicle targets in the SAR image into the optimized ByteTrack multi-target tracking framework to obtain the tracking results.

[0032] When embedding the SE attention module after the CSPlayer in the backbone network, the importance of channel features is captured by global average pooling, and the channel weights are dynamically adjusted by using the Sigmoid activation function to enhance the high-frequency feature response of vehicle scattering points and suppress background clutter interference.

[0033] When a bidirectional feature pyramid network is integrated into the detection head layer of a SAR target perception enhancement network, features from layers P3 to P5 are fused through bidirectional cross-scale connections. Learnable weights are used to achieve adaptive weighting of features at different resolutions, thereby enhancing the transmission of semantic information for dense small targets.

[0034] Figure 1 This image shows a comparison of vehicle targets in a SAR scene containing numerous vehicles from two different angles. As can be seen from the image, the characteristics of vehicle targets in the SAR image change significantly under different angles, making them difficult to distinguish with the naked eye. Therefore, it is necessary to improve our target perception capabilities and multi-scale fusion capabilities.

[0035] The fusion of the SE attention mechanism effectively addresses challenges such as strong speckle noise, weak target features, and multi-scale variations in SAR images. The Bidirectional Feature Pyramid Network (BiFPN) enhances the representation of vehicles at different scales through bidirectional multi-scale feature fusion, while the SE module dynamically strengthens key target feature channels and suppresses noise interference through channel attention. The combination of these two mechanisms forms a dual optimization mechanism of "hierarchy + channel," significantly improving detection accuracy and robustness. Experiments show that this scheme is highly effective in suppressing background false alarms, enhancing small target features, and adapting to complex scenes, providing an efficient feature enhancement and fusion paradigm for SAR target detection. This invention replaces the original CIoU with EIoU. EIoU, through geometric decoupling loss design and dynamic adaptive optimization, solves the localization ambiguity problem caused by low resolution, multi-scale, and strong noise in SAR vehicle detection, providing high-precision, low-noise detection input for subsequent tracking tasks, forming a closed-loop optimization link of "accurate detection - stable tracking." Figure 2 This is a network architecture diagram. Figure 3 This is a schematic diagram of the EIoU loss function.

[0036] Figure 4 This is for the Bidirectional Feature Pyramid Network (BiFPN).

[0037] A two-branch comparative experiment was designed—the experimental group used a YOLOv8 detector enhanced with SE-BiFPN, while the control group retained the original network structure. Tracking performance was quantified using metrics such as MOTA (Multi-Target Tracking Accuracy) and IDF1 (Identity Consistency). A dynamic scene stress testing strategy was proposed to verify the model's generalization ability under extreme SAR scenarios such as low resolution and strong noise.

[0038] To verify the effectiveness of this invention, two comparative experiments were conducted. The flowcharts of the experiments are as follows: Figure 5 As shown. The specific steps can be described as follows: In the comparison of the detection module, three groups of models are set up: the original YOLOv8 (Group A), the SE-enhanced YOLOv8 (Group B), and the SE-BiFPN-YOLOv8+EIoU (Group C). Figure 5 This is a flowchart of the experiment.

[0039] The accuracy comparison of the test results of the three networks (Group A, Group B, and Group C) on the input SAR images is as follows:

[0040] Group P R mAP0.5 A 0.865 0.847 0.914 B 0.883 0.892 0.926 C 0.894 0.917 0.965

[0041] The results of the proposed methods for SAR vehicle target tracking are as follows:

[0042] Group MOTA IDs A 0.714 0.51 B 0.762 0.77 C 0.891 0.90

[0043] The results show that the test results output by the proposed network can achieve a MOTA index of 89.1%, which greatly improves the accuracy of vehicle tracking in SAR images and achieves a high level of tracking.

[0044] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0045] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for tracking SAR vehicles based on deep learning in continuous video frames, characterized in that, Includes the following steps: Step 1) Based on the YOLOv8 framework, an SE attention module is embedded after the CSPrayer in the backbone network to obtain a YOLOv8 framework with an embedded SE attention module; the SE attention module is used to perform channel recalibration to enhance the scattering features of vehicle targets in SAR images. Step 2) Replace the CIoU loss function of the YOLOv8 framework with the embedded SE attention module with the EIoU loss function to obtain the SAR target perception enhancement network; the EIoU loss function is used to optimize the bounding box regression of vehicle targets in SAR images to reduce localization errors in complex backgrounds; Step 3) Integrate the bidirectional feature pyramid network into the detection head level of the SAR target perception enhancement network. The bidirectional feature pyramid network is used to improve the detection robustness of dense vehicle targets through cross-level feature interaction. Step 4) Input the SAR image into the SAR target perception enhancement network that integrates a bidirectional feature pyramid network to obtain the detection boxes of all vehicle targets in the SAR image; Step 5) Based on the trajectory-feature dual-drive association mechanism, the dynamic weighted matching of Kalman filter predicted trajectory and ReID appearance features is used to solve the ID switching problem caused by occlusion in SAR image, and the ByteTrack multi-target tracking framework is optimized to obtain the optimized ByteTrack multi-target tracking framework. Step 6) Input the detection bounding boxes of all vehicle targets in the SAR image into the optimized ByteTrack multi-target tracking framework to obtain the tracking results.

2. The deep learning-based continuous video frame SAR vehicle target tracking method according to claim 1, characterized in that, Step 1) When embedding the SE attention module after the CSPlayer in the backbone network, the importance of channel features is captured by global average pooling, and the channel weights are dynamically adjusted by using the Sigmoid activation function to enhance the high-frequency feature response of the vehicle scattering point and suppress background clutter interference.

3. The deep learning-based continuous video frame SAR vehicle target tracking method according to claim 1, characterized in that, Step 3) When integrating the bidirectional feature pyramid network into the detection head layer of the SAR target perception enhancement network, the features of layers P3-P5 are fused through bidirectional cross-scale connections, and learnable weights are used to achieve adaptive weighting of features at different resolutions, thereby enhancing the semantic information transmission of dense small targets.

Images