A marine radar image target tracking method based on deep learning

Abstract

The present application relates to the technical field of visual detection and method, and particularly relates to a marine radar image target tracking method based on deep learning, comprising the following steps: collecting and analyzing echo data, rendering echo information into marine radar images and manually labeling to build a labeled training dataset; optimizing a basic detection model and establishing a target detection model; performing data enhancement processing on the marine radar images, training the target detection model and outputting a weight file; outputting a multi-scale feature map, obtaining a target detection frame based on threshold filtering and non-maximum suppression; inputting the target detection frame into a DeepSORT tracker, extracting appearance features and motion information of the target, and continuously tracking and maintaining identity consistency of multiple targets in the image sequence. The present application realizes accurate, stable and real-time detection and tracking of ship targets in marine radar images by optimizing the structure of the detection model and the tracking algorithm, and the model is lightweight and convenient for actual deployment.

Description

Technical Field

[0001] This invention relates to the field of visual detection and methods, specifically to a deep learning-based method for tracking targets in marine radar images. Background Technology

[0002] With the development of intelligent sensing and deep learning technologies, image-based target detection and tracking technologies have been widely used in various fields such as security monitoring, intelligent transportation, maritime supervision, and military reconnaissance. Especially in marine environments, traditional optical images have limited imaging capabilities under adverse conditions such as nighttime, fog, and rain, while radar images, due to their all-day and all-weather imaging capabilities, have become an important data source for ship target detection and monitoring.

[0003] Marine radar images are characterized by high resolution and strong anti-interference capabilities. However, due to their imaging method differing from optical images, they suffer from problems such as blurred textures, irregular target shapes, and severe clutter interference, making it difficult for traditional texture- or template-based detection methods to achieve ideal results in marine radar images. In recent years, the widespread success of convolutional neural networks (CNNs) in visual tasks has provided new ideas for target detection in radar images. However, at present, target detection and tracking in radar images based on deep learning are still in the exploratory stage.

[0004] Therefore, there is an urgent need for a joint target detection and multi-target tracking method that takes into account the characteristics of radar images, which can achieve stable and accurate continuous tracking while ensuring detection accuracy, and has the capability for engineering deployment. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a deep learning-based method for tracking targets in marine radar images, specifically including the following steps:

[0006] S1: Collect echo data based on marine radar equipment, parse the echo data and obtain echo information based on communication protocol, render the echo information into a two-dimensional image using OpenGL and output marine radar image, and manually annotate the marine radar image to construct a labeled training dataset.

[0007] S2: Establish a basic detection model based on lightweight YOLOv8, optimize the basic detection model according to the morphological characteristics of the ship target, and establish a target detection model.

[0008] S3: Perform data augmentation processing on the marine radar image, train the target detection model based on the labeled training dataset, and output a weight file.

[0009] S4: Perform forward inference based on the weights with the best performance during training, and output a multi-scale feature map. The multi-scale includes a class prediction branch and a bounding box regression branch. Obtain the target detection box based on threshold filtering and non-maximum suppression.

[0010] S5: Input the target detection box into the DeepSORT tracker, extract the appearance features and motion information of the target in the target detection box, perform data association based on Kalman filtering and Hungarian algorithm, and continuously track and maintain identity consistency of multiple targets in the image sequence.

[0011] Furthermore, constructing a labeled training dataset includes: collecting echo data based on marine radar equipment; parsing the echo data and obtaining echo information based on a communication protocol; rendering the echo information into a two-dimensional image using OpenGL and outputting a marine radar image; saving the marine radar image as a static image file; manually labeling the ship targets in the marine radar image using a labeling tool; during the manual labeling process, using AIS ship position data to assist in confirming the ship's true position and boundary contour, and constructing a training dataset with YOLO format labels.

[0012] Furthermore, the basic detection model is optimized by replacing the CBS module in YOLOv8 with ADown downsampling, introducing parameterless pooling operations in ADwon, and introducing lightweight convolutions in sub-branches using a channel partitioning strategy.

[0013] Furthermore, optimizing the basic detection model also includes replacing the CIoU loss function in YOLOv8 with the MPDIoU loss function and introducing the Euclidean distance between the diagonal corners of the predicted bounding box and the ground truth bounding box as a perceptual constraint.

[0014] Furthermore, the data augmentation process includes Mosaic puzzle enhancement and adaptive anchor box calculation.

[0015] Furthermore, the target detection model is trained, with a total of 150 training rounds. In each training iteration, the loss function is calculated and the network parameters are updated. The model weight with the smallest loss function value is selected as the optimal inference weight of the target detection model.

[0016] Furthermore, the multi-scale feature map includes three scales: 80×80, 40×40, and 20×20.

[0017] Furthermore, in the nonmaximum suppression operation, the confidence threshold is set to 0.75.

[0018] Furthermore, the DeepSORT tracker estimates the target state based on a Kalman filter, performs data association based on a Hungarian algorithm, initializes the ID of newly appearing targets, and cancels the ID of targets that have not been matched for a long time, thereby realizing the entry, exit, and continuous tracking management of targets.

[0019] Compared with the prior art, the present invention has the following beneficial effects: This invention constructs a complete perception system that is efficient, accurate, and easy to deploy. It improves detection performance through targeted optimization of the YOLOv8 model and replaces the traditional convolutional structure with a lightweight downsampling module, reducing model parameters and computational complexity while enhancing feature fusion and context modeling capabilities. This invention uses the MPDIoU loss function instead of CIoU, significantly improving the positioning accuracy of small-sized ship targets in complex radar clutter backgrounds. This invention introduces the DeepSORT algorithm to achieve stable tracking, and effectively solves the problems of target occlusion and identity switching through Kalman filtering prediction and appearance feature matching, ensuring the continuity and uniqueness of multi-target trajectories over time. The entire system of this invention is lightweight, possesses strong anti-interference capabilities and good adaptability, and can meet the high real-time requirements of scenarios such as maritime supervision. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a deep learning-based target tracking method for marine radar images according to the present invention.

[0022] Figure 2 This is a radar echo image of a deep learning-based target tracking method for marine radar images according to the present invention.

[0023] Figure 3 This is a structural diagram of the improved YOLOv8 algorithm for a deep learning-based target tracking method for marine radar images according to the present invention.

[0024] Figure 4 This is an ADwon structure diagram of a deep learning-based target tracking method for marine radar images according to the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] This invention discloses a deep learning-based target tracking method for marine radar images, which mainly includes the following steps: S1: Collect echo data based on marine radar equipment, parse the echo data and obtain echo information based on communication protocol, render the echo information into a two-dimensional image using OpenGL and output marine radar image, and manually annotate the marine radar image to construct a labeled training dataset.

[0028] S2: Establish a basic detection model based on lightweight YOLOv8, optimize the basic detection model according to the morphological characteristics of the ship target, and establish a target detection model.

[0029] S3: Perform data augmentation processing on the marine radar image, train the target detection model based on the labeled training dataset, and output a weight file.

[0030] S4: Perform forward inference based on the weights with the best performance during training, and output a multi-scale feature map. The multi-scale includes a class prediction branch and a bounding box regression branch. Obtain the target detection box based on threshold filtering and non-maximum suppression.

[0031] S5: Input the target detection box into the DeepSORT tracker, extract the appearance features and motion information of the target in the target detection box, perform data association based on Kalman filtering and Hungarian algorithm, and continuously track and maintain identity consistency of multiple targets in the image sequence.

[0032] In a preferred embodiment of this application, marine radar is deployed at the dock for real-world data acquisition. During the acquisition process, raw radar echo data is received via UDP protocol and decoded according to a specified format. Subsequently, OpenGL is used to render the radar echoes in real time, and the generated echo images are saved locally to construct the original dataset. Considering the large amount of clutter information in radar echoes, AIS information is introduced during dataset construction for auxiliary fusion to improve the accuracy and reliability of data annotation. In the data annotation stage, the LabelImg tool is used to manually annotate ship targets in the images, and the annotation results are exported in YOLO format. The dataset is then divided into training, validation, and test sets in an 8:1:1 ratio to facilitate the subsequent training of deep learning detection models.

[0033] As a preferred implementation of this application, the YOLOv8 algorithm consists of three main components: a backbone network for feature extraction, a neck structure for multi-scale feature fusion, and a head for generating the final bounding boxes and class prediction results. The backbone network comprises CBS, C2f, and SPPF modules. The CBS module is used for feature extraction and downsampling; the C2f module achieves efficient gradient propagation and feature reuse through multi-branch feature aggregation; and the SPPF module enhances the model's adaptability to different target sizes through multi-scale pooling. The neck structure consists of FPN and PAN, achieving efficient fusion of multi-scale features through top-down semantic information transmission and bottom-up detail supplementation. The C2f module is also introduced into PANet to further enhance feature representation capabilities. The multi-layer features output from the backbone network are effectively integrated at this stage, taking into account both shallow details and deep semantics. The detection head employs a decoupled structure, separating the classification and regression branches to effectively improve feature learning efficiency and detection accuracy. Combined with an anchorless mechanism, it avoids dependence on pre-set anchor boxes, enhancing adaptability to targets with different aspect ratios. In terms of loss function design, binary cross-entropy (BCE) is used for classification tasks, while distributed focus loss (DFL) and complete intersection-union (CIoU) loss are combined to optimize bounding box regression, resulting in significant improvements in both localization accuracy and detection robustness.

[0034] As a preferred embodiment of this application, the YOLOv8 algorithm is optimized, and the structure diagram of the improved YOLOv8 algorithm is shown below. Figure 3 As shown.

[0035] A lightweight downsampling method, ADwon, is used instead of the CBS module, and its structure is as follows: Figure 4As shown in the diagram, its core design idea is to significantly reduce computational cost while maintaining downsampling effectiveness by combining pooling and lightweight convolution operations through a multi-branch mechanism. In operation, the input feature map is first subjected to average pooling to compress spatial information. Then, the feature map is divided into two sub-branches along the channel dimension. The upper branch uses a 3×3 convolution with a stride of 2 to achieve spatial downsampling, focusing on extracting local details. The lower branch first strengthens salient response regions through MaxPooling, and then uses a 1×1 convolution for feature adjustment and compression. Finally, the outputs of the two sub-branches are concatenated along the channel dimension to achieve the fusion of multi-path features.

[0036] As a preferred implementation of this application, the ADwon module significantly reduces parameter redundancy and computational complexity in its structural design compared to the traditional CBS module. When performing downsampling, the CBS module needs to perform standard convolution operations on the entire high-dimensional feature map, supplemented by batch normalization and non-linear activation functions. While this improves expressive power, it also introduces a large model size and inference overhead. ADwon, however, replaces some downsampling convolutions with parameterless pooling operations (such as AvgPool and MaxPool) and employs a channel partitioning strategy, introducing only a small number of lightweight convolutions (such as 3×3 or 1×1 convolutions) in sub-branches, thereby significantly reducing learnable parameters and computational cost. This structural design is similar to several successful deep network modules; for example, the Inception module uses multi-scale parallel paths to extract features, and Res2Net uses channel grouping to achieve fine-grained hierarchical representation. Therefore, ADwon can achieve richer semantic and detail fusion, compensating for the shortcomings of traditional single-path convolutional downsampling in contextual modeling. Furthermore, ADwon introduces implicit nonlinear expressions through structural heterogeneity without requiring explicit nonlinear activation, effectively enhancing the diversity and discriminative power of feature representations. This can improve model detection performance while reducing the number of model parameters.

[0037] As a preferred embodiment of this application, the steps for optimizing the regression box loss function are as follows: In the YOLOv8 detector head, CIOU (Complete Intersection over Union) is used for bounding box regression. Its loss function comprehensively considers three key factors: the overlap area between the predicted box and the ground truth box, the Euclidean distance between their center points, and the aspect ratio difference of the bounding boxes, as shown in the equation:

[0038] in This indicates the calculation of the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. These represent the algorithm-predicted bounding box and the ground truth bounding box, respectively. This represents the intersection of the detection bounding box and the ground truth bounding box. This represents the union of the detected bounding box and the ground truth bounding box. 'l' represents the diagonal length of the bounding boxes of the predicted and ground truth bounding boxes. These are the weighting coefficients. This is a penalty item for aspect ratio.

[0039]

[0040]

[0041] in and The values ​​represent the width and height of the true bounding box, while w and h represent the width and height of the predicted bounding box.

[0042] The formula shows that the CIoU loss function introduces center point distance and aspect ratio constraints on top of IoU to improve the geometric fitting accuracy of bounding box regression. A shape consistency term W is defined in the equation to measure the difference in aspect ratio between the predicted and ground truth boxes. However, further analysis reveals that when there is a significant difference in aspect ratio between the two boxes, the dominant role of the W term in the overall loss is significantly enhanced, easily leading to excessive penalty for low-quality samples and affecting the model's stable convergence and generalization performance.

[0043] We replace CIoU with Minimum Intersection of Points (MPDIoU). MPDIoU introduces the Euclidean distance between the diagonal corners (top left and bottom right) of the predicted bounding box and the ground truth bounding box as a perceptual constraint. While maintaining the core structure of IoU, it effectively enhances the model's ability to perceive the spatial position and scale deviation of the bounding box. Compared to traditional regression losses such as CIoU, MPDIoU avoids complex angle or scale calculations, significantly reducing computational complexity. Furthermore, this method provides more stable gradient information during regression, which helps improve the model's localization accuracy in small object detection and occlusion scenarios. and These represent the predicted bounding box and the ground truth bounding box, respectively. , ,in and The coordinates of the top left and bottom right corners of the predicted bounding box. and The coordinates of the top left and bottom right corners represent the actual bounding box.

[0044] In order to calculate A gt The area is calculated using the following formula:

[0045] The same A pred The calculation method is as follows:

[0046] A pred and A gt The overlap between surfaces can be calculated using the following formula:

[0047] The formula satisfies , , , The formula for calculating IoU is as follows:

[0048] The final calculation method for MPDIOU is as follows:

[0049]

[0050]

[0051] Where d1 represents the sum of the squares of the distances between the detection box and the ground truth boxes x1 and y1, and d2 represents the sum of the squares of the distances between the detection box and the ground truth boxes x2 and y2.

[0052] Based on the MPDIoU metric, a corresponding loss function is defined, including key parameters such as the diagonal distance between the predicted box and the ground truth box, the width and height of the bounding box.

[0053]

[0054] As a preferred implementation of this application, the MPDIOU loss function is introduced into the bounding box regression task to replace the traditional CIOU, which effectively enhances the model's ability to perceive the spatial location and scale deviation of the bounding box, thereby achieving higher accuracy in target localization in small target detection and dense scenes.

[0055] The training uses the PyTorch framework and utilizes the GPU for experiments. The specific test environment is shown in Table 1.

[0056] Table 1 Test Experimental Environment

[0057] In a preferred embodiment of this application, data augmentation is performed before the dataset is input into the detection network, including Mosaic mosaic enhancement and adaptive anchor box calculation, to improve the small object detection capability and model robustness. The enhanced images are used as training samples, and the YOLOv8 detection network is trained using supervised learning. In each training iteration, the loss function is calculated and the network parameters are updated until the model converges. The training process is set to a total of 150 training rounds, and the model weights with the smallest loss function value on the validation set are finally selected as the optimal inference weights of the object detection model.

[0058] In a preferred embodiment of this application, the saved optimal training weights are loaded into the target detection model, and the radar image is input into the model for forward inference, resulting in feature maps at three different scales (80×80, 40×40, and 20×20). Each feature map at each scale contains a class prediction branch and a bounding box regression branch. The inference results are filtered by setting a confidence threshold (0.75), and combined with the non-maximum suppression (NMS) algorithm, the precise bounding boxes of all targets in the input radar image are finally obtained.

[0059] In a preferred embodiment of this application, after target detection is completed, the pixel coordinates of the target bounding box, the class confidence score, and the appearance feature vector output by the detector in each frame are input into the DeepSORT tracker. DeepSORT first predicts the state of historical trajectories using Kalman filtering, and then calculates the similarity between the current frame detection result and the historical trajectories by combining the cosine distance of the appearance features and the spatial Mahalanobis distance. Finally, it uses the Hungarian algorithm to complete the optimal association matching. For successfully matched targets, their historical IDs are inherited and the trajectory state is updated; for unmatched targets, a new ID is assigned and the trajectory is initialized; for trajectories that are unmatched for multiple consecutive frames, the process is terminated. Ultimately, stable association and unique identifier assignment for the same target in different frame images are achieved.

[0060] This invention significantly improves the detection accuracy and localization capability of YOLOv8 for small targets on ships in complex radar images by using the lightweight downsampling module ADwon and the optimized loss function MPDIoU. Combined with the DeepSORT tracking algorithm, it achieves stable tracking and identity preservation of multiple targets in consecutive frames. The entire system is highly lightweight, suitable for deployment on embedded devices, and can further calculate the motion parameters of the targets to meet the needs of real-time maritime monitoring.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based target tracking method for marine radar images, characterized in that, Includes the following steps: Based on the echo data collected by the marine radar equipment, the echo data is parsed and the echo information is obtained based on the communication protocol. The echo information is rendered into a two-dimensional image using OpenGL and the marine radar image is output. The marine radar image is manually labeled to construct a labeled training dataset. A basic detection model based on lightweight YOLOv8 is established, and the basic detection model is optimized according to the morphological characteristics of ship targets to establish a target detection model; The marine radar image is subjected to data augmentation processing, and the target detection model is trained based on the labeled training dataset, and a weight file is output. Forward inference is performed based on the weights with the best performance during training, and a multi-scale feature map is output. The multi-scale feature map includes a category prediction branch and a bounding box regression branch. The target detection box is obtained based on threshold filtering and non-maximum suppression. The target detection box is input into the DeepSORT tracker, which extracts the appearance features and motion information of the target in the target detection box. Data association is performed based on Kalman filtering and Hungarian algorithm to continuously track multiple targets in the image sequence and maintain identity consistency.

2. The deep learning-based target tracking method for marine radar images according to claim 1, characterized in that, The construction of the labeled training dataset includes: Echo data is collected based on marine radar equipment, and echo data is parsed and echo information is obtained based on communication protocols. The echo information is rendered into a two-dimensional image using OpenGL and output as a marine radar image. Save the marine radar image as a static image file, and manually annotate the ship targets in the marine radar image using an annotation tool; The manual annotation process uses AIS ship position data to help confirm the ship's true position and boundary contour, and constructs a training dataset with YOLO format labels.

3. The deep learning-based target tracking method for marine radar images according to claim 1, characterized in that, The basic detection model is optimized by replacing the CBS module in YOLOv8 with ADown downsampling, introducing parameterless pooling operations in ADwon, and introducing lightweight convolutions in sub-branches using a channel partitioning strategy.

4. The deep learning-based target tracking method for marine radar images according to claim 3, characterized in that, Optimizing the basic detection model also includes replacing the CIoU loss function in YOLOv8 with the MPDIoU loss function and introducing the Euclidean distance between the diagonal corners of the predicted bounding box and the ground truth bounding box as a perceptual constraint.

5. The deep learning-based target tracking method for marine radar images according to claim 1, characterized in that, The data augmentation process includes Mosaic puzzle enhancement and adaptive anchor frame calculation.

6. The deep learning-based target tracking method for marine radar images according to claim 1, characterized in that, The target detection model is trained with a total of 150 training rounds. In each training iteration, the loss function is calculated and the network parameters are updated. The model weight with the smallest loss function value is selected as the optimal inference weight of the target detection model.

7. The deep learning-based target tracking method for marine radar images according to claim 1, characterized in that, The multi-scale feature maps include three scales: 80×80, 40×40, and 20×20.

8. The deep learning-based target tracking method for marine radar images according to claim 1, characterized in that, In the nonmaximum suppression operation, the confidence threshold is set to 0.

75.

9. A deep learning-based target tracking method for marine radar images according to claim 1, characterized in that, The DeepSORT tracker estimates the target state based on a Kalman filter, performs data association based on a Hungarian algorithm, initializes the ID of newly appearing targets, and cancels the ID of targets that have not been matched for a long time, thereby realizing the entry, exit, and continuous tracking management of targets.

Images