A SAR image arbitrary orientation ship target detection and identification method

Abstract

The application discloses a SAR image arbitrary orientation ship target detection and recognition method. The application adopts a separate regression branch to predict the angle, decouples the angle prediction branch from other branches, realizes more accurate and flexible estimation of the target angle, adopts the integral form expression method proposed in the DFL to predict the angle and the rectangular frame, effectively improves the prediction accuracy, strengthens the sensitivity of the model to the target size and direction change, further improves the performance in the target detection task, and simultaneously, adds the radian ratio in the loss function design, can directly minimize the angle difference between the predicted frame and the real frame, and significantly improves the convergence speed of the model when processing the rotating target detection. The application not only improves the accuracy of the model for the rotating target detection, but also enhances the adaptability of the model to the angle change, so that when facing the complex background and the target with various postures, the application can provide more robust detection performance.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image target detection technology, specifically to a method for detecting and recognizing ship targets with arbitrary orientations in SAR images. Background Technology

[0002] Synthetic Aperture Radar (SAR) is an active microwave imaging system capable of penetrating soil and vegetation, insensitive to cloud cover, and possessing unique advantages such as all-weather, all-day operation, long range, and high resolution. Due to its unique microwave penetration and high resolution, SAR can reliably detect targets even in adverse weather conditions, making it a promising application in both military and civilian fields. Ship target detection, as an important branch of SAR image interpretation, is of great significance for maritime traffic, fisheries management, battlefield situation assessment, and military target reconnaissance.

[0003] Based on different methods, SAR image target detection and recognition techniques can be broadly divided into two categories: traditional SAR image target detection and deep learning-based SAR image target detection. Traditional SAR image target detection methods are mainly based on features and classifiers, generally consisting of three stages: detection, identification, and recognition. Traditional detection methods require experts to rely on their knowledge and experience to manually design and extract salient features from the image, making the detection process very complex and cumbersome. Currently, deep learning (DL) has shown significant advantages in various tasks such as target classification, detection, and segmentation due to its powerful representation and feature learning capabilities. Convolutional Neural Networks (CNNs), with their powerful feature extraction capabilities, have rapidly emerged in the field of deep learning and also provide a possible new approach for SAR image target detection research.

[0004] Currently, CNN-based object detection methods have end-to-end training and inference capabilities, and can generally be divided into two categories: two-stage and single-stage. Two-stage detectors require obtaining a large number of candidate regions before classifying the candidate boxes and regressing their bounding boxes. Single-stage object detection networks only need to view the image once to directly predict the object's category and bounding box location information, thereby improving the speed of object detection.

[0005] SAR images are mostly acquired through airborne or spaceborne radar, and their imaging angles are generally much larger than those of optical images. This large imaging span results in some targets being very small. SAR images are typically taken from a bird's-eye view, with complex and unintuitive scenes. Therefore, near-shore ship targets often exhibit arbitrary orientations and dense clustering. Horizontal bounding boxes introduce excessive background in SAR ship target detection and can cause adjacent bounding boxes to overlap, leading to missed detections. However, existing SAR image rotation target detection and recognition methods do not address the challenges of detecting small targets and their limited number. Furthermore, many methods rely on classification to predict rotation angles, resulting in imprecise predictions and boundary effect problems. Summary of the Invention

[0006] In view of this, the present invention provides a method for detecting and recognizing ship targets with arbitrary orientation in SAR images. By simultaneously regressing and predicting the position and orientation of ship targets, it effectively realizes the detection of small rotating targets in densely docked port ship scenarios, with high detection accuracy and efficiency.

[0007] The SAR image arbitrary orientation ship target detection and identification method of the present invention includes:

[0008] Step 1: Construct a single-stage anchorless network; the head of the network includes an angle regression branch, a rectangular box regression branch, and a classification branch, wherein the angle regression branch is used to predict the rotation angle of the target; the rectangular box regression branch is used to predict the position of the target; and the classification branch is used to predict the probability of the target belonging to each category; wherein, in the angle regression branch, the integral form representation proposed in DFL is used.

[0009] Step 2: Train the network constructed in Step 1 based on the training set data; where the loss function includes classification loss and regression loss.

[0010] In the classification loss, for positive samples, the true class of the target is replaced by the rotation IoU score between the predicted bounding box and the true bounding box; for negative samples, it is 0.

[0011] The regression loss includes the DFL (Distribution Failure) for angle and rectangle regression, and the loss for rotated box regression. riou loss riou = 1 - RIoU; where RIoU is the rotational intersection-union ratio;

[0012]

[0013] Among them, IoU r The overlap ratio between the predicted bounding box and the ground truth bounding box; b and b * ρ represents the center point of the predicted bounding box and the ground truth bounding box, respectively. 2 (b,b * ω is the Euclidean distance between the two center points; ω and ω* These are the predicted and actual values ​​of the radius of the rectangle, respectively; w and w * These are the widths of the predicted bounding box and the ground truth bounding box, respectively; h and h * These are the heights of the predicted bounding box and the ground truth bounding box, respectively.

[0014] Step 3: Based on the model trained in Step 2, perform SAR image ship target detection and identification.

[0015] A better approach is to use data augmentation strategies to increase the amount of training data.

[0016] Preferably, the data augmentation strategy is as follows:

[0017] For targets smaller than 32×32, perform copy-rotate-paste operations; without intersecting with any other targets in the image, the center point of the new target is randomly selected, while the width and height remain unchanged, and the target can be rotated at any angle as long as the angle does not exceed the constraint range.

[0018] Preferably, the rectangular frame is defined using the long side method; in the data augmentation strategy, the rotation angle of the new target is added. The constraint range is:

[0019]

[0020] Where θ1 is the angle of the target to be copied, θ1∈[-π / 2,π / 2).

[0021] Preferably, in step 1, the single-stage anchor-free network uses YOLOx, YOLOv6, YOLOv8, YOLOv9, or FCOS.

[0022] A better approach is to use binary cross-entropy loss for classification.

[0023] Beneficial effects:

[0024] (1) This invention employs a separate regression branch to predict angles, decoupling the angle prediction branch from other branches, thus achieving more accurate and flexible estimation of target angles. It uses the integral form representation proposed in DFL to predict angles and bounding boxes, effectively improving prediction accuracy and enhancing the model's sensitivity to changes in target size and orientation, further improving performance in target detection tasks. Simultaneously, the addition of radian ratio to the loss function design directly minimizes the angle difference between the predicted and ground truth boxes, significantly improving the model's convergence speed when handling rotating target detection. This invention not only improves the model's accuracy in detecting rotating targets but also enhances its adaptability to angle changes, thereby providing more robust detection performance when facing targets with complex backgrounds and diverse poses.

[0025] (2) In the model training process, this invention employs the CopyRPaste data augmentation method, which generates targets at arbitrary angles by copying and rotating small targets. This increases the diversity of target angle data, helps the model better understand the characteristics of targets at different angles, enhances the model's ability to recognize small targets, and improves network accuracy. By training the model using the augmented samples, the model can be more flexibly adapted to target detection tasks in different scenarios and angles, thereby improving its generalization ability. Attached Figure Description

[0026] Figure 1 This is a flowchart of the SAR image arbitrary orientation ship target detection and recognition method of the present invention;

[0027] Figure 2 This is an example diagram of the CopyRPaste data augmentation method of the present invention;

[0028] Figure 3 This is a schematic diagram of the Head structure of the YOLOv8-R network model of the present invention;

[0029] Figure 4 This is a schematic diagram of the long side definition method for the rotating frame of the dataset in this invention;

[0030] Figure 5 This is a comparison of the AP50% of YOLOv8-R and advanced detection models of this invention on the RSDD-SAR dataset;

[0031] Figure 6 The results of the YOLOv8-R model of this invention in different scenes on the RSDD-SAR dataset are shown. Detailed Implementation

[0032] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] This invention provides a method for detecting and recognizing ship targets with arbitrary orientation in SAR images. It is based on a single-stage anchor-free network, such as the YOLO series YOLOx, YOLOv6, YOLOv8, YOLOv9, and FCOS network. The method predicts the angle by adding a branch using the DFL method to the head region, and accelerates the convergence of the model by designing a regression loss function, thereby improving the detection efficiency of the model.

[0034] This embodiment uses a YOLOv8-R network based on YOLOv8 as an example for illustration. The SAR image arbitrary orientation ship target detection and recognition method of this invention includes data augmentation strategies, a YOLOv8-R network model, a loss function, and model training and inference, such as… Figure 1 As shown, the specific steps are as follows:

[0035] Step 1: Develop a data augmentation strategy.

[0036] To improve the generalization ability of a model, data augmentation can be performed to increase the number of samples the network learns from and improve the training accuracy of the network.

[0037] There are many existing data augmentation methods, such as Rotate, Pad, CenterCrop, Cutout, RandomAffine, etc.; YOLOv8 also comes with data augmentation techniques such as Resize, Mosaic, and Mixup.

[0038] Considering the relatively homogeneous nature of SAR image scenes, this embodiment prioritizes the Albumentations library and CopyRPaste for data augmentation to further enhance the diversity of sample data. Firstly, the Albumentations library is used for data augmentation; specifically, only flipping and hue / saturation / brightness adjustment are retained from the basic augmentation operations provided by the Albumentations library to achieve effective and appropriate enhancement of the input data.

[0039] Then, for small targets smaller than 32×32, further angle-related enhancements are performed. The CopyRPaste method of this invention belongs to the anchor-frame level data augmentation method, specifically: performing a copy-rotate-paste operation on targets smaller than 32×32. For example... Figure 2 As shown, without intersecting with any other targets in the image, the center point of the newly added target is randomly selected, while its width and height remain unchanged. The target can be rotated at any angle, provided the angle does not exceed the constraints. If the angle of the target to be copied is θ1, the rotation angle is... The new target's true bounding box angle θ * It can be obtained from the following formula:

[0040]

[0041] Step 2: Construct the YOLOv8-R network model.

[0042] This embodiment is based on the YOLOv8 network model, with an additional angle regression branch added to the Head region. The Head structure is as follows: Figure 3 As shown, it includes a rectangular regression branch, an angular regression branch, and a classification branch. The angular regression branch and the rectangular regression branch together form a rotated box regression, i.e. Figure 3In the diagram, RBox represents the classification branch; cls represents the classification branch; each branch consists of two Conv layers and one Conv2d layer; where Conv2d is a two-dimensional convolution; Conv is a set of two-dimensional convolution, batch normalization (BN) layers, and activation functions; k, s, p, and c represent the kernel size, kernel stride, image padding pixel size, and number of output channels, respectively. The angle regression branch uses the integral form proposed in Distribution Focal Loss (DFL) and is decoupled from the classification and bounding box regression branches. In this embodiment, the bounding box adopts the long-side definition method, that is, the rotation angle of the bounding box is in the range of [-π / 2, π / 2). To prevent the angle DFL calculation from exceeding the right boundary, the angle regression branch predicts a discrete distribution of 181 angle values. After decoding the angle and bounding box regression branches, the center point position, width, height, and angle information (cx, cy, w, h, θ) of the rotated box are obtained.

[0043] Step 3: Construct the loss function.

[0044] The loss function includes regression loss and classification loss, and the loss is obtained using the following formula:

[0045] Loss = λ1 × loss clc (c,t,t * )+λ2×loss reg (t,t * )

[0046] Where λ1 and λ2 are weight coefficients, c is the class score predicted by the model, and t is the bounding box parameter (cx, cy, w, h, θ). * For the true bounding box parameters (cx) * ,cy * ,w * ,h * ,θ * ), where (cx * ,cy * ,w * ,h * ) represent the center point, width, and height of the actual bounding box, respectively.

[0047] loss clc (c,t,t * For classification loss, this embodiment uses Binary CrossEntropy Loss (BCE Loss), as shown below:

[0048] loss clc (c,t,t *)=-(q(t,t * )×log(c)+(1-q(t,t * ))×log(1-c))

[0049] Where q(t,t) * The target score is represented by the rotation IoU value between the predicted bounding box and the ground truth bounding box for positive samples, and 0 for negative samples.

[0050] Classification loss can also use loss functions such as VFL (varifocal loss) and FL (Flocal Loss), and the target class label can be replaced with the IoU value.

[0051] loss reg (t,t * The regression loss for the rotated bounding box includes the Rotated Intersection over Union Loss (RIoU Loss) and the Distribution Focal Loss (DFL), as shown below:

[0052] loss reg (t,t * ) = loss dfl (S i ,S i+1 )+loss riou (t,t * )

[0053] loss dfl (S i ,S i+1 () is the Discrete Probability Distribution of Angular Regression and Rectangular Regression. The model predictions are obtained by performing a softmax operation S(·) to obtain the discrete probability distributions of the prediction parameters l, r, t, b, and θ, where S i S i+1 It is relative to the true parameter l * r * t * b * θ * The probabilities of the left and right nearest integer boundaries, l, r, t, b, and l * r * t * b * These are the distances from the center point of the predicted bounding box and the ground truth bounding box to their left, right, top, and bottom boundaries, respectively. The DFL (Distance-to-Flatness) is obtained using the following formula:

[0054] loss dfl (S i ,S i+1)=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))

[0055] Where y is the actual value of the rotated frame, y i and y i+1 It is obtained by rounding y down and up, representing the left and right integer boundaries closest to y.

[0056] loss riou (t,t * The ) is the rotation intersection-union ratio loss, using CIoU as a reference. Based on the rotated bounding box features, while considering the overlap rate between the predicted and ground truth bounding boxes, it introduces the Euclidean distance of the center point, aspect ratio, and radian ratio. RIoU is the rotation intersection-union ratio, as shown below:

[0057] loss riou (t,t * )=1-RIoU

[0058]

[0059] Where b and b * ρ represents the center point of the predicted bounding box and the ground truth bounding box, respectively. 2 (b,b * IoU is the Euclidean distance between two center points. r ω and ω' are the overlap ratio between the predicted bounding box and the ground truth bounding box. * These represent the predicted and actual values ​​of the radius of the rotated frame, respectively.

[0060] Step 4: Model training and inference.

[0061] The publicly available SAR image ship rotation target detection dataset RSDD-SAR was used as the training and inference dataset, with all samples being 512×512 pixels. This dataset was segmented according to the COCO dataset criteria, classifying ship sizes accordingly. Small targets accounted for 81.17%, while medium-sized targets accounted for 18.78%. The RSDD-SAR dataset's rotation bounding box definition method is as follows: Figure 4 As shown, the long-side definition method is used. The rotated bounding box of the real target is labeled (cx * ,cy * ,w * ,h * ,θ * ).

[0062] After applying the above data augmentation strategies to the data, it is input into the YOLOv8-R network model. The output feature map is decoded, and backpropagation is performed using the loss function to update the model parameters. Once the model converges, the model weight file is obtained. According to... Figure 5 It can be seen that, compared with some currently highly competitive rotating bounding box prediction models, YOLOv8-R has the highest average precision (AP) at a 50% intersection-over-union (IoU) threshold, when the model parameters are less than 20M, for both nearshore and offshore scenes. The detection accuracy of all algorithms for nearshore scenes is significantly lower than that for offshore scenes. Using the model weight file generated from model training, inference was performed on different scenes in SAR images, including port scenes, densely packed scenes, waterway scenes, low-resolution offshore scenes, and high-resolution offshore scenes. The detection results are as follows: Figure 6 As shown, the method proposed in this invention can detect almost all targets in waterway and offshore high-resolution scenarios. It also performs excellently in port, densely packed, and offshore low-resolution scenarios, efficiently identifying and locating the vast majority of targets. The method proposed in this invention is adaptable to multiple scenarios, maintaining a high detection rate even in complex environments, demonstrating excellent robustness.

[0063] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting and recognizing ship targets with arbitrary orientations in SAR images, characterized in that, include: Step 1: Construct a single-stage anchor-free network; The head portion of the network includes an angle regression branch, a bounding box regression branch, and a classification branch. The angle regression branch is used to predict the rotation angle of the target; the bounding box regression branch is used to predict the target position; and the classification branch is used to predict the probability that the target belongs to each category. The angle regression branch uses the integral form proposed in DFL. Step 2: Train the network constructed in Step 1 based on the training set data; where the loss function includes classification loss and regression loss. In the classification loss, for positive samples, the true class of the target is replaced by the rotation IoU score between the predicted bounding box and the true bounding box; for negative samples, it is 0. The regression loss includes the DFL (Distribution Failure) for angle and rectangle regression, and the loss for rotated box regression. riou loss riou = 1 - RIoU; where RIoU is the rotational intersection-union ratio; Among them, IoU r The overlap ratio between the predicted bounding box and the ground truth bounding box; b and b * ρ represents the center point of the predicted bounding box and the ground truth bounding box, respectively. 2 (b,b * ω is the Euclidean distance between the two center points; ω and ω * These are the predicted and actual values ​​of the radius of the rectangle, respectively; w and w * These are the widths of the predicted bounding box and the ground truth bounding box, respectively; h and h * These are the heights of the predicted bounding box and the ground truth bounding box, respectively. Step 3: Based on the model trained in Step 2, perform SAR image ship target detection and identification.

2. The method as described in claim 1, characterized in that, Data augmentation strategies are employed to increase the amount of data in the training set.

3. The method as described in claim 2, characterized in that, The data augmentation strategy is specifically as follows: For targets smaller than 32×32, perform copy-rotate-paste operations; without intersecting with any other targets in the image, the center point of the new target is randomly selected, while the width and height remain unchanged, and the target can be rotated at any angle as long as the angle does not exceed the constraint range.

4. The method as described in claim 3, characterized in that, The rectangle is defined using the long side method; in the data augmentation strategy, the rotation angle of the new target is added. The constraint range is: Where θ1 is the angle of the target to be copied, θ1∈[-π / 2,π / 2).

5. The method according to any one of claims 1 to 4, characterized in that, In step 1, the single-stage anchorless network uses YOLOx, YOLOv6, YOLOv8, YOLOv9, or FCOS.

6. The method as described in claim 1, characterized in that, The classification loss uses binary cross-entropy loss.

Images