A SAR ship rotating target detection method

Abstract

The present invention relates to a method for detecting rotating ship targets in SAR images, comprising the following steps: obtaining ship data to be detected in SAR images; inputting the ship data to be detected in the SAR images into a trained rotating target detection model for inference to obtain the angle, position, and category information of the ship target; constructing the rotating target detection model based on a YOLOv5 network with a convolutional block attention mechanism and a midpoint offset representation method for a rotated rectangular box; training the trained rotating target detection model based on a pre-constructed training set and using a multi-task joint network loss function adapted for rotation detection; the pre-constructed training set is constructed based on original SAR image ship sample data and a label file annotated with a rotated rectangular box containing the ship target to be detected. This method can avoid loss boundary discontinuity, has the advantage of a lightweight network with a small number of parameters, and can detect ship targets in SAR images in any direction and obtain corresponding angle information of the ship.

Description

Technical Field

[0001] The present invention belongs to the technical field of image target detection, and in particular relates to a SAR ship rotating target detection method. Background Art

[0002] Object detection is a crucial task in computer vision. In recent years, with the rapid development of deep convolutional neural networks (CNNs), CNN-based object detection networks have become a research hotspot. Due to their ability to autonomously extract rich image feature information, CNN-based object detection methods have significantly improved the performance of object detection in natural scene images, addressing the shortcomings of traditional object detection methods. Generally speaking, current CNN-based object detection methods are categorized into two types: two-stage and single-stage. Two-stage object detection networks, including RCNN, Fast RCNN, and Faster RCNN, extract features of regions of interest after obtaining a large number of candidate regions, followed by secondary classification and regression. Single-stage object detection networks, including SSD and YOLO, can directly regress the bounding box and position of an object, thereby accelerating object detection.

[0003] However, Synthetic Aperture Radar (SAR) images differ significantly from natural scene images, and SAR image interpretation presents significant difficulties and challenges. SAR images typically present a bird's-eye view, with complex and non-intuitive scene representations. The images are large, while the targets are small. Unlike the generally horizontal distribution of objects in natural images, objects in SAR images, such as ships and vehicles, are often arranged in arbitrary orientations, especially in dense scenes where multiple targets are arranged adjacent to each other. Therefore, general-purpose horizontal target detection has inherent limitations in the field of remote sensing image target detection, hindering its direct application. Rotational target detection, which introduces angle parameters, has attracted widespread attention and discussion in the field of remote sensing image target detection because it is more suitable for targets with arbitrary postures, further increasing the difficulty of accurate detection.

[0004] For example, in 2018, Jianqi Ma proposed a method based on Faster RCNN to realize scene text detection in arbitrary directions based on rotated candidate boxes. However, the design based on rotated candidate boxes increased the calculation of a large number of network parameters and memory usage. For example, in 2019, Jian Ding proposed the RoI Transformer structure, which aims to learn the rotated region of interest directly from the horizontal region of interest. However, the RoI Transformer structure is relatively heavy and complex. For example, in 2021, Xingxing Xie proposed a two-stage rotation target detector based on the midpoint offset representation of the rotation box. However, it is slower than the single-stage network.

[0005] At the same time, due to the introduction of angle parameters, current rotational object detection suffers from boundary discontinuities such as angular periodicity and edge commutativity. The challenge of achieving both fast and efficient detection is a pressing issue in the field of rotational object detection. Summary of the Invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a SAR ship rotating target detection method. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] The embodiment of the present invention provides a SAR ship rotating target detection method, comprising the steps of:

[0008] Obtain the ship data of the SAR image to be detected;

[0009] Inputting the ship data to be detected in the SAR image into the trained rotating target detection model for inference to obtain the angle, position and category information of the ship target;

[0010] Among them, the rotating target detection model is constructed based on the YOLOv5 network with convolutional block attention mechanism and the midpoint offset representation of the rotated rectangular box; the trained rotating target detection model is obtained by training the rotating target detection model based on a pre-constructed training set and using a multi-task joint network loss function adapted to rotation detection; the pre-constructed training set is constructed based on the original SAR image ship sample data and the label file that annotates the rotated rectangular box where the ship target to be detected is located.

[0011] In one embodiment of the present invention, the method for constructing the training set includes:

[0012] Obtain raw SAR image ship sample data;

[0013] The original SAR image ship sample data is cropped according to a preset overlap ratio to obtain a plurality of slice images;

[0014] Mark each slice image with a rotating rectangular box where the ship target to be detected is located, obtain the vertex coordinate values, category and detection difficulty of the rotating rectangular box, and form a label file;

[0015] The plurality of slice images and the label file are divided into a training set and a test set.

[0016] In one embodiment of the present invention, the rotating object detection model includes an input end, a backbone network, a neck network and a prediction end connected in sequence, wherein:

[0017] The input end is used to perform data enhancement processing on the plurality of slice images during the training phase, and convert the label file into a long side representation of a rotated rectangular box and a midpoint offset representation of the rotated rectangular box to obtain processed data; and in the inference phase, perform data enhancement processing on the ship data of the SAR image to be detected to obtain processed data;

[0018] The backbone network is used to extract features from the processed data to obtain feature information of different scales;

[0019] The neck network is used to fuse shallow graphic information and deep semantic information from the backbone network to construct feature information with local details and regional semantic representation capabilities;

[0020] The prediction end is used to process the feature information output by the neck network and output feature maps of three different scales to classify targets of different scales and locate them based on the midpoint offset representation of the rotated rectangular box.

[0021] In one embodiment of the present invention, a convolutional block attention mechanism module is inserted into the backbone network.

[0022] In one embodiment of the present invention, the midpoint offset of the rotated rectangular box includes the coordinates of the center point of the envelope rectangle of the rotated rectangular box, the width of the envelope rectangle of the rotated rectangular box, the height of the envelope rectangle of the rotated rectangular box, the offset of the upper corner point of the rotated rectangular box corresponding to the midpoint of the upper side of the envelope rectangle, and the offset of the right corner point of the rotated rectangular box corresponding to the midpoint of the right side of the envelope rectangle.

[0023] In one embodiment of the present invention, in each feature map output by the prediction end, the prediction format for each position is:

[0024] n a ×(6+conf+n c );

[0025] Among them, n a Indicates the number of anchor boxes of different scales, conf indicates the confidence that there is an object in the prediction box, n c Represents the number of target categories, a 6-dimensional vector ti =(t x , t y , t w , t h , t α , t β ) represents the offset of the predicted rotated rectangle relative to the anchor box under the midpoint offset representation of the rotated rectangle.

[0026] In one embodiment of the present invention, the multi-task joint network loss function is:

[0027] L=λ1L cls +λ2L obj +λ3L box

[0028] Among them, λ1, λ2, λ3 are network adaptation weight parameters, L cls is the classification loss, L obj is the confidence loss, L box Regression loss adapted to rotation detection;

[0029] L box Used to calculate the positioning loss of the predicted rotated rectangular box and the corresponding true labeled rotated rectangular box:

[0030] L box (t′ i ,σ(t i ))=L smooth-l1 (t′ i ,σ(t i ))

[0031] Among them, the regression loss uses the smooth-L1 function, t i =(t x , t y , t w , t h , t α , t β ) is the offset of the predicted rotated rectangle relative to the anchor box, σ(·) represents the sigmoid function, and t′ i =(t′ x , t′ y ,,t′ w , t′ h , t′ α , t′ β ) is the offset of the real annotation rotated rectangle relative to the anchor box;

[0032] σ(t i ) and t′ i The details are as follows:

[0033]

[0034] Among them, (x g ,y g ) is the coordinate of the center point of the real marked rotation rectangle, w g and h g is the width and height of the envelope rectangle of the real annotation rotation rectangle, Δα g and Δβ g They are the offsets of the real annotation rotated rectangle relative to the upper midpoint and the right midpoint of its enveloping rectangle.

[0035] In one embodiment of the present invention, the training method of the rotating object detection model includes:

[0036] Load pre-trained weight initialization parameters;

[0037] The training samples in the training set are input into the rotation target detection model, and the rotation target detection model is trained using a multi-task joint network loss function. The model parameters are updated according to the gradient reverse iteration, and the multi-task joint network loss function is minimized to obtain a trained rotation target detection model.

[0038] In one embodiment of the present invention, the ship data to be detected in the SAR image is input into a trained rotating target detection model for inference to obtain the angle, position and category information of the ship target, including:

[0039] Cutting the ship data of the SAR image to be detected into a plurality of slice images according to a preset overlapping ratio;

[0040] Inputting the plurality of slice images into a trained rotation object detection model for inference, so that the prediction end outputs feature maps of three different scales, wherein the prediction format of each position in the feature map output by the prediction end includes the category of the corresponding object, the confidence level, and the offset of the predicted rotated rectangular box relative to the anchor box using the midpoint offset representation of the rotated rectangular box;

[0041] The prediction end decodes the offset of the predicted rotated rectangular frame relative to the anchor frame to obtain a predicted rotated rectangular frame represented by a midpoint offset;

[0042] Converting the predicted rotated rectangular frame represented by the midpoint offset into a predicted rotated rectangular frame represented by the long side;

[0043] The predicted rotated rectangular frame represented by the long side is subjected to non-maximum suppression (NMS) processing to obtain the angle, position and category information of the ship target.

[0044] In one embodiment of the present invention, the process of decoding the offset of the predicted rotated rectangular frame relative to the anchor frame to obtain the predicted rotated rectangular frame represented by the midpoint offset is:

[0045]

[0046] Among them, (x, y) is the coordinate of the center point of the predicted rotating rectangle, w and h are the width and height of the envelope rectangle of the predicted rotating rectangle, Δα and Δβ are the offsets of the predicted rotating rectangle relative to the upper midpoint and the right midpoint of the envelope rectangle, σ(,) represents the sigmoid function, c x and c y Respectively represent the position coordinates of the feature points in the feature map, w a and h a They represent the width and height of the anchor box respectively, and s is the stride of the feature map relative to the original image.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. The rotating target detection model in the present invention is constructed based on the YOLOv5 network of the convolutional block attention mechanism and the midpoint offset representation of the rotated rectangular box, and has the following advantages: First, a detection backbone network based on the convolutional block attention mechanism is proposed to automatically obtain the importance of different channels and spaces of the feature map, thereby enhancing the feature extraction capability of the model and improving the detection performance of the rotating target detection network; Second, the use of the midpoint offset representation of the rotated rectangular box avoids the boundary discontinuity problem that often occurs in the rotation box detection due to the commutativity of the edges and the periodicity of the angle parameters, reduces the difficulty of network training, and improves the positioning accuracy of the rotation target detection; Third, the constructed rotating target detection model is a new single-stage rotating box detection network based on the midpoint offset representation, which not only avoids the loss of boundary discontinuity, but also has the advantage of a lightweight network with a small number of parameters, ultimately realizing the arbitrary direction detection of ship targets in SAR images and obtaining the corresponding angle information of the ship, especially improving the detection rate of ship targets in complex near-shore scenes.

[0049] 2. The rotated rectangular box of the present invention directly inherits the regression mechanism of the horizontal box based on the single-stage rotated box detection network, and only adds two offset factors on the basis of the horizontal box to predict the rotated rectangular box, thereby preventing the introduction of the rotated anchor box from bringing heavy network prediction layer parameters, thereby obtaining a simple, efficient and lightweight rotated object detection network. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic flow chart of a SAR ship rotating target detection method provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the structure of a rotating target detection model provided by an embodiment of the present invention;

[0052] Figure 3Schematic diagram of different representation methods of a rotating rectangular frame provided by an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of the structure of each module in the backbone network provided by an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of the structure of a convolutional block attention mechanism provided by an embodiment of the present invention;

[0055] Figure 6 A flowchart of a method for constructing a training set and a method for training a rotation target detection model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0057] Example 1

[0058] See Figure 1 , Figure 1 A flow chart of a SAR ship rotating target detection method provided by an embodiment of the present invention. The SAR ship rotating target detection method comprises the following steps:

[0059] S1. Obtain the SAR image ship data to be detected.

[0060] S2. Input the ship data to be detected in the SAR image into the trained rotating target detection model for inference to obtain the angle, position and category information of the ship target.

[0061] Among them, the rotating target detection model is constructed based on the YOLOv5 network with convolutional block attention mechanism and the midpoint offset representation of the rotated rectangular box; the trained rotating target detection model is obtained by training the rotating target detection model based on a pre-constructed training set and using a multi-task joint network loss function adapted to rotation detection; the pre-constructed training set is constructed based on the original SAR image ship sample data and the label file that annotates the rotated rectangular box where the ship target to be detected is located.

[0062] The rotating target detection model in this embodiment is constructed based on the YOLOv5 network with a convolutional block attention mechanism and a midpoint offset representation of a rotated rectangular box, and has the following advantages: First, a detection backbone network based on a convolutional block attention mechanism is proposed to automatically obtain the importance of different channels and spaces in the feature map, thereby enhancing the feature extraction capability of the model and improving the detection performance of the rotating target detection network; Second, the use of the midpoint offset representation of the rotated rectangular box avoids the boundary discontinuity problem that often occurs in rotated box detection due to the commutativity of edges and the periodicity of angle parameters, reduces the difficulty of network training, and improves the positioning accuracy of rotated target detection; Third, the constructed rotating target detection model is a new single-stage rotating box detection network based on the midpoint offset representation, which not only avoids the loss of boundary discontinuity but also has the advantage of a lightweight network with a small number of parameters, ultimately realizing arbitrary direction detection of ship targets in SAR images and obtaining the corresponding angle information of the ship, especially improving the detection rate of ship targets in complex near-shore scenes.

[0063] See Figure 2 , Figure 2 This is a schematic diagram of the structure of a rotating object detection model provided by an embodiment of the present invention. This rotating object detection model is constructed by adding a convolutional block attention mechanism module before the prediction end of the YOLOv5 network and utilizing the midpoint offset representation of a rotated rectangle. Specifically, the rotating object detection model includes an input end (Input), a backbone network (Backbone), a neck network (Neck), and a prediction end (Detect), all connected in sequence. In this embodiment, the backbone network uses New CSPDarknet53, and the neck network uses PANet.

[0064] Specifically, the input end is used to perform data augmentation processing on the plurality of slice images during the training phase and convert the label file into a representation of the long side of a rotated rectangular box and a representation of the midpoint offset of the rotated rectangular box to obtain processed data. During the inference phase, data augmentation processing is performed on the SAR image ship data to obtain processed data. The input end performs data augmentation processing on the data, including random scaling, translation, flipping, mosaicing, color gamut enhancement, etc.

[0065] In the training phase, the label file of the original SAR image ship sample data can be a txt file in the DOTA annotation format. While the input end performs data enhancement processing, the DOTA annotated file is converted into the long side representation of the rotated rectangular box. le90 , the midpoint offset of the rotated rectangle represents D mo For backup. Figure 3 , Figure 3Schematic diagram of different representation methods of a rotating rectangular frame provided by an embodiment of the present invention. le90 The format is [x le ,y le , w le , h le ,θ le ],θ le ∈[-π / 2,π / 2), which are the coordinates of the center point, long side, short side and the angle between the x-axis and the long side of the rotated rectangle; D mo The format is [x mo ,y mo , w mo , h mo , Δα mo , Δβ mo ] are the coordinates of the center point of the rotated rectangle's envelope, the width of the rotated rectangle, the height of the rotated rectangle, the offset between the upper corner of the rotated rectangle and the upper midpoint of the envelope, and the offset between the right corner of the rotated rectangle and the right midpoint of the envelope. During the inference phase, the input performs data augmentation processing and then directly outputs it to the backbone network.

[0066] Specifically, the backbone network is used to extract features from the processed data to obtain feature information of different scales.

[0067] like Figure 2 As shown in the figure, the backbone network includes the basic convolution module CBS, the cross-stage local network module C3, the convolutional block attention mechanism module CBAM, and the spatial pyramid pooling module SPPF. In this embodiment, the backbone network includes CBS1, CBS2, C31, CBS3, C32, CBS4, C33, CBS5, C34, CBAM, and SPPF connected in sequence.

[0068] See Figure 4 , Figure 4 This is a schematic diagram of the structure of each module in the backbone network provided by an embodiment of the present invention.

[0069] The basic convolution module CBS contains the convolution layer Conv, the batch normalization layer BN, and the activation function layer Silu connected in sequence.

[0070] The cross-stage local network module C3 includes a basic convolutional module CBS, multiple residual layers Resunit*n connected to CBS, a convolutional layer Conv, a cross-stage merging layer Concat between Resunit*n and Conv, and a basic convolutional module CBS connected to Concat. The cross-stage local network module C3 merges the input feature layer, which passes through the convolutional layer Conv and n residual components Resunit*n, with the input convolution layer after passing through a single convolutional layer Conv, using Concat. The residual component Resunit adds the input feature layer after passing through two CBS modules to the original input feature layer. This residual component acts on the C3 module to avoid the problem of vanishing gradients caused by the deepening of the number of layers.

[0071] The spatial pyramid pooling module SPPF includes the basic convolution module CBS, three maximum pooling layers MaxPool connected in parallel after CBS, the cross-stage merging layer Concat connected to CBS and MaxPool, and the basic convolution module CBS connected to Concat; the spatial pyramid pooling module SPPF adopts feature pooling layers of different scales to realize the fusion of local feature and global feature information.

[0072] See Figure 5 , Figure 5 A schematic diagram of the convolutional block attention mechanism structure provided for an embodiment of the present invention. The convolutional block attention mechanism CBAM includes a channel attention module CAM and a spatial attention module SAM. CBAM is used to adaptively extract and fuse important features of the channel and spatial dimensions to improve detection performance. The CAM module passes the H*W*C input feature map through a parallel adaptive maximum pooling layer MaxPool and an adaptive average pooling layer AvgPool to obtain two 1*1*C feature maps, which are respectively passed through the weight-sharing multi-layer perceptron module Conv-Relu-Conv, added together, and passed through a sigmoid activation function, and finally multiplied with the input feature map to restore it to a feature map of H*W*C size. Among them, the multi-layer perceptron module Conv-Relu-Conv compresses the number of channels in turn, passes through the Relu activation function, and expands it to the original number of channels for output. The SAM module takes the feature map output by the CAM module as input, and obtains two 1*H*W feature maps through parallel maximum pooling Max and average pooling Mean operations along the channel axis and concatenates them. It then passes through convolution Conv and sigmoid activation function in sequence, and finally multiplies the input feature map to restore it to a feature map of size H*W*C.

[0073] Specifically, the neck network is used to fuse shallow graphic information and deep semantic information from the backbone network to construct feature information with local details and regional semantic representation capabilities.

[0074] Please see again Figure 2 The neck network includes CBS6, upsampling layers UpSample1, Concat1, C35, CBS7, UpSample2, Concat2, C36, CBS8, Concat3, C37, CBS9, Concat4, and C38, which are connected to SPPF in sequence. Among them, CBS6 is connected to SPPF, Concat1 is connected to C33, Concat2 is connected to C32, Concat3 is connected to CBS7, and Concat4 is connected to CBS6.

[0075] Specifically, the prediction end Detect is used to process the feature information output by the neck network and output feature maps of three different scales to classify targets of different scales and locate them based on the midpoint offset of the rotated rectangular box. It should be noted that the prediction end Detect passes the feature information of the three scales output by the neck network through the convolution layer and adjusts the dimension of the feature tensor to obtain the output feature map. If it is currently in the training stage, the output feature map is directly output to the loss calculation end. If it is currently in the inference stage, the output feature map is decoded and output to the post-processing end.

[0076] Please see again Figure 2 The prediction end Detect includes Detect1, Detect2, and Detect3, among which Detect1 is connected to C36, Detect2 is connected to C37, and Detect3 is connected to C38.

[0077] The prediction side Detect corresponds to three different scales of output feature maps. The prediction format for each position in each output feature map is:

[0078] n a ×(6+conf+n c );

[0079] Among them, n a Indicates the number of anchor boxes of different scales, usually 3; conf indicates the confidence that there is an object in the prediction box, n c Represents the number of target categories, a 6-dimensional vector t i =(t x , t y , t w , t h , t α , t β ) represents the offset of the predicted rotated rectangle relative to the anchor box under the midpoint offset representation of the rotated rectangle.

[0080] The rotated rectangular box of this embodiment directly inherits the regression mechanism of the horizontal box based on the single-stage rotated box detection network. It only adds two offset factors on the basis of the horizontal box to predict the rotated rectangular box. This prevents the introduction of the rotated anchor box from bringing heavy network prediction layer parameters, thereby obtaining a simple, efficient and lightweight rotated object detection network.

[0081] It should be noted that the convolutional block attention mechanism module (CBAM) can be inserted into any layer before the Detect layer of the YOLOv5 network prediction end. For example, it can be inserted between the input and the backbone network, between the backbone network and the neck network, after any module layer in the backbone network, or after any module layer in the neck network. In this embodiment, the CBAM module is inserted into the ninth layer of the backbone network, that is, before the spatial pyramid pooling module (SPPF).

[0082] In this embodiment, after the convolutional block attention mechanism (CBAM) is inserted into the backbone network, the importance of high-level semantic information can be materialized in the representation of the feature map.

[0083] Furthermore, the multi-task joint network loss function of the above rotation target detection model is:

[0084] L=λ1L cls +λ2L obj +λ3L box

[0085] Among them, λ1, λ2, λ3 are network adaptation weight parameters, L cls is the classification loss, L obj is the confidence loss, L box Regression loss adapted to rotation detection;

[0086] L box Used to calculate the positioning loss of the predicted rotated rectangular box and the corresponding true labeled rotated rectangular box:

[0087] L box (t′ i ,σ(t i ))=L smooth-l1 (t′ i ,σ(t i ))

[0088] Among them, the regression loss uses the smooth-L1 function, t i =(t x , t y , t w , t h , t α , t β) is the offset of the predicted rotated rectangle relative to the anchor box, σ(·) represents the sigmoid function, and t′ i =(t′ x , t′ y , t′ w , t′ h , t′ α , t′ β ) is the offset of the real annotation rotated rectangle relative to the anchor box;

[0089] σ(t i ) and t′ i The details are as follows:

[0090]

[0091] Among them, (x g ,y g ) is the coordinate of the center point of the real marked rotation rectangle, w g and h g is the width and height of the envelope rectangle of the real annotation rotation rectangle, Δα g and Δβ g They are the offsets of the real annotation rotated rectangle relative to the upper midpoint and the right midpoint of its enveloping rectangle.

[0092] See Figure 6 , Figure 6 A flowchart of a method for constructing a training set and a method for training a rotation target detection model provided in an embodiment of the present invention.

[0093] The method for constructing a training set and the method for training a rotating target detection model in this embodiment include the following steps:

[0094] S601: Acquire original SAR image ship sample data. Specifically, collect original wide-band SAR marine ship target images.

[0095] S602: Crop the original SAR image ship sample data according to a preset overlap ratio to obtain a plurality of slice images.

[0096] Specifically, since the size of the original wide-swath SAR image is usually more than 10,000 pixels, each wide-swath SAR image is cropped with a certain overlap ratio to obtain multiple N*N pixel slice images, where N is usually less than or equal to 1024.

[0097] S603 , annotate each slice image with a rotating rectangular frame where the ship target to be detected is located, obtain the vertex coordinate values, category and detection difficulty of the rotating rectangular frame, and form a label file.

[0098] Specifically, a sample label in a unified format is annotated for each slice image. In this embodiment, the rotation annotation tool roLabelImg is used to annotate the rotated rectangular box where the ship target to be detected is located, and the label is converted into a txt file in the DOTA annotation format commonly used for rotated rectangular box detection. The DOTA format is [x1, y1, x2, y2, x3, y3, x4, y4, category, difficult], which are the coordinate values, category, and detection difficulty of the four vertices of the rotated box respectively.

[0099] S604: Divide the plurality of slice images and the label file into a training set and a test set.

[0100] Specifically, a number of slice images and label files are divided into a training set and a test set in a certain ratio, for example, the ratio of the training set to the test set is 8:2.

[0101] S605: Load pre-training weight initialization parameters.

[0102] Specifically, during the training process, in order to speed up the convergence of model training, the pre-trained weight initialization parameters obtained on the optical image ImageNet are loaded.

[0103] S606. Input the training samples in the training set into the rotating object detection model, and use the multi-task joint network loss function to train the rotating object detection model. Update the model parameters according to the gradient reverse iteration, minimize the multi-task joint network loss function, and obtain a trained rotating object detection model.

[0104] Specifically, the training samples in the training set are input into the rotating object detection model. After being processed by the input terminal, the backbone network, and the neck network, the prediction terminal Detect outputs three different scale output feature maps: small, medium, and large. This is used to classify objects of different scales and locate them based on the midpoint offset of the rotated rectangular box. Then, the model parameters are optimized through gradient backpropagation to minimize the multi-task joint network loss function. A total of 300 epochs are trained to obtain a trained rotating object detection model. Furthermore, after obtaining the trained rotating object detection model, it can be tested using a test set.

[0105] Specifically, when training the rotating target detection model in this embodiment, the initial learning rate is set to 0.01, and the optimizer selects the SGD method.

[0106] Furthermore, after obtaining the tested rotating target detection model, the tested rotating target detection model is used to infer the ship data of the SAR image to be detected, that is, step S2 specifically includes:

[0107] S21 , cutting the to-be-detected SAR image ship data into a plurality of slice images according to a preset overlapping ratio.

[0108] S22. Input the several slice images into a trained rotation target detection model for inference, so that the prediction end outputs feature maps of three different scales, wherein the prediction format of each position in the feature map output by the prediction end includes the category of the corresponding target, the confidence level, and the offset of the predicted rotated rectangular box relative to the anchor box under the midpoint offset representation of the rotated rectangular box.

[0109] S23: Enable the prediction end to decode the offset of the predicted rotated rectangular frame relative to the anchor frame to obtain the predicted rotated rectangular frame represented by the midpoint offset.

[0110] Specifically, the process of decoding the offset of the predicted rotated rectangular frame relative to the anchor frame to obtain the predicted rotated rectangular frame represented by the midpoint offset is:

[0111]

[0112] Where (x, y) is the coordinate of the center point of the predicted rotating rectangle, w and h are the width and height of the envelope of the predicted rotating rectangle, Δα and Δβ are the offsets of the predicted rotating rectangle relative to the upper and right midpoints of the envelope, σ(·) represents the sigmoid function, and c x and c y Respectively represent the position coordinates of the feature points in the feature map, w a and h a They represent the width and height of the anchor box respectively, and s is the stride of the feature map relative to the original image.

[0113] S24, rotating the predicted rectangular frame D represented by the midpoint offset mo Convert the predicted rotated rectangle D into the long side representation le90 .

[0114] S25. Perform non-maximum suppression (NMS) processing on the predicted rotated rectangular frame represented by the long side, filter out repeated rotated rectangular frames, obtain the angle, position and category information of the ship target, and perform visualization processing.

[0115] The specific implementation of steps S21-S22 can be found in the training process. The specific implementation of steps S24-S25 is an existing technical solution and will not be repeated here.

[0116] The method in this embodiment uses the YOLOv5 network to construct a fine-grained rotated target detection network for SAR images based on the midpoint offset representation method, and designs a loss function for rotated target detection. The final model directly predicts the rotation box in a nearly cost-free manner, achieving high-performance detection based on a lightweight network. In particular, the detection accuracy for complex near-shore scenes in SAR images is significantly improved, and it is immune to the loss boundary discontinuity problem, achieving the effect of simple and efficient rotated target detection.

[0117] Furthermore, this embodiment verifies the SAR ship rotating target detection method through simulation experiments.

[0118] The simulation experiments used the SAR Ship Rotating Rectangular Box Detection Dataset (RSDD-SAR). This dataset contains SAR images from the Gaofen-3 and TerraSAR satellites, encompassing multiple imaging modes and polarization methods. The dataset contains 10,263 ship instances and 7,000 slices, including 5,000 training and 2,000 test images.

[0119] Simulation 1: Detection of the RSDD-SAR dataset using the single-stage rotation detection algorithm R3Det.

[0120] Simulation 2: Use the two-stage rotation detection algorithm Oriented RCNN to detect the RSDD-SAR dataset.

[0121] Simulation 3: The RSDD-SAR dataset is detected using the SAR ship rotating target detection method based on the improved YoloV5 of the present invention.

[0122] Simulation experiment results:

[0123] The number of parameters of the simulation 1 model is 60.8M, and the average precision AP of the test data is 80.92%, of which the nearshore AP is 57.73% and the offshore AP50 is 90.09%.

[0124] The number of parameters of the simulation 2 model is 60.34M, and the average precision AP of the test data is 88.85%, of which the nearshore AP is 66.77% and the offshore AP50 is 90.28%.

[0125] The number of parameters of the simulation 3 model is 7.2M, and the average precision AP of the test data is 90.0%, of which the nearshore AP is 76.3% and the offshore AP is 90.6%.

[0126] Observation results show that the number of model parameters in Simulation 3 is only one-tenth of that in Simulations 1 and 2. In terms of detection performance, the nearshore AP in Simulation 3 improves by 18.57% compared to Simulation 1 and 9.53% compared to Simulation 2. The offshore AP improves slightly compared to both Simulations 1 and 2. The overall AP improves by 9.08% compared to Simulation 1 and 1.15% compared to Simulation 2. This demonstrates the effectiveness of this method for detecting rotating ship targets in SAR images.

[0127] Simulations 1 and 2 illustrate that current mainstream rotating target detection algorithms, on the one hand, face significant challenges and low detection performance in complex nearshore scenarios; on the other hand, their large number of parameters makes them unsuitable for hardware deployment and incapable of addressing practical needs. This invention addresses these issues.

[0128] To sum up, the SAR ship rotating target detection method of this embodiment first uses the midpoint offset representation to express the rotated rectangular box, which can avoid the phenomenon of loss value boundary jumping caused by the angle, and performs adaptive rotation reconstruction on the basis of the YOLOv5 model. The data enhancement method and the cross-stage local network based on the attention mechanism enable the network to ensure the detection effect on a lightweight basis. The designed rotation network loss function enables the model to converge quickly, and finally directly predicts the rotating box in an almost cost-free manner, achieving fast and good detection. Especially for complex near-shore scenes, the performance of SAR image ship rotating target detection is greatly improved.

[0129] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A SAR ship rotating target detection method, characterized in that: Including steps: Obtain the ship data of the SAR image to be detected; Inputting the ship data to be detected in the SAR image into the trained rotating target detection model for inference to obtain the angle, position and category information of the ship target; The rotating target detection model is constructed based on a YOLOv5 network with a convolutional block attention mechanism and a midpoint offset representation of a rotated rectangular box. The trained rotating target detection model is trained based on a pre-constructed training set using a multi-task joint network loss function adapted for rotation detection. The pre-constructed training set is constructed based on original SAR image ship sample data and a label file containing a rotated rectangular box containing the ship target to be detected. The training set construction method includes: obtaining original SAR image ship sample data; cropping the original SAR image ship sample data according to a preset overlap ratio to obtain a plurality of slice images; marking each slice image with a rotated rectangular box containing a ship target to be detected, obtaining vertex coordinate values, categories, and detection difficulty of the rotated rectangular box, and forming a label file; and dividing the plurality of slice images and the label file into a training set and a test set; The rotating target detection model includes an input end, a backbone network, a neck network and a prediction end connected in sequence, wherein the input end is used to perform data enhancement processing on the plurality of slice images in the training phase, and convert the label file into the long side representation of the rotated rectangular box and the midpoint offset representation of the rotated rectangular box to obtain processed data; in the inference phase, the data enhancement processing is performed on the ship data of the SAR image to be detected to obtain processed data; the backbone network is used to perform feature extraction on the processed data to obtain feature information of different scales; the neck network is used to fuse shallow graphic information and deep semantic information from the backbone network to construct feature information with local detail and regional semantic representation capabilities; the prediction end is used to process the feature information output by the neck network and output feature maps of three different scales to classify targets of different scales and locate them based on the midpoint offset representation of the rotated rectangular box; The multi-task joint network loss function is: in, 、 、 is the network adaptation weight parameter, is the classification loss, is the confidence loss, Regression loss adapted to rotation detection; Used to calculate the positioning loss of the predicted rotated rectangular box and the corresponding true labeled rotated rectangular box: Among them, the regression loss uses the smooth-L1 function, is the offset of the predicted rotated rectangle relative to the anchor box, represents the sigmoid function, is the offset of the real annotation rotated rectangle relative to the anchor box; and The details are as follows: in, The coordinates of the center point of the real marked rotating rectangle. and is the width and height of the envelope rectangle of the real annotation rotation rectangle. and They are the offsets of the real annotation rotated rectangle relative to the upper midpoint and the right midpoint of its enveloping rectangle respectively; To predict the center coordinates of the rotating rectangle, and They are respectively the width and height of the envelope rectangle of the predicted rotation rectangle. and They are the offsets of the predicted rotating rectangle relative to the upper midpoint and the right midpoint of the envelope rectangle, represents the sigmoid function, and Respectively represent the position coordinates of the feature points in the feature map, and Represent the width and height of the anchor box respectively, is the stride of the feature map relative to the original image.

2. The SAR ship rotating target detection method according to claim 1, characterized in that: A convolution block attention mechanism module is inserted into the backbone network.

3. The SAR ship rotating target detection method according to claim 1, characterized in that: The midpoint offset of the rotated rectangular box includes the coordinates of the center point of the envelope rectangle of the rotated rectangular box, the width of the envelope rectangle of the rotated rectangular box, the height of the envelope rectangle of the rotated rectangular box, the offset of the upper corner point of the rotated rectangular box corresponding to the midpoint of the upper side of the envelope rectangle, and the offset of the right corner point of the rotated rectangular box corresponding to the midpoint of the right side of the envelope rectangle.

4. The SAR ship rotating target detection method according to claim 1, characterized in that: In each feature map output by the prediction end, the prediction format for each position is: ； in, represents the number of anchor boxes of different scales, Indicates the confidence that there is an object in the prediction box, Indicates the number of target categories, a 6-dimensional vector Indicates the offset of the predicted rotated rectangle relative to the anchor box under the midpoint offset representation of the rotated rectangle.

5. The SAR ship rotating target detection method according to claim 1, characterized in that: The training method of the rotating target detection model includes: Load pre-trained weight initialization parameters; The training samples in the training set are input into the rotation target detection model, and the rotation target detection model is trained using a multi-task joint network loss function. The model parameters are updated according to the gradient reverse iteration, and the multi-task joint network loss function is minimized to obtain a trained rotation target detection model.

6. The SAR ship rotating target detection method according to claim 1, characterized in that: The ship data of the SAR image to be detected is input into the trained rotating target detection model for inference to obtain the angle, position and category information of the ship target, including: Cutting the ship data of the SAR image to be detected into a plurality of slice images according to a preset overlapping ratio; Inputting the plurality of slice images into a trained rotation object detection model for inference, so that the prediction end outputs feature maps of three different scales, wherein the prediction format of each position in the feature map output by the prediction end includes the category of the corresponding object, the confidence level, and the offset of the predicted rotated rectangular box relative to the anchor box using the midpoint offset representation of the rotated rectangular box; The prediction end decodes the offset of the predicted rotated rectangular frame relative to the anchor frame to obtain a predicted rotated rectangular frame represented by a midpoint offset; Converting the predicted rotated rectangular frame represented by the midpoint offset into a predicted rotated rectangular frame represented by the long side; The predicted rotated rectangular frame represented by the long side is subjected to non-maximum suppression (NMS) processing to obtain the angle, position and category information of the ship target.

7. The SAR ship rotating target detection method according to claim 6, characterized in that: The process of decoding the offset of the predicted rotated rectangular frame relative to the anchor frame to obtain the predicted rotated rectangular frame represented by the midpoint offset is: in, To predict the center coordinates of the rotating rectangle, and They are respectively the width and height of the envelope rectangle of the predicted rotation rectangle. and They are the offsets of the predicted rotating rectangle relative to the upper midpoint and the right midpoint of the envelope rectangle, represents the sigmoid function, and Respectively represent the position coordinates of the feature points in the feature map, and Represent the width and height of the anchor box respectively, is the stride of the feature map relative to the original image.

Images