A multi-class SAR image ship detection method based on a rotation detection frame

Abstract

The application discloses a multi-class SAR image ship detection method based on a rotating detection frame, which is based on a deep learning theory and mainly comprises a rotating detection frame mechanism, a balanced self-attention pyramid, feature alignment and a rotating regression network with a feature decoupling mechanism. The rotating detection frame mechanism reduces the overlapping part of adjacent ship detection frames by using a rotating detection frame and reduces the interference of the background, thereby improving the detection precision; the balanced self-attention pyramid network extracts features with more multi-scale detection capabilities through a feature fusion and enhancement method, thereby solving the imbalance problem of ship scale features; the feature alignment aligns the rotating detection frame and the horizontal feature map, thereby solving the misalignment problem between the rotating detection frame and the feature map; and the rotating classification regression network contains a feature decoupling mechanism, thereby solving the classification regression sensitivity conflict and the multi-class ship detection problem.

Description

Technical Field

[0001] This invention belongs to the field of Synthetic Aperture Radar (SAR) image interpretation technology, and relates to a method for ship detection in multi-category SAR images based on a rotating detection box. Background Technology

[0002] Synthetic Aperture Radar (SAR) is an advanced active microwave sensor used for high-resolution Earth observation, capable of operating around the clock and in all weather conditions. Currently, SAR has wide applications in marine monitoring, such as maritime traffic control, disaster relief, and fisheries management. While optical or hyperspectral satellites provide some monitoring services, the complex and variable marine environment makes it difficult for them to provide accurate information under many weather conditions. Therefore, SAR is an indispensable remote sensing tool for marine area perception.

[0003] Maritime ship detection has immense value in areas such as shipwreck rescue, maritime traffic control, and fisheries management. Therefore, the detection of maritime ship targets has long been a focus of research in marine area perception. Since the launch of the first SAR satellite, Seasonat-1, by the United States, research on marine ship monitoring based on SAR images has developed rapidly. Currently, with the increasing number of SAR sensors, a large amount of data has been generated, making intelligent detection of marine targets urgently needed. See the literature "Wang Zhiyong, Dou Hao, Tian Jinwen. Research on rapid detection methods for ship targets in SAR images [J]. Ship Electronic Engineering, 2016, 36(09):27-30+88."

[0004] In recent years, with the rapid rise of deep learning (DL), many scholars in the SAR field have begun to study DL-based detection methods. Compared with traditional feature-based methods, DL-based methods have outstanding advantages such as simplicity, full automation (i.e., no need for complex land-sea segmentation, coastline detection, speckle correction, and other basic stages), high speed, and high accuracy. Although their underlying principles have not yet been fully understood, they can liberate productivity and greatly improve work efficiency. This has enabled a qualitative leap in the intelligent interpretation of SAR images. See "Du Lan, Wang Zhaocheng, Wang Yan, Wei Di, Li Lu. A review of research progress on single-channel SAR target detection and identification in complex scenes [J]. Journal of Radar, 2020, 9(01):34-54."

[0005] However, existing deep learning-based SAR ship detectors have several problems that potentially hinder further improvements in accuracy. Specifically: 1) When detecting densely packed ships near the shore, horizontal bounding boxes are suppressed during non-maximum suppression due to significant overlap, leading to missed detections. 2) Ship scale features are imbalanced, i.e., multi-scale ship features are unbalanced. For dynamic ship detection, ship sizes vary greatly due to different spatial resolutions and ship categories. 3) The opposing spatial sensitivity requirements of classification and regression tasks hinder further improvements in detection and classification accuracy. 4) Most existing networks do not achieve multi-category SAR ship detection and classification.

[0006] Therefore, to address the aforementioned issues, this paper proposes a multi-class SAR image ship detection method based on rotated detection boxes. This method includes a rotated detection box mechanism, a balanced self-attention pyramid, feature alignment, and a rotational regression network incorporating feature decoupling mechanisms. Experimental results on the SRSDD dataset demonstrate that the proposed method outperforms other deep learning-based detection methods. Summary of the Invention

[0007] This invention belongs to the field of synthetic aperture radar (SAR) image interpretation technology and discloses a multi-class SAR image ship detection method based on rotated detection boxes. This method addresses the problems of dense ship missed detection, ship scale imbalance, classification-regression sensitivity conflicts, and multi-class ship detection in existing deep learning-based SAR ship detection methods. Based on deep learning theory, this method mainly includes a rotated detection box mechanism, a balanced self-attention pyramid, feature alignment, and a rotated regression network with a feature decoupling mechanism. The rotated detection box mechanism reduces the overlap between adjacent ship detection boxes and reduces background interference, thereby improving detection accuracy. The balanced self-attention pyramid network extracts features with greater multi-scale detection capabilities through feature fusion and enhancement methods, thus solving the ship scale feature imbalance problem. Feature alignment aligns the rotated detection boxes with the horizontal feature maps, thus solving the misalignment problem between the rotated detection boxes and the feature maps. The rotated classification-regression network includes a feature decoupling mechanism, resolving classification-regression sensitivity conflicts and the problem of detecting multiple ship classes. Experiments show that on the SRSDD dataset, the ship detection method based on rotating detection boxes for multi-class SAR images has an average detection accuracy of 63.42%, while other existing deep learning-based SAR ship detection methods have an accuracy of 56.23%. The ship detection method based on rotating detection boxes for multi-class SAR images improves the ship detection accuracy.

[0008] To facilitate the description of the present invention, the following terms are defined first:

[0009] Definition 1: Method for obtaining the SRSDD dataset

[0010] The SRSDD dataset refers to the SAR Rotation Ship Detection Dataset. It contains 666 SAR images from the Gaofen-3 satellite, each with a resolution of 1024×1024 pixels. The SRSDD dataset includes 2884 ships across six ship types: oil tankers (166), bulk carriers (2053), fishing vessels (288), law enforcement vessels (25), container ships (89), and dredgers (263). 532 images were selected as the training set, and the remaining 134 images were used as the test set. The SRSDD dataset can be obtained from the reference "Lei, S.;Lu, D.;Qiu, X.;Ding, C.SRSDD-v1.0:AHigh-ResolutionSAR Rotation Ship Detection Dataset.Remote Sens.2021,13,5104."

[0011] Definition 2: Downsampling method

[0012] Downsampling is a method of reducing the size of an image or feature map. For an image I of size M*N, downsampling by a factor of s yields an image with a resolution of (M / s)*(N / s), where s is the common divisor of M and N. If considering a matrix image, this means transforming the image within an s*s window of the original image into a single pixel, and the value of this pixel is the average of all pixels within the window. For a detailed explanation of classic downsampling operations, see "https: / / blog.csdn.net / liulina603 / article / details / 47727277".

[0013] Definition 3: Feature Alignment Network Construction Method

[0014] The feature alignment network is a type of network based on rotational convolution that aligns rotated anchor points with feature maps. It first extracts features using two convolutional layers, then uses a regression prediction head to predict the offset of the rotated anchor points relative to the feature map. Finally, the offset is added to the convolution to obtain the aligned feature map. For details on the construction method of the feature alignment network, please refer to "Han J, Ding J, Li J, et al. Align deep features for oriented object detection[J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:1-11."

[0015] Definition 4: The classic Adam algorithm

[0016] The classic Adam algorithm is an extension of stochastic gradient descent and has recently been widely used in deep learning applications in computer vision and natural language processing. Classic Adam differs from classic stochastic gradient descent. Stochastic gradient descent maintains a single learning rate for all weight updates, and this learning rate remains unchanged during training. Each network weight, however, maintains a learning rate that is adjusted individually as learning progresses. This method computes adaptive learning rates for different parameters from the budgets of the first and second moments of the gradient. For a detailed explanation of the classic Adam algorithm, see "Kingma, D.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2014, arXiv:1412.6980."

[0017] Definition 5: Forward Propagation Method

[0018] Forward propagation is one of the most fundamental methods in deep learning. It involves performing forward reasoning on the input based on the network's parameters and connection methods to obtain the network's output. For details on forward propagation, please see "https: / / www.jianshu.com / p / f30c8daebebb".

[0019] Definition 6: Classical Residual Network Construction Method

[0020] Residual networks are convolutional neural networks proposed by four researchers from Microsoft Research. They won the image classification and object recognition categories in the 2015 ImageNet Large Scale Visual Recognition Challenge (ILSVRC). Residual networks are characterized by their ease of optimization and the ability to improve accuracy by increasing depth. Their internal residual blocks use skip connections, mitigating the vanishing gradient problem inherent in increasing depth in deep neural networks. For a detailed explanation of classic residual network construction methods, see "K. He et al., 'Deep Residual Learning for Image Recognition,' IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770-778."

[0021] Definition 7: Convolutional Kernel Construction Method

[0022] A convolutional kernel is a node that performs weighted summation of values ​​within a small rectangular region of the input feature map or image as the output. Each convolutional kernel requires multiple manually specified parameters. One type of parameter is the length and width of the node matrix processed by the kernel; the size of this node matrix is ​​also the size of the convolutional kernel. Another type of parameter is the depth of the resulting unit node matrix; the depth of the unit node matrix is ​​also the depth of the convolutional kernel. During the convolution operation, each convolutional kernel slides across the input data, calculates the inner product of the entire kernel with the corresponding positions in the input data, and then applies the inner product through a nonlinear function to obtain the final result. Finally, the results from all corresponding positions form a two-dimensional feature map. Each convolutional kernel generates a two-dimensional feature map, and the feature maps generated by multiple convolutional kernels are superimposed to form a three-dimensional feature map. For details on convolutional kernel operations, see "Fan Lili, Zhao Hongwei, Zhao Haoyu, Hu Huangshui, Wang Zhen. A Review of Research on Object Detection Based on Deep Convolutional Neural Networks [J]. Optics and Precision Engineering, 2020, 28(05):1152-1164."

[0023] Definition 8: Upsampling method

[0024] Upsampling is an operation that modifies an image or feature map. The mainstream upsampling operation typically uses interpolation methods, which insert new elements between pixels in the original image using a suitable interpolation algorithm. Among the mainstream interpolation algorithms, neighbor-to-neighbor interpolation is relatively simple and easy to implement, and was widely used in the past. However, this method can produce noticeable jagged edges and mosaic effects in the new image. Bilinear interpolation has a smoothing function and can effectively overcome the shortcomings of neighbor-to-neighbor interpolation, but it degrades the high-frequency components of the image, blurring image details. At higher magnifications, higher-order interpolation, such as bicubic and cubic spline interpolation, performs better than lower-order interpolation. These interpolation algorithms can ensure that the gray values ​​of the interpolated pixels continue the continuity of gray-level changes in the original image, thus making the changes in shades in the magnified image natural and smooth. However, in an image, some pixels have abrupt changes in gray-level values ​​with their neighboring pixels, i.e., there are gray-level discontinuities. These pixels with abrupt changes in gray-level values ​​are the edge pixels that describe the contours or textures of objects in the image. For details on classic upsampling operations, please refer to "https: / / blog.csdn.net / weixin_43960370 / article / details / 106049708?utm_term=%E5%8D%B7%E7%A7%AF%E7%89%B9%E5%BE%81%E5%9B%BE%E4%B8%8A%E9%87%87%E6%A0%B7&utm_medium=distribute.pc_aggpage_search_result.none-task-blog-2~all~sobaiduweb~default-1-106049708&spm=3001.4430".

[0025] Definition 9: Pooling Operation Method

[0026] Pooling is a very common operation in CNNs. Pooling layers mimic the human visual system to reduce the dimensionality of data. It's also commonly called subsampling or downsampling. In building convolutional neural networks, it's often used after convolutional layers to reduce the feature dimension of the convolutional layer's output, effectively reducing network parameters and preventing overfitting. For a detailed explanation of classic pooling operations, see "https: / / www.zhihu.com / question / 303215483 / answer / 615115629".

[0027] Definition 10: Deconvolution Method

[0028] The Region Proposal Network (RPN) is a subnetwork in Faster R-CNN used to extract regions in an image where objects may exist. The RPN is a fully convolutional network that takes the convolutional feature map output from the base network as input and outputs the object confidence score for each candidate box. For details on the construction method of the RPN, please refer to "Ren S, He K, Girshick R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149."

[0029] Definition 11: Traditional Fully Connected Layer Method

[0030] Fully connected layers are part of convolutional neural networks. The input and output dimensions of a fully connected layer are fixed, and each node is connected to all nodes in the previous layer, used to synthesize the features extracted earlier. For details on fully connected layer methods, see "Haoren Wang, Haotian Shi, Ke Lin, Chengjin Qin, Liqun Zhao, Yixiang Huang, Chengliang Liu. A high-precision arrhythmia classification method based on dual fully connected neural network[J]. Biomedical Signal Processing and Control, 2020, 58."

[0031] Definition 12: Nonmaximum suppression method

[0032] Non-maximum suppression (NMS) is an algorithm used in object detection to remove redundant bounding boxes. In the forward propagation results of classic detection networks, multiple bounding boxes often correspond to the same object. Therefore, an algorithm is needed to select the highest-quality bounding box with the highest score from these multiple boxes. NMS performs local maximum search by calculating an overlap rate threshold. For details on NMS, see "https: / / www.cnblogs.com / makefile / p / nms.html".

[0033] Definition 13: Methods for calculating recall and precision

[0034] Recall R refers to the number of correctly predicted samples out of all positive samples, expressed as: Precision P refers to the proportion of correct predictions out of the total number of predictions that are positive. The expression is: Wherein, TP (true positive) represents a positive sample predicted as positive by the model; FN (false negative) represents a negative sample predicted as negative by the model; and FP (false positive) represents a positive sample predicted as negative by the model. The recall and precision curves P(R) refer to functions with R as the independent variable and P as the dependent variable. For details on how to calculate the values ​​of the above parameters, please refer to the reference "Li Hang. Statistical Learning Methods [M]. Beijing: Tsinghua University Press, 2012."

[0035] Definition 14: Method for constructing a rotating detection head

[0036] A rotating detection head is a technology that integrates rotating anchor points into a traditional detection head. It achieves the detection of rotating targets by setting eight anchor points in different directions and multiple anchor points of varying sizes. Unlike traditional horizontal anchor points, rotating anchor points can better conform to targets in any direction. For details on how to construct a rotating detection head, please see "https: / / zhuanlan.zhihu.com / p / 346460693".

[0037] Definition 15: Pyramid Network Construction Method

[0038] Pyramid networks are networks that scale feature maps to different resolutions and then perform corresponding operations on each resolution. Pyramid networks allow multi-scale feature maps to better and more accurately describe targets of different sizes. They are currently an important component in object detection, semantic segmentation, and action recognition, demonstrating good performance in improving model performance. In feature extraction, low-level features have less semantic information but accurate target location and high resolution. High-level features have richer semantic information but coarser target location, lower resolution, and are more abstract. Therefore, some algorithms use multi-scale feature fusion to make predictions based on the fused features. Pyramid network predictions are performed independently at different feature layers; that is, they simultaneously utilize the high resolution of low-level features and the high semantic information of high-level features, achieving predictive accuracy by fusing these features from different layers. For details on the pyramid network construction method, please refer to "https: / / blog.csdn.net / qq_41204464 / article / details / 119063011?utm_medium=distribute.pc_aggpage_search_result.none-task-blog-2~aggregatepage~first_rank_ecpm_v1~rank_v31_ecpm-8-119063011-null-null.pc_agg_new_rank&utm_term=%E7%89%B9%E5%BE%81%E9%87%91%E5%AD%97%E5%A1%94%E7%BD%91%E7%BB%9C&spm=1000.2123.3001.4430".

[0039] This invention provides a method for ship detection in multi-class SAR images based on a rotated detection box, which includes the following steps:

[0040] Step 1: Initialize the dataset

[0041] The SAR images in the SRSDD dataset are ordered using a traditional random method to obtain a new SRSDD dataset. The SRSDD dataset is then divided into a training set (Train) and a test set (Test) using the method described in Definition 1.

[0042] Step 2: Build a forward propagation network

[0043] Step 2.1: Construct a balanced self-attention pyramid network

[0044] A residual network with 50 layers, denoted as Res-50, is constructed using the classic residual network construction method in Definition 6. The feature maps of different sizes generated by the last layer of the residual network Res-50 are denoted as feature map F1, feature map F2, feature map F3, feature map F4, and feature map F5, respectively, in descending order of feature map size.

[0045] For feature maps F1, F2, F3, F4, and F5, the formula is used. The calculation yields a feature map, denoted as H, where k represents the index of the feature map and (i,j) represents the spatial sampling position of the feature map.

[0046] The feature map H is used as input, and the formula is applied. The feature map O is obtained through calculation. Where H... i This represents the feature at the i-th position on the feature map H; O i This represents the feature at the i-th position on the feature map O; Represents the normalization factor; f(H) i H j ) is used to calculate H i and H j The function of similarity between them is expressed as follows: Wherein, θ(H i ) = W θ H i ,φ(H j ) = W φ H j W θ and W φ The matrix obtained by performing a 1×1 convolution kernel operation using the traditional convolution kernel operation method defined in 7; g(H) j ) = W g H j W g It is the matrix obtained by operating a 1×1 convolution kernel using the traditional convolution kernel operation method in Definition 7; the feature map O is denoted as feature map E3.

[0047] Using feature map O as input, the traditional downsampling operation method in Definition 2 is adopted. The size of feature map O is made consistent with feature map F5 through downsampling operation, resulting in a new feature map, denoted as feature map E5.

[0048] Using feature map O as input, the traditional downsampling operation method in Definition 2 is adopted. The size of feature map O is made consistent with feature map F4 through downsampling operation, resulting in a new feature map, denoted as feature map E4.

[0049] Using feature map O as input, the traditional upsampling operation method in Definition 8 is adopted. The size of feature map O is made consistent with feature map F2 through the upsampling operation, resulting in a new feature map, denoted as feature map E2.

[0050] Using feature map O as input, the traditional upsampling operation method in Definition 8 is adopted. The size of feature map O is made consistent with feature map F1 through the upsampling operation, resulting in a new feature map, denoted as feature map E1.

[0051] For feature maps E1, E2, E3, E4, and E5, the traditional pyramid network method in Definition 15 is used to construct a balanced feature pyramid network, denoted as BAFPN0.

[0052] Step 2.2: Construct a feature alignment network

[0053] Using the traditional feature alignment network construction method in Definition 3, a feature alignment network, denoted as FAN0, is constructed. The output of FAN0 is denoted as F... A .

[0054] Step 2.3: Construct a rotational classification and regression network

[0055] Pooling layer P1 is constructed using the traditional pooling operation method in Definition 9. Fully connected layers FC1, FC2, and FC3 are constructed using the traditional fully connected layer method in Definition 11. Four convolutional layers, Conv11, Conv12, Conv21, and Conv22, are constructed using the traditional convolutional kernel method in Definition 7. Two rotating detector heads, denoted as RC and RR, are constructed using the traditional rotating detector head construction method in Definition 14. F... A The output of pooling layer P1 is used as the input to fully connected layer FC1, and the output of fully connected layer FC1 is denoted as F. E F E As the input to the fully connected layer FC2, let the output of the fully connected layer FC2 be F. D1 F E As the input to the fully connected layer FC3, let the output of the fully connected layer FC3 be F. D2 F A and F D1 Multiply, and denote the output as F. cls F A and F D2 Multiply, and denote the output as F. reg F cls The output of convolutional layer Conv11 is used as the input to convolutional layer Conv12, and the output of convolutional layer Conv12 is used as the input to the rotation detection head RC. The output of the rotation detection head RC is denoted as OUT.cls F reg The output of convolutional layer Conv21 is used as the input of convolutional layer Conv22, and the output of convolutional layer Conv22 is used as the input of rotation detection head RR. The output of rotation detection head RR is denoted as OUT. reg .

[0056] The rotating classification and regression network, denoted as RDN0, is composed of the pooling layer P1, fully connected layers FC1, FC2, and FC3, convolutional layers Conv11, Conv12, Conv21, and Conv22, and rotating detection heads RC and RR from step 2.3.

[0057] Step 3: Train the network

[0058] Set the iteration parameter epoch, and initialize the epoch value to 1.

[0059] Step 3.1: Perform forward propagation on the balanced self-attention pyramid network.

[0060] The training set Train of the dataset SRSDD obtained in step 1 is used as the input of the Balanced Self-Attention Pyramid Network BAFPN0. The traditional forward propagation method in Definition 5 is used to feed the training set Train into the Balanced Self-Attention Pyramid Network BAFPN0 for computation, and the output of the network BAFPN0 is obtained, denoted as Result0.

[0061] Step 3.2: Perform forward propagation on the feature alignment network.

[0062] The output result Result0 of the balanced self-attention pyramid network obtained in step 3.1 is used as the input of the feature alignment network FAN0. The traditional forward propagation method in Definition 5 is used to feed the output result Result0 of the balanced self-attention pyramid network into the feature alignment network FAN0 for operation, and the output of the feature alignment network FAN0 is obtained, which is denoted as Result1.

[0063] Step 3.3: Perform forward propagation on the rotated classification and regression network.

[0064] The output result Result1 of the feature alignment network obtained in step 3.2 is used as the input of the rotated classification and regression network RDN0. The traditional forward propagation method in Definition 5 is used to feed the output result Result1 of the feature alignment network into the rotated classification and regression network RDN0 for computation, and the output of the rotated classification and regression network RDN0 is obtained, denoted as Result. cls and Result reg .

[0065] Step 3.4: Train and optimize all networks.

[0066] The output result of the rotated classification and regression network RDN0 obtained in step 3.3 is Result cls and Result reg As input, the classic Adam algorithm in Definition 4 is used to train and optimize the rotated classification and regression network RDN0, the feature alignment network FAN0, and the balanced self-attention pyramid network BAFPN0, resulting in the trained and optimized rotated classification and regression network RDN1, feature alignment network FAN1, and balanced self-attention pyramid network BAFPN1.

[0067] Step 4: Perform alternating training

[0068] Determine if the epoch set in step 3 is equal to 12.

[0069] If epoch is not equal to 12, then let epoch = epoch + 1, BAFPN0 = BAFPN1, FAN0 = FAN1, RDN0 = RDN1, and repeat steps 3.1, 3.2, 3.3, and 3.4 in sequence, and then return to step 4 to judge epoch again;

[0070] If the epoch equals 12, then let the trained rotational classification and regression network RDN1, feature alignment network FAN1, and balanced self-attention pyramid network BAFPN1 be denoted as network RBFA-Net, and then proceed to step 5.

[0071] Step 5, Evaluation Method

[0072] Step 5.1, Forward Propagation

[0073] Using the network RBFA-Net obtained in step 4 and the test set Test obtained in step 1 as input, the traditional forward propagation method defined in definition 5 is used for detection, and the detection result is denoted as R.

[0074] Using the detection result R as input, the traditional nonmaximum suppression method in Definition 12 is used to remove redundant boxes in the detection result R. The specific steps are as follows:

[0075] Step (1) First, let the highest-scoring bounding box in the detection result R be denoted as BS;

[0076] Step (2) Then the calculation formula is as follows: Calculate the overlap ratio threshold (IoU) of all bounding boxes in the detection result R; discard boxes with IoU > 0.5;

[0077] Step (3) Select the highest-scoring box (BS) from the remaining boxes;

[0078] Repeat the process of calculating IoU and discarding boxes in step (2) above until there are no more boxes to discard. The remaining boxes are the final detection results, denoted as R. F .

[0079] Step 5.2, Calculate the indicators

[0080] The detection result R obtained in step 5.1 F As input, the traditional recall and precision calculation methods in Definition 13 are used to obtain the network's precision P, recall R, and precision-recall curve P(R); the formula is used... Calculate the average accuracy (mAP) of SAR ship detection in multi-class SAR images with rotated detection boxes.

[0081] The innovation of this invention lies in the introduction of a rotated detection box, balancing the self-attention pyramid, rotating feature alignment, and feature decoupling, thereby solving the problems of dense ship missed detection, ship scale imbalance, classification regression sensitivity conflict, and multi-class ship detection in existing deep learning-based SAR ship detection methods. The mAP of SAR image ship detection using this method is 63.42%, exceeding the suboptimal SAR image ship detector by 7.19 percentage points.

[0082] The advantage of this invention lies in its ability to overcome the multi-category ship detection problem in existing technologies and improve the average detection accuracy of ships in SAR images. A multi-category SAR image ship detection method based on a rotated detection box. Attached Figure Description

[0083] Figure 1 This is a flowchart illustrating the multi-category SAR image ship detection method based on a rotating detection box in this invention.

[0084] Figure 2 This is a schematic diagram of the balanced self-attention pyramid network in the multi-class SAR image ship detection method based on rotating detection boxes in this invention.

[0085] Figure 3 This is a schematic diagram of the rotational classification regression network in the multi-class SAR image ship detection method based on rotating detection boxes in this invention.

[0086] Figure 4 This refers to the detection accuracy of the multi-category SAR image ship detection method based on rotating detection boxes in this invention. Detailed Implementation

[0087] The following is in conjunction with the appendix Figure 1 Appendix Figure 2 Appendix Figure 3 Appendix Figure 4 The present invention will be described in further detail.

[0088] Step 1: Initialize the dataset

[0089] like Figure 1 As shown, the order of SAR images in the SRSDD dataset is randomly adjusted to obtain a new SRSDD dataset. Using the SRSDD dataset acquisition method in Definition 1, the SRSDD dataset is divided into a training set (Train) and a test set (Test).

[0090] Step 2: Build a forward propagation network

[0091] Step 2.1: Construct a balanced self-attention pyramid network

[0092] like Figure 2 As shown, a residual network with 50 layers, denoted as Res-50, is constructed using the classic residual network construction method in Definition 6. Meanwhile, the feature maps of different sizes generated by the last layer of the residual network Res-50 are denoted as feature map F1, feature map F2, feature map F3, feature map F4, and feature map F5, respectively, in descending order of feature map size.

[0093] Feature maps F1, F2, F3, F4, and F5 are used with the formula The calculation yields a feature map, denoted as H, where k represents the index of feature map H and (i,j) represents the spatial sampling position of the feature map.

[0094] The feature map H is used as input, and the formula is applied. The feature map O is obtained through calculation. Where H... i This represents the feature at the i-th position on the feature map H; O i This represents the feature at the i-th position on the feature map O; Represents the normalization factor; f(H) i H j ) is used to calculate H i and H j The function of similarity between them is expressed as follows: Wherein, θ(H i ) = W θ H i ,φ(H j ) = W φ H j W θ and W φ It is a matrix learned through the 1×1 convolution operation in Definition 7; g(H j ) = W g H j W gIt is a matrix learned through the 1×1 convolution operation in Definition 7. Feature map O is further denoted as feature map E3.

[0095] Using feature map O as input, the downsampling operation method in Definition 2 is adopted to make the size of feature map O consistent with feature map F5, resulting in a new feature map, denoted as feature map E5.

[0096] Using feature map O as input, the downsampling operation method in Definition 2 is adopted to make the size of feature map O consistent with feature map F4, resulting in a new feature map, denoted as feature map E4.

[0097] Using feature map O as input, the upsampling operation method in Definition 8 is adopted to make the size of feature map O consistent with feature map F2, resulting in a new feature map, denoted as feature map E2.

[0098] Using feature map O as input, the upsampling operation method in Definition 8 is adopted to make the size of feature map O consistent with feature map F1, resulting in a new feature map, denoted as feature map E1.

[0099] For feature map E1, feature map E2, feature map E3, feature map E4, feature map E 5， Using the pyramid network method in Definition 15, a balanced feature pyramid network, denoted as BAFPN0, is obtained.

[0100] Step 2.2: Construct a feature alignment network

[0101] Using the feature alignment network construction method in Definition 3, a feature alignment network, denoted as FAN0, is constructed. The output of FAN0 is denoted as F... A .

[0102] Step 2.3: Construct a rotational classification and regression network

[0103] like Figure 3 As shown, pooling layer P1 is constructed using the pooling operation method in Definition 9. Fully connected layers FC1, FC2, and FC3 are constructed using the traditional fully connected layer method in Definition 11. Four convolutional layers, Conv11, Conv12, Conv21, and Conv22, are constructed using the convolutional kernel method in Definition 7. Two rotating detector heads, denoted as RC and RR, are constructed using the rotating detector head construction method in Definition 14. F... A The output of pooling layer P1 is used as the input to fully connected layer FC1, and the output of fully connected layer FC1 is denoted as F. E F E As the input to the fully connected layer FC2, let the output of the fully connected layer FC2 be F. D1 FE As the input to the fully connected layer FC3, let the output of the fully connected layer FC3 be F. D2 F A and F D1 Multiply, and denote the output as F. cls F A and F D2 Multiply, and denote the output as F. reg F cls The output of convolutional layer Conv11 is used as the input to convolutional layer Conv12, and the output of convolutional layer Conv12 is used as the input to the rotation detection head RC. The output of the rotation detection head RC is denoted as OUT. cls F reg The output of convolutional layer Conv21 is used as the input of convolutional layer Conv22, and the output of convolutional layer Conv22 is used as the input of rotation detection head RR. The output of rotation detection head RR is denoted as OUT. reg .

[0104] The rotating classification and regression network, denoted as RDN0, is composed of the pooling layer P1, fully connected layers FC1, FC2, and FC3, convolutional layers Conv11, Conv12, Conv21, and Conv22, and rotating detection heads RC and RR from step 2.3.

[0105] Step 3: Train the network

[0106] Set the iteration parameter epoch, and initialize the epoch value to 1.

[0107] Step 3.1: Perform forward propagation on the balanced self-attention pyramid network.

[0108] The training set Train of the dataset SRSDD obtained in step 1 is used as the input of the Balanced Self-Attention Pyramid Network BAFPN0. The forward propagation method in Definition 5 is used to feed the training set Train into the Balanced Self-Attention Pyramid Network BAFPN0 for computation, and the output of the network BAFPN0 is obtained, denoted as Result0.

[0109] Step 3.2: Perform forward propagation on the feature alignment network.

[0110] The output Result0 of the balanced self-attention pyramid network obtained in step 3.1 is used as the input of the feature alignment network FAN0. The forward propagation method in Definition 5 is used to feed the output Result0 of the balanced self-attention pyramid network into the feature alignment network FAN0 for operation, and the output of the feature alignment network FAN0 is obtained, which is denoted as Result1.

[0111] Step 3.3: Perform forward propagation on the rotated classification and regression network.

[0112] The output result Result1 of the feature alignment network obtained in step 3.2 is used as the input of the rotated classification and regression network RDN0. The forward propagation method in Definition 5 is used to feed the output result Result1 of the feature alignment network into the rotated classification and regression network RDN0 for computation, and the output of the rotated classification and regression network RDN0 is obtained, which is denoted as Result. cls and Result reg .

[0113] Step 3.4: Train and optimize all networks.

[0114] The output result of the rotated classification and regression network RDN0 obtained in step 3.3 is Result cls and Result reg As input, the classic Adam algorithm in Definition 4 is used to train and optimize the rotated classification and regression network RDN0, the feature alignment network FAN0, and the balanced self-attention pyramid network BAFPN0, resulting in the trained and optimized rotated classification and regression network RDN1, feature alignment network FAN1, and balanced self-attention pyramid network BAFPN1.

[0115] Step 4: Perform alternating training

[0116] Determine if the epoch set in step 3 is equal to 12. If the epoch is not equal to 12, then set epoch = epoch + 1, BAFPN0 = BAFPN1, FAN0 = FAN1, and RDN... 0＝ For RDN1, repeat steps 3.1, 3.2, 3.3, and 3.4 in sequence, and then return to step 4 to re-evaluate the epoch. If the epoch equals 12, then let the trained rotational classification and regression network RDN1, feature alignment network FAN1, and balanced self-attention pyramid network BAFPN1 be denoted as network RBFA-Net, and then proceed to step 5.

[0117] Step 5, Evaluation Method

[0118] Step 5.1, Forward Propagation

[0119] Using the network RBFA-Net obtained in step 4 and the test set Test obtained in step 1 as input, the traditional forward propagation method defined in definition 5 is used for detection, and the detection result is denoted as R.

[0120] Using the detection result R as input, the traditional nonmaximum suppression method in Definition 12 is used to remove redundant boxes in the detection result R. The specific steps are as follows:

[0121] Step (1) First, let the highest-scoring bounding box in the detection result R be denoted as BS;

[0122] Step (2) Then the calculation formula is as follows: Calculate the overlap ratio threshold (IoU) of all bounding boxes in the detection result R; discard boxes with IoU > 0.5;

[0123] Step (3) Select the highest-scoring box (BS) from the remaining boxes;

[0124] Repeat the process of calculating IoU and discarding boxes in step (2) above until there are no more boxes to discard. The remaining boxes are the final detection results, denoted as R. F .

[0125] Step 5.2, Calculate the indicators

[0126] The detection result R obtained in step 5.1 F As input, the traditional recall and precision calculation methods in Definition 13 are used to obtain the network's precision P, recall R, and precision-recall curve P(R); the formula is used... like Figure 4 As shown, the mean accuracy (mAP) of SAR ship detection in multi-class SAR images with rotating detection boxes is calculated.

Claims

1. A method for ship detection in multi-class SAR images based on a rotated detection box, characterized in that, Includes the following steps: Step 1: Initialize the dataset The SAR images in the SRSDD dataset were ordered using a traditional random method to obtain a new SRSDD dataset. The SRSDD dataset was then divided into a training set (Train) and a test set (Test) using the traditional SRSDD dataset acquisition method. Step 2: Build a forward propagation network Step 2.1: Construct a balanced self-attention pyramid network A 50-layer residual network, denoted as Res-50, is constructed using the classic residual network construction method. The feature maps of different sizes generated by the last layer of the Res-50 residual network are then denoted as feature maps F, in descending order of size. 1， Feature map F 2， Feature map F 3， Feature map F4, feature map F5; For feature maps F1, F2, F3, F4, and F5, the formula is used. The calculation yields a feature map, denoted as H, where k represents the index of the feature map and (i,j) represents the spatial sampling position of the feature map. The feature map H is used as input, and the formula is applied. The feature map O is obtained through calculation; where H i This represents the feature at the i-th position on the feature map H; O i This represents the feature at the i-th position on the feature map O; Represents the normalization factor; f(H) i H j ) is used to calculate H i and H j The function of similarity between them is expressed as follows: Wherein, θ(H i ) = W θ H i ,φ(H j ) = W φ H j W θ and W φ The matrix obtained by performing a 1×1 convolution kernel operation using traditional convolution kernel operation methods; g(H j ) = W g H j W g It is a matrix obtained by operating a 1×1 convolution kernel using traditional convolution kernel operations; feature map O is denoted as feature map E3; Using feature map O as input, a traditional downsampling operation is performed to make the size of feature map O consistent with feature map F5, resulting in a new feature map, denoted as feature map E5. Using feature map O as input, a traditional downsampling operation is performed to make the size of feature map O consistent with feature map F4, resulting in a new feature map, denoted as feature map E4. Using feature map O as input, the traditional upsampling operation method is used to make the size of feature map O consistent with feature map F2, resulting in a new feature map, denoted as feature map E2. Using feature map O as input, the traditional upsampling operation method is used to make the size of feature map O consistent with feature map F1, resulting in a new feature map, denoted as feature map E1. For feature maps E1, E2, E3, E4, and E5, the traditional pyramid network method is used to construct a balanced self-attention pyramid network, denoted as BAFPN0. Step 2.2: Construct a feature alignment network A feature alignment network, denoted as FAN0, is constructed using a traditional feature alignment network construction method; the output of FAN0 is denoted as F. A ; Step 2.3: Construct a rotational classification and regression network Pooling layer P1 is constructed using traditional pooling operations; fully connected layers FC1, FC2, and FC3 are constructed using traditional fully connected layer methods; four convolutional layers, Conv11, Conv12, Conv21, and Conv22, are constructed using traditional convolutional kernel methods; two rotating detector heads, denoted as RC and RR, are constructed using traditional rotating detector head construction methods; F... A The output of pooling layer P1 is used as the input to fully connected layer FC1, and the output of fully connected layer FC1 is denoted as F. E ; F E As the input to the fully connected layer FC2, let the output of the fully connected layer FC2 be F. D1 ; F E As the input to the fully connected layer FC3, let the output of the fully connected layer FC3 be F. D2 ; F A and F D1 Multiply, and denote the output as F. cls ; F A and F D2 Multiply, and denote the output as F. reg ; F cls The output of convolutional layer Conv11 is used as the input to convolutional layer Conv12, and the output of convolutional layer Conv12 is used as the input to the rotation detection head RC. The output of the rotation detection head RC is denoted as OUT. cls ; F reg The output of convolutional layer Conv21 is used as the input of convolutional layer Conv22, and the output of convolutional layer Conv22 is used as the input of rotation detection head RR. The output of rotation detection head RR is denoted as OUT. reg ; The rotating classification and regression network, denoted as RDN0, is composed of the pooling layer P1, fully connected layers FC1, FC2, FC3, convolutional layers Conv11, Conv12, Conv21, Conv22, and rotating detection heads RC and RR from step 2.

3. Step 3: Train the network Set the iteration parameter epoch, and initialize the epoch value to 1; Step 3.1: Perform forward propagation on the balanced self-attention pyramid network. The training set Train of the dataset SRSDD obtained in step 1 is used as the input of the Balanced Self-Attention Pyramid Network BAFPN0. The traditional forward propagation method is used to feed the training set Train into the Balanced Self-Attention Pyramid Network BAFPN0 for operation, and the output of the network BAFPN0 is obtained, which is denoted as Result0. Step 3.2: Perform forward propagation on the feature alignment network. The output result Result0 of the balanced self-attention pyramid network obtained in step 3.1 is used as the input of the feature alignment network FAN0. The traditional forward propagation method is used to feed the output result Result0 of the balanced self-attention pyramid network into the feature alignment network FAN0 for operation, and the output of the feature alignment network FAN0 is obtained, which is denoted as Result1. Step 3.3: Perform forward propagation on the rotated classification and regression network. The output result Result1 of the feature alignment network obtained in step 3.2 is used as the input of the rotated classification and regression network RDN0. The traditional forward propagation method is used to feed the output result Result1 of the feature alignment network into the rotated classification and regression network RDN0 for calculation, and the output of the rotated classification and regression network RDN0 is obtained, which is denoted as Result. cls and Result reg ; Step 3.4: Train and optimize all networks. The output result of the rotated classification and regression network RDN0 obtained in step 3.3 is Result cls and Result reg As input, the classic Adam algorithm is used to train and optimize the rotation classification and regression network RDN0, feature alignment network FAN0, and balanced self-attention pyramid network BAFPN0, resulting in the trained and optimized rotation classification and regression network RDN1, feature alignment network FAN1, and balanced self-attention pyramid network BAFPN1. Step 4: Perform alternating training Determine if the epoch set in step 3 is equal to 12; If epoch is not equal to 12, then let epoch = epoch + 1, BAFPN0 = BAFPN1, FAN0 = FAN1, RDN0 = RDN1, and repeat steps 3.1, 3.2, 3.3, and 3.4 in sequence, and then return to step 4 to judge epoch again; If the epoch equals 12, then let the trained rotational classification and regression network RDN1, feature alignment network FAN1, and balanced self-attention pyramid network BAFPN1 be denoted as network RBFA-Net, and then proceed to step 5; Step 5, Evaluation Method Step 5.1, Forward Propagation Using the network RBFA-Net obtained in step 4 and the test set Test obtained in step 1 as input, the traditional forward propagation method is used for detection, and the detection result is denoted as R. Using the detection result R as input, the traditional nonmaximum suppression method is used to remove redundant boxes in the detection result R. The specific steps are as follows: Step (1) First, let the highest-scoring bounding box in the detection result R be denoted as BS; Step (2) Then the calculation formula is as follows: Calculate the overlap ratio threshold (IoU) of all bounding boxes in the detection result R; discard boxes with IoU > 0.5; Step (3) Select the highest-scoring box (BS) from the remaining boxes; Repeat the process of calculating IoU and discarding boxes in step (2) above until there are no more boxes to discard. The remaining boxes are the final detection results, denoted as R. F ; Step 5.2, Calculate the indicators The detection result R obtained in step 5.1 F As input, using traditional recall and precision calculation methods, the network's precision P, recall R, and precision-recall curve P(R) are obtained; the formula is used... Calculate the average accuracy (mAP) of SAR ship detection in multi-class SAR images with rotated detection boxes.

Images