A method and system for detecting small target ships in SAR remote sensing images
Through the improved YOLOx network, the feature pyramid and multi-branch receptive field feature enhancement module are used, combined with the group fusion attention module, to solve the problem of low accuracy in detecting small target ships in SAR remote sensing images and improve the detection accuracy.
Patent Information
- Application Number
- CN202411021427.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2044-07-29
AI Technical Summary
In the existing technology, the detection accuracy of small target ships in SAR remote sensing images is low and easily affected by background noise.
An improved YOLOx network is used to enhance multi-scale context features and improve detection accuracy through feature pyramid and multi-branch receptive field feature enhancement modules, combined with group fusion attention modules.
It effectively improves the detection accuracy of small target ships in SAR remote sensing images, reduces the redundancy between feature maps, and makes full use of the features extracted by the network.
Smart Images

Figure CN119131607B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a method for detecting small target ships in SAR remote sensing images. Background Art
[0002] Synthetic Aperture Radar (SAR) is an active imaging sensor capable of acquiring high-resolution data in all weather conditions and at all times. Its multi-band, multi-polarization, variable viewing angle, and high penetration capabilities offer broad research and application prospects. Using SAR imagery to detect vessels on the sea surface can be used in areas such as marine environmental protection, maritime rescue, and fishery supervision.
[0003] In recent years, with the advancement of deep learning, many researchers have introduced convolutional neural networks (CNNs) to SAR vessel detection. However, the detection of small vessels remains a challenge that limits SAR vessel detection accuracy. Ships occupy a relatively small proportion of SAR remote sensing imagery and are represented by distinct sets of pixels. Small vessels, in particular, occupy a small number of pixels, resulting in fewer usable features and being highly susceptible to background noise. Consequently, existing techniques for detecting small vessels have low accuracy. Summary of the Invention
[0004] The purpose of the present invention is to address the problem of low detection accuracy of small target ships in the prior art and to propose a method for detecting small target ships in SAR remote sensing images.
[0005] The technical solution adopted by the present invention to solve the above technical problems is:
[0006] A method for detecting small target ships in SAR remote sensing images comprises the following steps:
[0007] Obtain SAR ship remote sensing images and input the SAR ship remote sensing images into the trained YOLOx network for small target ship detection;
[0008] The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps:
[0009] Step 1: Use the YOLOx backbone network CSPdarknet53 to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively;
[0010] Step 2: Use the improved feature pyramid to process C1, C2, and C3 to obtain multi-scale output feature maps P1, P2, and P3. The improved feature pyramid specifically performs the following steps:
[0011] First, after the feature map C3 is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C3 to obtain the feature map C3'. After that, the feature map C2 is fused with the feature map C3' and processed with CSPLayer to obtain the feature map C2'. After the feature map C2' is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and input the feature map C1' into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed with CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed with CSPLayer to obtain the output feature map P3;
[0012] The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module, and the third multi-branch receptive field feature enhancement module specifically perform the following steps:
[0013] First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the channel number of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number. The feature maps after channel dimensionality reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively.
[0014] In the first branch, the input feature map M1 is processed using a multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, and the feature map output by the first branch is obtained. ;
[0015] In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map output by the second branch. ;
[0016] In the third branch, the group fusion attention module is used to extract the input feature map M3 to obtain the feature map output by the third branch ;
[0017] In the fourth branch, a convolution with a kernel size of 3×3 is used to extract the input feature map M4, and the feature map output by the fourth branch is obtained. ;
[0018] Afterwards, 、 、 as well as The fused features are transformed into an output feature map with the same number of channels as the input feature map through a 1×1 convolution layer.
[0019] Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLO Head to obtain the detection results.
[0020] Furthermore, the group fusion attention module specifically performs the following steps:
[0021] Step 21: First, split the input feature map into N sub-tensors. The shape of each sub-tensor is H×W×C / N, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map.
[0022] Step 22: For each sub-tensor, each sub-tensor is subjected to feature extraction through two branches to obtain the output feature map of each sub-tensor, as follows:
[0023] In the first branch, the input sub-tensor F1 is processed using global average pooling and global adaptive maximum pooling to obtain global average information and global maximum information. The global average information and global maximum information are then fused to obtain fused features. The fused features are processed through a 1×1 convolution layer, and the attention weights are generated through the Sigmoid activation function. The attention weights are then multiplied by the sub-tensor F1, and the weighted feature maps are generated through the residual connection. , expressed as:
[0024]
[0025] in, represents the activation function;
[0026] In the second branch, the input sub-tensor F2 is first processed using Haar wavelet downsampling and dynamic upsampling operators to obtain a feature map. The feature map is then processed through a 1×1 convolution layer, and the attention weight is generated through the Sigmoid activation function. After that, the input sub-tensor F2 is passed through a 3×3 convolution layer and weighted with the attention weight to generate a weighted feature map. , expressed as:
[0027] ;
[0028] Finally, the output of the two branches and Splice to form the output feature map of each sub-tensor;
[0029] Step 2 and 3: Concatenate the output feature maps of all sub-tensors to obtain the final output feature map.
[0030] Furthermore, the global average pooling is expressed as:
[0031]
[0032] in, Represents the value of the feature map x at the i-th row and j-th column.
[0033] Furthermore, the global adaptive maximum pooling is expressed as:
[0034] .
[0035] Furthermore, the Haar wavelet downsampling is expressed as:
[0036]
[0037] in, represents the original image, LL represents the low-frequency component, and LH, HL, and HH represent the high-frequency components.
[0038] Furthermore, the dynamic upsampling operator is expressed as:
[0039]
[0040] in, represents the output pixel after upsampling, represents the sampling point in the input feature map, represents the weight, , represents the convolutional layer, Indicates the number of sampling points.
[0041] Furthermore, the Expressed as:
[0042]
[0043] in, Indicates the upsampling ratio, Indicates the offset.
[0044] Furthermore, the size of the SAR ship remote sensing image is 640*640*3.
[0045] A detection system for small target ships in SAR remote sensing images, the system comprising a recognition module for acquiring SAR ship remote sensing images and inputting the SAR ship remote sensing images into a trained YOLOx network for small target ship detection;
[0046] The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps:
[0047] Step 1: Use the YOLOx backbone network CSPdarknet53 to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively;
[0048] Step 2: Use the improved feature pyramid to process C1, C2, and C3 to obtain multi-scale output feature maps P1, P2, and P3. The improved feature pyramid specifically performs the following steps:
[0049] First, after the feature map C3 is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C3 to obtain the feature map C3'. After that, the feature map C2 is fused with the feature map C3' and processed with CSPLayer to obtain the feature map C2'. After the feature map C2' is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and input the feature map C1' into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed with CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed with CSPLayer to obtain the output feature map P3;
[0050] The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module, and the third multi-branch receptive field feature enhancement module specifically perform the following steps:
[0051] First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the channel number of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number. The feature maps after channel dimensionality reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively.
[0052] In the first branch, the input feature map M1 is processed using a multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, and the feature map output by the first branch is obtained. ;
[0053] In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map output by the second branch. ;
[0054] In the third branch, the group fusion attention module is used to extract the input feature map M3 to obtain the feature map output by the third branch ;
[0055] In the fourth branch, a convolution with a kernel size of 3×3 is used to extract the input feature map M4, and the feature map output by the fourth branch is obtained. ;
[0056] Afterwards, 、 、 as well as The fused features are transformed into an output feature map with the same number of channels as the input feature map through a 1×1 convolution layer.
[0057] Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLO Head to obtain the detection results;
[0058] The group fusion attention module specifically performs the following steps:
[0059] Step 21: First, split the input feature map into N sub-tensors. The shape of each sub-tensor is H×W×C / N, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map.
[0060] Step 22: For each sub-tensor, each sub-tensor is subjected to feature extraction through two branches to obtain the output feature map of each sub-tensor, as follows:
[0061] In the first branch, the input sub-tensor F1 is processed using global average pooling and global adaptive maximum pooling to obtain global average information and global maximum information. The global average information and global maximum information are then fused to obtain fused features. The fused features are processed through a 1×1 convolution layer, and the attention weights are generated through the Sigmoid activation function. The attention weights are then multiplied by the sub-tensor F1, and the weighted feature maps are generated through the residual connection. , expressed as:
[0062]
[0063] in, represents the activation function;
[0064] In the second branch, the input sub-tensor F2 is first processed using Haar wavelet downsampling and dynamic upsampling operators to obtain a feature map. The feature map is then processed through a 1×1 convolution layer, and the attention weight is generated through the Sigmoid activation function. After that, the input sub-tensor F2 is passed through a 3×3 convolution layer and weighted with the attention weight to generate a weighted feature map. , expressed as:
[0065] ;
[0066] Finally, the output of the two branches and Splice to form the output feature map of each sub-tensor;
[0067] Step 2 and 3: Concatenate the output feature maps of all sub-tensors to obtain the final output feature map.
[0068] Furthermore, the global average pooling is expressed as:
[0069]
[0070] in, Represents the value of the feature graph x at the i-th row and j-th column;
[0071] The global adaptive maximum pooling is expressed as:
[0072] ;
[0073] The Haar wavelet downsampling is expressed as:
[0074]
[0075] in, represents the original image, LL represents the low-frequency component, LH, HL and HH represent the high-frequency components;
[0076] The dynamic upsampling operator is expressed as:
[0077]
[0078] in, represents the output pixel after upsampling, represents the sampling point in the input feature map, represents the weight, , represents the convolutional layer, Indicates the number of sampling points;
[0079] described Expressed as:
[0080]
[0081] in, Indicates the upsampling ratio, Indicates the offset.
[0082] The beneficial effects of the present invention are:
[0083] This application enhances the multi-branch receptive field of the contextual features after top-down and bottom-up fusion of the feature pyramid, and inputs these different multi-scale contextual features into the back-end network to reduce the redundancy between the corresponding feature maps, thereby making fuller use of the features extracted by the network and effectively improving the accuracy of small target ship detection in remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS
[0084] Figure 1 A flowchart of this application;
[0085] Figure 2 This is the network structure diagram of this application;
[0086] Figure 3 This is the structure diagram of the multi-branch receptive field module in this application;
[0087] Figure 4 This is the structural diagram of the group fusion attention module in this application;
[0088] Figure 5 This is the detection effect diagram of this application in the entire Gaofen-3 SAR remote sensing image. DETAILED DESCRIPTION
[0089] It should be noted that, unless there is any conflict, the various embodiments disclosed in this application can be combined with each other.
[0090] Specific implementation method 1: refer to Figure 1 Specifically describing this embodiment, a method for detecting small target ships in SAR remote sensing images described in this embodiment is characterized by comprising the following steps:
[0091] Obtain SAR ship remote sensing images and input the SAR ship remote sensing images into the trained YOLOx network for small target ship detection;
[0092] The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps:
[0093] Step 1: Use the YOLOx backbone network CSPdarknet53 to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively;
[0094] Step 2: Use the improved feature pyramid to process C1, C2, and C3 to obtain multi-scale output feature maps P1, P2, and P3. The improved feature pyramid specifically performs the following steps:
[0095] First, after convolution, the feature map C3 is divided into two paths. One path inputs the convolved feature map C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the convolved feature map C3 to obtain the feature map C3'. After that, the feature map C2 and the feature map C3' are fused and processed with CSPLayer to obtain the feature map C2'. After convolution, the feature map C2' is divided into two paths. One path inputs the convolved feature map C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the convolved feature map C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and input the feature map C1' into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed with CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed with CSPLayer to obtain the output feature map P3;
[0096] The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module, and the third multi-branch receptive field feature enhancement module specifically perform the following steps:
[0097] First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the channel number of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number. The feature maps after channel dimensionality reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively.
[0098] In the first branch, the input feature map M1 is processed using a multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, and the feature map output by the first branch is obtained. ;
[0099] In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map output by the second branch. ;
[0100] In the third branch, the group fusion attention module is used to extract the input feature map M3 to obtain the feature map output by the third branch ;
[0101] In the fourth branch, a convolution with a kernel size of 3×3 is used to extract the input feature map M4, and the feature map output by the fourth branch is obtained. ;
[0102] Afterwards, 、 、 as well as The fused features are transformed into an output feature map with the same number of channels as the input feature map through a 1×1 convolution layer.
[0103] Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLO Head to obtain the detection results.
[0104] Reference Figure 1 , the specific implementation steps of this application are:
[0105] (1) Obtain the SAR ship dataset SSDD dataset from the Internet. The image format of the dataset is jpg. There are 1160 images and 2456 ship targets as the input images of the network. The annotation file format of the image is xml. The training set, test set and validation set are divided into 7:2:1 ratios.
[0106] (2) Reference Figure 2 The backbone network receives the input image and completes the basic feature extraction through a series of CSP structures.
[0107] Perform basic feature extraction on the image and output multi-scale feature maps C1, C2, and C3 after 8, 16, and 32 times downsampling, with the number of channels being 256, 512, and 1024 respectively;
[0108] (3) Reference Figure 3 , build a multi-branch receptive field module as a component for constructing the attention feature pyramid;
[0109] The specific steps are as follows:
[0110] (3.1) Construct a branch structure consisting of a standard convolutional layer, an asymmetric convolutional layer, and dilated convolutional layers with different dilation rates. The standard convolution kernel size is 1×1, which is used to transform the number of channels. The asymmetric convolutional layer has a 1×3 and 3×1 convolution structure, which enhances the feature expression ability of the network while reducing the amount of network computation. The kernel size of the multi-layer dilated convolution is 3×3, and the dilation rates of different branches are 1, 2, and 4, respectively, to enhance the receptive field of different scales.
[0111] (3.2) A standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the number of channels of the input feature map, reducing the number of channels of the input feature map of the previous layer to 1 / 4 of the original number, balancing the representation of different parallel branches, and serving as the input feature maps M1, M2, M3, and M4 of each branch;
[0112] (3.3) Set the input feature map M1 in the first branch and use multi-layer dilated convolution with a kernel size of 3×3 and dilation rates of 1, 2, and 4, respectively. This method reduces parameters and computational complexity while maintaining resolution, retains more fine-grained features, and improves detection performance. Output the feature map of the first branch.
[0113] (3.4) Set the input feature map M2 in the second branch. First, use two layers of asymmetric convolution (1×3 and 3×1) to enhance the network's feature expression capability. Then use the designed group fusion attention module for feature extraction and output the feature map of the second branch.
[0114] (3.5) Set the input feature map M3 in the third branch, use the designed group fusion attention module to extract features, and output the feature map of the third branch;
[0115] (3.6) Set the input feature map M4 in the fourth branch, use convolution with a kernel size of 3×3 for feature extraction, and output the feature map of the fourth branch;
[0116] (3.7) Concatenate the four feature map outputs of different scales together, and transform the number of channels of the concatenated feature map through a 1×1 convolutional layer to keep the number of channels consistent with the initial input feature map, and output the feature map;
[0117] (3.8) Connect the input feature map through the residual edge and add the input feature map to the output feature map to achieve the purpose of fusing features of different scales and output the final feature map.
[0118] (4) Refer to the attached Figure 4 , build a group fusion attention module;
[0119] The specific steps are as follows:
[0120] (4.1) In this network structure, the input tensor is first split into N sub-tensors through channel processing, and the shape of each sub-tensor is H×W×C / N;
[0121] (4.2) For each sub-tensor, two different branches are set for feature extraction;
[0122] (4.3) Each sub-tensor is extracted through two different branches. The first branch uses global average pooling (GAP) and global adaptive maximum pooling (GMP) to process features, and its formulas are respectively,
[0123]
[0124]
[0125] These two pooling methods extract the global average information and maximum information respectively, and then add these two pieces of information together. Then, the fused features are further processed through a 1×1 convolution layer, and finally the attention weight is generated through the Sigmoid activation function. This weight is multiplied with the original input F1 and a weighted feature map is generated through the residual connection. .
[0126]
[0127] (4.4) In the second branch, Haar wavelet downsampling (HWD) and dynamic upsampling operators (DySample) are first used to adjust the size of the feature map. This reduces the spatial resolution of the feature map to extract high-level features and combines multi-scale information when restoring the original resolution, thereby improving the network's feature expression capabilities. The Haar wavelet transform decomposes the image into low-frequency and high-frequency components. For a two-dimensional image, the Haar wavelet transform can be expressed as:
[0128]
[0129] in, represents the original image, LL represents the low-frequency component, and LH, HL, and HH represent the high-frequency components.
[0130] DySample performs upsampling by point sampling. The formula is as follows:
[0131]
[0132] in, represents the output pixel after upsampling, represents the sampling point in the input feature map, represents the weight, , represents the convolutional layer, Indicates the number of sampling points.
[0133] Further sampling points The choice is based on the position of the input feature map and the upsampling ratio, and the formula is as follows:
[0134]
[0135] in, is the upsampling ratio, is an offset used to refine the location of the sampling points.
[0136] Then, another set of attention weights is generated by a 1×1 convolution layer and a Sigmoid activation function. This weight is used to weight a feature map F2 extracted by a 3×3 convolution layer to generate a weighted feature map .
[0137]
[0138] (4.5) Finally, the outputs of the two branches are and The output feature maps of all sub-tensors are concatenated to form the final output feature map H×W×C.
[0139] (5) The SSDD dataset was used to train the YOLOx deep learning network consisting of a backbone network, an attention feature pyramid, and a prediction end. During training, the input image scale was set to 640×640, the epoch was 150 rounds, the optimizer used the stochastic gradient descent method SGD, the initial learning rate was set to 0.01, and the cosine annealing learning strategy was adopted to dynamically adjust the learning rate so that the training set loss and the validation set loss tended to a balanced state, thus obtaining a trained network model.
[0140] (6) Refer to the attached Figure 5 ,The trained network is used to detect the entire L1A-level GF-3 SAR remote sensing image with HH polarization in the acquisition area, and the final detection results are obtained.
[0141] It should be noted that the specific embodiments are merely explanations and illustrations of the technical solutions of the present invention and cannot be used to limit the scope of protection. Any minor changes made based on the claims and description of the present invention shall still fall within the scope of protection of the present invention.
Claims
1. A method for detecting small target ships in SAR remote sensing images, characterized in that The following steps are involved: Obtain SAR ship remote sensing images and input the SAR ship remote sensing images into the trained YOLOx network for small target ship detection; The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps: Step 1: Use the YOLOx backbone network CSPdarknet53 to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively; Step 2: Use the improved feature pyramid to process C1, C2, and C3 to obtain multi-scale output feature maps P1, P2, and P3. The improved feature pyramid specifically performs the following steps: First, after the feature map C3 is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C3 to obtain the feature map C3'. After that, the feature map C2 is fused with the feature map C3' and processed with CSPLayer to obtain the feature map C2'. After the feature map C2' is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and input the feature map C1' into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed with CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed with CSPLayer to obtain the output feature map P3; The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module, and the third multi-branch receptive field feature enhancement module specifically perform the following steps: First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the channel number of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number. The feature maps after channel dimensionality reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively. In the first branch, the input feature map M1 is processed using a multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, and the feature map output by the first branch is obtained. ; In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map output by the second branch. ; In the third branch, the group fusion attention module is used to extract the input feature map M3 to obtain the feature map output by the third branch ; In the fourth branch, a convolution with a kernel size of 3×3 is used to extract the input feature map M4, and the feature map output by the fourth branch is obtained. ; Afterwards, 、 、 as well as The fused features are transformed into an output feature map with the same number of channels as the input feature map through a 1×1 convolution layer. Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLO Head to obtain the detection results.
2. A method for detecting small target ships in SAR remote sensing images according to claim 1, characterized in that The group fusion attention module specifically performs the following steps: Step 21: First, split the input feature map into N sub-tensors. The shape of each sub-tensor is H×W×C / N, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map. Step 22: For each sub-tensor, each sub-tensor is subjected to feature extraction through two branches to obtain the output feature map of each sub-tensor, as follows: In the first branch, the input sub-tensor F1 is processed using global average pooling and global adaptive maximum pooling to obtain global average information and global maximum information. The global average information and global maximum information are then fused to obtain fused features. The fused features are processed through a 1×1 convolution layer, and the attention weights are generated through the Sigmoid activation function. The attention weights are then multiplied by the sub-tensor F1, and the weighted feature maps are generated through the residual connection. , expressed as: in, represents the activation function; In the second branch, the input sub-tensor F2 is first processed using Haar wavelet downsampling and dynamic upsampling operators to obtain a feature map. The feature map is then processed through a 1×1 convolution layer, and the attention weight is generated through the Sigmoid activation function. After that, the input sub-tensor F2 is passed through a 3×3 convolution layer and weighted with the attention weight to generate a weighted feature map. , expressed as: ; Finally, the output of the two branches and Splice to form the output feature map of each sub-tensor; Step 2 and 3: Concatenate the output feature maps of all sub-tensors to obtain the final output feature map.
3. A method for detecting small target ships in SAR remote sensing images according to claim 2, characterized in that The global average pooling is expressed as: in, Represents the value of the feature map x at the i-th row and j-th column.
4. A method for detecting small target ships in SAR remote sensing images according to claim 3, characterized in that The global adaptive maximum pooling is expressed as: 。 5. A method for detecting small target ships in SAR remote sensing images according to claim 4, characterized in that The Haar wavelet downsampling is expressed as: in, represents the original image, LL represents the low-frequency component, and LH, HL, and HH represent the high-frequency components.
6. A method for detecting small target ships in SAR remote sensing images according to claim 5, characterized in that The dynamic upsampling operator is expressed as: in, represents the output pixel after upsampling, represents the sampling point in the input feature map, represents the weight, , represents the convolutional layer, Indicates the number of sampling points.
7. A method for detecting small target ships in SAR remote sensing images according to claim 6, characterized in that described Expressed as: in, Indicates the upsampling ratio, Indicates the offset.
8. The method for detecting small target ships in SAR remote sensing images according to claim 7, characterized in that The size of the SAR ship remote sensing image is 640*640*3.
9. A detection system for small target ships in SAR remote sensing images, characterized by The system includes a recognition module, which is used to obtain SAR ship remote sensing images and input the SAR ship remote sensing images into a trained YOLOx network to detect small target ships; The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps: Step 1: Use the YOLOx backbone network CSPdarknet53 to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively; Step 2: Use the improved feature pyramid to process C1, C2, and C3 to obtain multi-scale output feature maps P1, P2, and P3. The improved feature pyramid specifically performs the following steps: First, after the feature map C3 is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C3 to obtain the feature map C3'. After that, the feature map C2 is fused with the feature map C3' and processed with CSPLayer to obtain the feature map C2'. After the feature map C2' is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and input the feature map C1' into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed with CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed with CSPLayer to obtain the output feature map P3; The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module, and the third multi-branch receptive field feature enhancement module specifically perform the following steps: First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the channel number of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number. The feature maps after channel dimensionality reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively. In the first branch, the input feature map M1 is processed using a multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, and the feature map output by the first branch is obtained. ; In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map output by the second branch. ; In the third branch, the group fusion attention module is used to extract the input feature map M3 to obtain the feature map output by the third branch ; In the fourth branch, a convolution with a kernel size of 3×3 is used to extract the input feature map M4, and the feature map output by the fourth branch is obtained. ; Afterwards, 、 、 as well as The fused features are transformed into an output feature map with the same number of channels as the input feature map through a 1×1 convolution layer. Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLO Head to obtain the detection results; The group fusion attention module specifically performs the following steps: Step 21: First, split the input feature map into N sub-tensors. The shape of each sub-tensor is H×W×C / N, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map. Step 22: For each sub-tensor, each sub-tensor is subjected to feature extraction through two branches to obtain the output feature map of each sub-tensor, as follows: In the first branch, the input sub-tensor F1 is processed using global average pooling and global adaptive maximum pooling to obtain global average information and global maximum information. The global average information and global maximum information are then fused to obtain fused features. The fused features are processed through a 1×1 convolution layer, and the attention weights are generated through the Sigmoid activation function. The attention weights are then multiplied by the sub-tensor F1, and the weighted feature maps are generated through the residual connection. , expressed as: in, represents the activation function; In the second branch, the input sub-tensor F2 is first processed using Haar wavelet downsampling and dynamic upsampling operators to obtain a feature map. The feature map is then processed through a 1×1 convolution layer, and the attention weight is generated through the Sigmoid activation function. After that, the input sub-tensor F2 is passed through a 3×3 convolution layer and weighted with the attention weight to generate a weighted feature map. , expressed as: ; Finally, the output of the two branches and Splice to form the output feature map of each sub-tensor; Step 2 and 3: Concatenate the output feature maps of all sub-tensors to obtain the final output feature map.
10. A detection system for small target ships in SAR remote sensing images according to claim 9, characterized in that The global average pooling is expressed as: in, Represents the value of the feature graph x at the i-th row and j-th column; The global adaptive maximum pooling is expressed as: ; The Haar wavelet downsampling is expressed as: in, represents the original image, LL represents the low-frequency component, LH, HL and HH represent the high-frequency components; The dynamic upsampling operator is expressed as: in, represents the output pixel after upsampling, represents the sampling point in the input feature map, represents the weight, , represents the convolutional layer, Indicates the number of sampling points; described Expressed as: in, Indicates the upsampling ratio, Indicates the offset.
Citation Information
Patent Citations
SAR ship target detection method based on network pruning and knowledge distillation
CN112308019A
SAR image vessel detection method based on deep fusion optimization feature enhancement network
CN118212523A