Method and system for detecting small target ship in SAR remote sensing image
The improved YOLOx network addresses the low detection accuracy of small ships in SAR images by employing a feature pyramid with multi-branch enhancement and attention modules, effectively improving detection accuracy through enhanced feature extraction and fusion.
Patent Information
- Application Number
- CN202411021427.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2044-07-29
AI Technical Summary
The detection accuracy of small target ships in SAR remote sensing images is low due to their small size, limited features, and susceptibility to background noise, limiting the effectiveness of existing convolutional neural network methods.
An improved YOLOx network is used for small target ship detection, employing a backbone network with downsampling and a feature pyramid that includes multi-branch receptive field feature enhancement modules and a group fusion attention module to process multi-scale features, enhancing contextual feature extraction and reducing redundancy.
The method improves the accuracy of small target ship detection by fully utilizing extracted features through top-down and bottom-up fusion of multi-scale contextual features, reducing redundancy and enhancing detection performance.
Smart Images

Figure CN119131607A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of target detection, and in particular to a method for detecting small target ships in SAR remote sensing images. Background Art
[0002] Synthetic aperture radar (SAR) is an active imaging sensor with all-weather, all-day, high-resolution data acquisition capabilities. It also has the characteristics of multi-band, multi-polarization, variable viewing angle and penetration, and has broad research and application prospects. Using SAR images to detect ships on the sea surface can be used for marine environmental protection, maritime rescue and fishery supervision.
[0003] In recent years, with the development of deep learning, many scholars have introduced convolutional neural network (CNN) methods into SAR ship target detection. However, the detection of small target ships is still a difficulty that limits the accuracy of SAR ship target detection. In SAR remote sensing images, ship targets account for a small proportion and are reflected as different sets of pixels in the images. In particular, small ships occupy a small number of pixels, have fewer available features, and are easily affected by background noise. Therefore, the detection accuracy of small target ships in the existing technology is low. Summary of the invention
[0004] The purpose of the present invention is to propose a method for detecting small target ships in SAR remote sensing images in view of the problem that the detection accuracy of small target ships in the prior art is low.
[0005] The technical solution adopted by the present invention to solve the above technical problems is:
[0006] A method for detecting small target ships in SAR remote sensing images comprises the following steps:
[0007] Obtain SAR ship remote sensing images, and input the SAR ship remote sensing images into the trained YOLOx network for small target ship detection;
[0008] The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps:
[0009] Step 1: Use the backbone network CSPdarknet53 of YOLOx to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively;
[0010] Step 2: Use the improved feature pyramid to process C1, C2 and C3 to obtain multi-scale output feature maps P1, P2 and P3. The improved feature pyramid specifically performs the following steps:
[0011] First, after the feature map C3 is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C3 to obtain the feature map C3'. After that, the feature map C2 is fused with the feature map C3' and processed by CSPLayer to obtain the feature map C2'. After the feature map C2' is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and the feature map C1' is input into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed by CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed by CSPLayer to obtain the output feature map P3;
[0012] The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module and the third multi-branch receptive field feature enhancement module specifically perform the following steps:
[0013] First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the number of channels of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number, and the feature maps after channel number reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively;
[0014] In the first branch, the input feature map M1 is processed using multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, respectively, to obtain the feature map M1* output by the first branch;
[0015] In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map M2* output by the second branch;
[0016] In the third branch, the group fusion attention module is used to extract features from the input feature map M3 to obtain the feature map M3* output by the third branch;
[0017] In the fourth branch, a convolution with a convolution kernel size of 3×3 is used to extract features from the input feature map M4 to obtain the feature map M4* output by the fourth branch;
[0018] Afterwards, M1*, M2*, M3* and M4* are fused, and the number of channels of the fused feature map is transformed through a 1×1 convolution layer to obtain an output feature map with the same number of channels as the input feature map;
[0019] Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLO Head to obtain the detection results.
[0020] Furthermore, the group fusion attention module specifically performs the following steps:
[0021] Step 21: First, split the input feature map into N sub-tensors, each of which has a shape of H×W×C / N, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map;
[0022] Step 22: For each sub-tensor, each sub-tensor is subjected to feature extraction through two branches to obtain the output feature map of each sub-tensor, as follows:
[0023] In the first branch, the input sub-tensor F1 is processed by global average pooling and global adaptive maximum pooling to obtain global average information and global maximum information, and then the global average information and the global maximum information are fused to obtain fused features. After the fused features are processed by a 1×1 convolution layer, the attention weight is generated by the Sigmoid activation function, and then the attention weight is multiplied by the sub-tensor F1, and the weighted feature map F1* is generated through the residual connection, which is expressed as:
[0024] F1 * =F1×σ(F1)
[0025] Among them, σ represents the activation function;
[0026] In the second branch, the input sub-tensor F2 is first processed using the Haar wavelet downsampling and dynamic upsampling operators to obtain a feature map, and then the obtained feature map is processed by a 1×1 convolution layer, and the attention weight is generated by the Sigmoid activation function. After that, the input sub-tensor F2 passes through a 3×3 convolution layer and is weighted with the attention weight to generate a weighted feature map F2*, which is expressed as:
[0027] F2 * =F2×σ(F2);
[0028] Finally, the outputs F1* and F2* of the two branches are concatenated to form the output feature map of each sub-tensor;
[0029] Step 2 and 3: Concatenate the output feature maps of all sub-tensors to obtain the final output feature map.
[0030] Furthermore, the global average pooling is expressed as:
[0031]
[0032] Among them, x ij Represents the value of the feature graph x at the i-th row and j-th column.
[0033] Further, the global adaptive maximum pooling is expressed as:
[0034] GMP(x)=max i,j x ij .
[0035] Furthermore, the Haar wavelet downsampling is expressed as:
[0036]
[0037] Among them, l represents the original image, LL represents the low-frequency component, and LH, HL and HH represent the high-frequency components.
[0038] Furthermore, the dynamic upsampling operator is expressed as:
[0039]
[0040] Among them, y i represents the output pixel after upsampling, represents the sampling point in the input feature map, w ij represents the weight, f(·) represents the convolutional layer, and J represents the number of sampling points.
[0041] Furthermore, the It is expressed as:
[0042]
[0043] Among them, s represents the upsampling ratio, Δp ij Indicates the offset.
[0044] Furthermore, the size of the SAR ship remote sensing image is 640*640*3.
[0045] A detection system for small target ships in SAR remote sensing images, the system comprising a recognition module, the recognition module is used to obtain SAR ship remote sensing images, and input the SAR ship remote sensing images into a trained YOLOx network to perform small target ship detection;
[0046] The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps:
[0047] Step 1: Use the backbone network CSPdarknet53 of YOLOx to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively;
[0048] Step 2: Use the improved feature pyramid to process C1, C2 and C3 to obtain multi-scale output feature maps P1, P2 and P3. The improved feature pyramid specifically performs the following steps:
[0049] First, after the feature map C3 is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C3 to obtain the feature map C3'. After that, the feature map C2 is fused with the feature map C3' and processed by CSPLayer to obtain the feature map C2'. After the feature map C2' is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and the feature map C1' is input into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed by CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed by CSPLayer to obtain the output feature map P3;
[0050] The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module and the third multi-branch receptive field feature enhancement module specifically perform the following steps:
[0051] First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the number of channels of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number, and the feature maps after channel number reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively;
[0052] In the first branch, the input feature map M1 is processed using multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, respectively, to obtain the feature map M1* output by the first branch;
[0053] In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map M2* output by the second branch;
[0054] In the third branch, the group fusion attention module is used to extract features from the input feature map M3 to obtain the feature map M3* output by the third branch;
[0055] In the fourth branch, a convolution with a convolution kernel size of 3×3 is used to extract features from the input feature map M4 to obtain the feature map M4* output by the fourth branch;
[0056] Afterwards, M1*, M2*, M3* and M4* are fused, and the number of channels of the fused feature map is transformed through a 1×1 convolution layer to obtain an output feature map with the same number of channels as the input feature map;
[0057] Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLOHead to obtain the detection results;
[0058] The group fusion attention module specifically performs the following steps:
[0059] Step 21: First, split the input feature map into N sub-tensors, each of which has a shape of H×W×C / N, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map;
[0060] Step 22: For each sub-tensor, each sub-tensor is subjected to feature extraction through two branches to obtain the output feature map of each sub-tensor, as follows:
[0061] In the first branch, the input sub-tensor F1 is processed by global average pooling and global adaptive maximum pooling to obtain global average information and global maximum information, and then the global average information and the global maximum information are fused to obtain fused features. After the fused features are processed by a 1×1 convolution layer, the attention weight is generated by the Sigmoid activation function, and then the attention weight is multiplied by the sub-tensor F1, and the weighted feature map F1* is generated through the residual connection, which is expressed as:
[0062] F1 * =F1×σ(F1)
[0063] Among them, σ represents the activation function;
[0064] In the second branch, the input sub-tensor F2 is first processed using the Haar wavelet downsampling and dynamic upsampling operators to obtain a feature map, and then the obtained feature map is processed by a 1×1 convolution layer, and the attention weight is generated by the Sigmoid activation function. After that, the input sub-tensor F2 passes through a 3×3 convolution layer and is weighted with the attention weight to generate a weighted feature map F2*, which is expressed as:
[0065] F2 * =F2×σ(F2);
[0066] Finally, the outputs F1* and F2* of the two branches are concatenated to form the output feature map of each sub-tensor;
[0067] Step 2 and 3: Concatenate the output feature maps of all sub-tensors to obtain the final output feature map.
[0068] Furthermore, the global average pooling is expressed as:
[0069]
[0070] Among them, x ij Represents the value of feature graph x at the i-th row and j-th column;
[0071] The global adaptive maximum pooling is expressed as:
[0072] GMP(x)=max i,j x ij ;
[0073] The Haar wavelet downsampling is expressed as:
[0074]
[0075] Among them, l represents the original image, LL represents the low-frequency component, and LH, HL and HH represent the high-frequency components;
[0076] The dynamic upsampling operator is expressed as:
[0077]
[0078] Among them, y i represents the output pixel after upsampling, represents the sampling point in the input feature map, w ij represents the weight, f(·) represents the convolutional layer, J represents the number of sampling points;
[0079] Said It is expressed as:
[0080]
[0081] Among them, s represents the upsampling ratio, Δp ij Indicates the offset.
[0082] The beneficial effects of the present invention are:
[0083] This application enhances the multi-branch receptive field of the contextual features after top-down and bottom-up fusion of the feature pyramid, and inputs these different multi-scale contextual features into the back-end network to reduce the redundancy between the corresponding feature maps, thereby making fuller use of the features extracted by the network and effectively improving the accuracy of small target ship detection in remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS
[0084] Figure 1 A flowchart of this application;
[0085] Figure 2 This is the network structure diagram of this application;
[0086] Figure 3 This is a structural diagram of the multi-branch receptive field module in this application;
[0087] Figure 4 This is a structural diagram of the group fusion attention module in this application;
[0088] Figure 5 This is a diagram of the detection effect of this application in the entire GF-3 SAR remote sensing image. DETAILED DESCRIPTION
[0089] It should be particularly noted that, in the absence of conflict, the various embodiments disclosed in this application can be combined with each other.
[0090] Specific implementation method 1: refer to Figure 1 Specifically describing this embodiment, a method for detecting small target ships in SAR remote sensing images described in this embodiment is characterized by comprising the following steps:
[0091] Obtain SAR ship remote sensing images, and input the SAR ship remote sensing images into the trained YOLOx network for small target ship detection;
[0092] The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps:
[0093] Step 1: Use the backbone network CSPdarknet53 of YOLOx to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively;
[0094] Step 2: Use the improved feature pyramid to process C1, C2 and C3 to obtain multi-scale output feature maps P1, P2 and P3. The improved feature pyramid specifically performs the following steps:
[0095] First, after convolution, feature map C3 is divided into two paths. One path inputs the convolution feature map C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the convolution feature map C3 to obtain feature map C3'. After that, feature map C2 is fused with feature map C3' and processed by CSPLayer to obtain feature map C2'. After convolution, feature map C2' is divided into two paths. One path inputs the convolution feature map C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the convolution feature map C2' and fuses it with feature map C1. and CSPLayer to obtain a feature map C1', and the feature map C1' is input into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed by CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed by CSPLayer to obtain the output feature map P3;
[0096] The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module and the third multi-branch receptive field feature enhancement module specifically perform the following steps:
[0097] First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the number of channels of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number, and the feature maps after channel number reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively;
[0098] In the first branch, the input feature map M1 is processed using multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, respectively, to obtain the feature map M1* output by the first branch;
[0099] In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map M2* output by the second branch;
[0100] In the third branch, the group fusion attention module is used to extract features from the input feature map M3 to obtain the feature map M3* output by the third branch;
[0101] In the fourth branch, a convolution with a convolution kernel size of 3×3 is used to extract features from the input feature map M4 to obtain the feature map M4* output by the fourth branch;
[0102] Afterwards, M1*, M2*, M3* and M4* are fused, and the number of channels of the fused feature map is transformed through a 1×1 convolution layer to obtain an output feature map with the same number of channels as the input feature map;
[0103] Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLO Head to obtain the detection results.
[0104] Reference Figure 1 The specific implementation steps of this application are:
[0105] (1) Obtain the SAR ship dataset SSDD dataset from the Internet. The image format of the dataset is jpg. There are 1160 images and 2456 ship targets as the input image of the network. The annotation file format of the image is xml. The training set, test set and validation set are divided according to the ratio of 7:2:1.
[0106] (2)Reference Figure 2 The backbone network receives the input image and completes basic feature extraction through a series of CSP structures.
[0107] Perform basic feature extraction on the image and output multi-scale feature maps C1, C2, and C3 after 8, 16, and 32 times downsampling, with the number of channels being 256, 512, and 1024 respectively;
[0108] (3)Reference Figure 3 , build a multi-branch receptive field module as a component for constructing the attention feature pyramid;
[0109] The specific steps are as follows:
[0110] (3.1) Construct a branch structure consisting of a standard convolution layer, an asymmetric convolution layer, and an expanded convolution layer with different expansion rates, where the standard convolution kernel size is 1×1, which is used to transform the number of channels; the asymmetric convolution layer is a 1×3 and 3×1 convolution structure, which enhances the feature expression ability of the network while reducing the amount of network calculation; the kernel size of the multi-layer expanded convolution is 3×3, and the expansion rates of different branches are 1, 2, and 4, respectively, to enhance the receptive field of different scales;
[0111] (3.2) A standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the number of channels of the input feature map, reducing the number of channels of the input feature map of the previous layer to 1 / 4 of the original number, balancing the representation of different parallel branches, and serving as the input feature maps M1, M2, M3, and M4 of each branch;
[0112] (3.3) Set the input feature map M1 in the first branch, use multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4 respectively, reduce parameters and calculation amount while maintaining resolution, retain more fine-grained features to improve detection performance, and output the feature map of the first branch;
[0113] (3.4) Set the input feature map M2 in the second branch. First, use two layers of asymmetric convolution of 1×3 and 3×1 to enhance the feature expression ability of the network. Then use the designed group fusion attention module to extract features and output the feature map of the second branch.
[0114] (3.5) Set the input feature map M3 in the third branch, use the designed group fusion attention module to extract features, and output the feature map of the third branch;
[0115] (3.6) Set the input feature map M4 in the fourth branch, use convolution with a convolution kernel size of 3×3 for feature extraction, and output the feature map of the fourth branch;
[0116] (3.7) Connect the feature map outputs of the four different scales together, and transform the number of channels of the connected feature map through a 1×1 convolution layer to keep the number of channels consistent with the initial input feature map, and output the feature map;
[0117] (3.8) The input feature map is connected to the output feature map through the residual edge to achieve the purpose of fusing features of different scales and output the final feature map.
[0118] (4) Refer to the attached Figure 4 , construct the group fusion attention module;
[0119] The specific steps are as follows:
[0120] (4.1) In this network structure, the input tensor is first split into N sub-tensors through channel processing, and the shape of each sub-tensor is H×W×C / N;
[0121] (4.2) For each sub-tensor, two different branches are set for feature extraction;
[0122] (4.3) Each sub-tensor is extracted through two different branches. The first branch uses global average pooling (GAP) and global adaptive maximum pooling (GMP) to process features. The formulas are:
[0123]
[0124] GMP(x)=max i,j x ij
[0125] These two pooling methods extract the global average information and maximum information respectively, and then add the two information together. Then, the fused features are further processed through a 1×1 convolution layer, and finally the attention weight is generated through the Sigmoid activation function. This weight is multiplied by the original input F1, and the weighted feature map F1* is generated through the residual connection.
[0126] F1 * =F1×σ(F1)
[0127] (4.4) In the second branch, the Haar wavelet downsampling (HWD) and dynamic upsampling operator (DySample) are first used to adjust the size of the feature map, extract high-level features by reducing the spatial resolution of the feature map, and combine multi-scale information when restoring the original resolution, thereby improving the feature expression ability of the network. The Haar wavelet transform decomposes the image into low-frequency and high-frequency components. For a two-dimensional image, the Haar wavelet transform can be expressed as:
[0128]
[0129] Among them, l represents the original image, LL represents the low-frequency component, and LH, HL and HH represent the high-frequency components.
[0130] DySample performs upsampling by point sampling, the formula is as follows:
[0131]
[0132] Among them, y i represents the output pixel after upsampling, represents the sampling point in the input feature map, w ij represents the weight, f(·) represents the convolutional layer, and J represents the number of sampling points.
[0133] Further sampling point p ij The choice is based on the position of the input feature map and the upsampling ratio, and the formula is as follows:
[0134] p ij =floor(i / s)+Δp ij
[0135] Where s is the upsampling ratio, Δp ij is an offset used to refine the location of the sampling points.
[0136] Then, another set of attention weights is generated through a 1×1 convolution layer and a Sigmoid activation function. This weight is used to weight a feature map F2 extracted by a 3×3 convolution layer to generate a weighted feature map F2*.
[0137] F2 * =F2×σ(F2)
[0138] (4.5) Finally, the outputs F1* and F2* of the two branches are concatenated to form the output feature map of each sub-tensor. The output feature maps of all sub-tensors are concatenated again to form the final output feature map H×W×C.
[0139] (5) The SSDD dataset was used to train the YOLOx deep learning network consisting of the backbone network, the attention feature pyramid, and the prediction end. During the training, the scale of the input image was set to 640×640, the epoch was set to 150 rounds, the optimizer used the stochastic gradient descent method SGD, the initial learning rate was set to 0.01, and the cosine annealing learning strategy was adopted to dynamically adjust the learning rate so that the training set loss and the validation set loss tended to a balanced state, and a trained network model was obtained;
[0140] (6) Refer to the attached Figure 5 , the trained network is used to detect the L1A-level entire GF-3 SAR remote sensing image with the acquisition area of the Strait of Malacca and the polarization mode of HH, and the final detection results are obtained.
[0141] It should be noted that the specific implementation is only an explanation and description of the technical solution of the present invention, and cannot be used to limit the scope of protection of the rights. Any partial changes made according to the claims and description of the present invention should still fall within the scope of protection of the present invention.
Claims
1. A method for detecting small target ships in SAR remote sensing images, characterized in that The following steps are involved: Obtain SAR ship remote sensing images, and input the SAR ship remote sensing images into the trained YOLOx network for small target ship detection; The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps: Step 1: Use the backbone network CSPdarknet53 of YOLOx to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively; Step 2: Use the improved feature pyramid to process C1, C2 and C3 to obtain multi-scale output feature maps P1, P2 and P3. The improved feature pyramid specifically performs the following steps: First, after the feature map C3 is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C3 to obtain the feature map C3'. After that, the feature map C2 is fused with the feature map C3' and processed by CSPLayer to obtain the feature map C2'. After the feature map C2' is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and the feature map C1' is input into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed by CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed by CSPLayer to obtain the output feature map P3; The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module and the third multi-branch receptive field feature enhancement module specifically perform the following steps: First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the number of channels of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number, and the feature maps after channel number reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively; In the first branch, the input feature map M1 is processed using multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, respectively, to obtain the feature map M1* output by the first branch; In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map M2* output by the second branch; In the third branch, the group fusion attention module is used to extract features from the input feature map M3 to obtain the feature map M3* output by the third branch; In the fourth branch, a convolution with a convolution kernel size of 3×3 is used to extract features from the input feature map M4 to obtain the feature map M4* output by the fourth branch; Afterwards, M1*, M2*, M3* and M4* are fused, and the number of channels of the fused feature map is transformed through a 1×1 convolution layer to obtain an output feature map with the same number of channels as the input feature map; Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLOHead to obtain the detection results.
2. A method for detecting small target ships in SAR remote sensing images according to claim 1, characterized in that The group fusion attention module specifically performs the following steps: Step 21: First, split the input feature map into N sub-tensors, each of which has a shape of H×W×C / N, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map; Step 22: For each sub-tensor, each sub-tensor is subjected to feature extraction through two branches to obtain the output feature map of each sub-tensor, as follows: In the first branch, the input sub-tensor F1 is processed by global average pooling and global adaptive maximum pooling to obtain global average information and global maximum information, and then the global average information and the global maximum information are fused to obtain fused features. After the fused features are processed by a 1×1 convolution layer, the attention weight is generated by the Sigmoid activation function, and then the attention weight is multiplied by the sub-tensor F1, and the weighted feature map F1* is generated through the residual connection, which is expressed as: F1 * =F1×σ(F1) Among them, σ represents the activation function; In the second branch, the input sub-tensor F2 is first processed using the Haar wavelet downsampling and dynamic upsampling operators to obtain a feature map, and then the obtained feature map is processed by a 1×1 convolution layer, and the attention weight is generated by the Sigmoid activation function. After that, the input sub-tensor F2 passes through a 3×3 convolution layer and is weighted with the attention weight to generate a weighted feature map F2*, which is expressed as: <h2 style=";text-align:left;direction:ltr">F2<h2 style=";text-align:left;direction:ltr"> * <h2 style=";text-align:left;direction:ltr"> (F2×σ(F2)) Finally, the outputs F1* and F2* of the two branches are concatenated to form the output feature map of each sub-tensor; Step 2 and 3: Concatenate the output feature maps of all sub-tensors to obtain the final output feature map.
3. A method for detecting small target ships in SAR remote sensing images according to claim 2, characterized in that The global average pooling is expressed as: Among them, x ij Represents the value of the feature graph x at the i-th row and j-th column.
4. A method for detecting small target ships in SAR remote sensing images according to claim 3, characterized in that The global adaptive maximum pooling is expressed as: GMP(x)=max i,j x ij 。 5. A method for detecting small target ships in SAR remote sensing images according to claim 4, characterized in that The Haar wavelet downsampling is expressed as: Among them, l represents the original image, LL represents the low-frequency component, and LH, HL and HH represent the high-frequency components.
6. A method for detecting small target ships in SAR remote sensing images according to claim 5, characterized in that The dynamic upsampling operator is expressed as: Among them, y i represents the output pixel after upsampling, represents the sampling point in the input feature map, w ij represents the weight, f(·) represents the convolutional layer, and J represents the number of sampling points.
7. A method for detecting small target ships in SAR remote sensing images according to claim 6, characterized in that Said It is expressed as: Among them, s represents the upsampling ratio, Δp ij Indicates the offset.
8. A method for detecting small target ships in SAR remote sensing images according to claim 7, characterized in that The size of the SAR ship remote sensing image is 640*640*3.
9. A detection system for small target ships in SAR remote sensing images, characterized in that The system includes a recognition module, which is used to obtain SAR ship remote sensing images and input the SAR ship remote sensing images into a trained YOLOx network to perform small target ship detection; The YOLOx network is an improved YOLOx network, and the improved YOLOx network specifically performs the following steps: Step 1: Use the backbone network CSPdarknet53 of YOLOx to downsample the input SAR ship remote sensing image by 8 times, 16 times, and 32 times respectively to obtain feature maps C1, C2, and C3. The number of channels of C1, C2, and C3 are 256, 512, and 1024 respectively; Step 2: Use the improved feature pyramid to process C1, C2 and C3 to obtain multi-scale output feature maps P1, P2 and P3. The improved feature pyramid specifically performs the following steps: First, after the feature map C3 is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C3 into the first multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C3 to obtain the feature map C3'. After that, the feature map C2 is fused with the feature map C3' and processed by CSPLayer to obtain the feature map C2'. After the feature map C2' is convolved, it is divided into two paths. One path inputs the feature map after the convolution of C2' into the second multi-branch receptive field feature enhancement module, and the other path upsamples the feature map after the convolution of C2' and fuses it with the feature map C1. and CSPLayer to obtain a feature map C1', and the feature map C1' is input into the third multi-branch receptive field feature enhancement module. The output of the third multi-branch receptive field feature enhancement module is two-way, one way is output as the feature map P1, and the other way is down-sampled and fused with the output of the second multi-branch receptive field feature enhancement module and processed by CSPLayer. The processing result is divided into two ways, one way is output as the feature map P2, and the other way is down-sampled and fused with the output of the first multi-branch receptive field feature enhancement module and processed by CSPLayer to obtain the output feature map P3; The first multi-branch receptive field feature enhancement module, the second multi-branch receptive field feature enhancement module and the third multi-branch receptive field feature enhancement module specifically perform the following steps: First, a standard convolutional layer with a convolution kernel size of 1×1 is used to reduce the number of channels of each input feature map, reducing the number of channels of the feature map to 1 / 4 of the original number, and the feature maps after channel number reduction are used as the input feature maps M1, M2, M3 and M4 of the four branches respectively; In the first branch, the input feature map M1 is processed using multi-layer dilated convolution with a convolution kernel size of 3×3 and dilation rates of 1, 2, and 4, respectively, to obtain the feature map M1* output by the first branch; In the second branch, the input feature map M2 is first processed using two layers of asymmetric convolution with kernel sizes of 1×3 and 3×1, and then the group fusion attention module is used to extract features to obtain the feature map M2* output by the second branch; In the third branch, the group fusion attention module is used to extract features from the input feature map M3 to obtain the feature map M3* output by the third branch; In the fourth branch, a convolution with a convolution kernel size of 3×3 is used to extract features from the input feature map M4 to obtain the feature map M4* output by the fourth branch; Afterwards, M1*, M2*, M3* and M4* are fused, and the number of channels of the fused feature map is transformed through a 1×1 convolution layer to obtain an output feature map with the same number of channels as the input feature map; Step 3: Input the multi-scale output feature maps P1, P2 and P3 into YOLO Head to obtain the detection results; The group fusion attention module specifically performs the following steps: Step 21: First, split the input feature map into N sub-tensors, each of which has a shape of H×W×C / N, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map; Step 22: For each sub-tensor, each sub-tensor is subjected to feature extraction through two branches to obtain the output feature map of each sub-tensor, as follows: In the first branch, the input sub-tensor F1 is processed by global average pooling and global adaptive maximum pooling to obtain global average information and global maximum information, and then the global average information and the global maximum information are fused to obtain fused features. After the fused features are processed by a 1×1 convolution layer, the attention weight is generated by the Sigmoid activation function, and then the attention weight is multiplied by the sub-tensor F1, and the weighted feature map F1* is generated through the residual connection, which is expressed as: F1 * =F1×σ(F1) Among them, σ represents the activation function; In the second branch, the input sub-tensor F2 is first processed using the Haar wavelet downsampling and dynamic upsampling operators to obtain a feature map, and then the obtained feature map is processed by a 1×1 convolution layer, and the attention weight is generated by the Sigmoid activation function. After that, the input sub-tensor F2 passes through a 3×3 convolution layer and is weighted with the attention weight to generate a weighted feature map F2*, which is expressed as: <h2 style=";text-align:left;direction:ltr">F2<h2 style=";text-align:left;direction:ltr"> * <h2 style=";text-align:left;direction:ltr"> (F2×σ(F2)) Finally, the outputs F1* and F2* of the two branches are concatenated to form the output feature map of each sub-tensor; Step 2 and 3: Concatenate the output feature maps of all sub-tensors to obtain the final output feature map.
10. A detection system for small target ships in SAR remote sensing images according to claim 9, characterized in that The global average pooling is expressed as: Among them, x ij Represents the value of feature graph x at the i-th row and j-th column; The global adaptive maximum pooling is expressed as: GMP(x)=max i,j x ij ; The Haar wavelet downsampling is expressed as: Among them, l represents the original image, LL represents the low-frequency component, and LH, HL and HH represent the high-frequency components; The dynamic upsampling operator is expressed as: Among them, y i represents the output pixel after upsampling, represents the sampling point in the input feature map, w ij represents the weight, f(·) represents the convolutional layer, J represents the number of sampling points; Said It is expressed as: Among them, s represents the upsampling ratio, Δp ij Indicates the offset.
Citation Information
Patent Citations
Satellite-borne optical remote sensing image ship target detection method based on lightweight receptive field pyramid
CN110796037A
SAR ship target detection method based on network pruning and knowledge distillation
CN112308019A
Intelligent ship target detection method based on remote sensing image
CN114972851A
Remote sensing image target detection method and system based on improved YOLOV8 algorithm
CN117197669A
SAR image vessel detection method based on deep fusion optimization feature enhancement network
CN118212523A
Cited By
Small target detection method and device and storage medium
CN119919739A
Remote sensing image target detection method based on adaptive downsampling and scale enhancement detection head
CN120032249A
SAR remote sensing image ship detection method based on double-backbone network
CN121259587A
