Lightweight SAR (Synthetic Aperture Radar) image ship detection method based on feature guidance fusion

Abstract

The invention discloses a lightweight SAR image ship detection method based on feature guidance fusion. The lightweight SAR image ship detection method comprises the steps of image preprocessing, target detection network establishment, target detection network training and ship target detection in an image. According to the method, features are extracted stage by stage through a backbone network, and the features are fused through a PAN network; according to the backbone network, two lightweight modules including a lightweight downsampling module LDSM and an attention enhancement module LACSP are used for feature extraction, a lightweight convolution module is used for realizing model lightweight, and an attention mechanism is used for enhancing representation of a ship target. Carrying out feature fusion by using an attention guidance fusion module AGFM; the high-level features are used as attention to process the low-level features and then are fused with the low-level features, so that interference of complex backgrounds can be effectively suppressed. According to the method, the ship target in any given SAR image can be accurately detected.

Description

Technical Field

[0001] The invention relates to a lightweight SAR image ship detection method, in particular to a lightweight SAR image ship detection method based on feature-guided fusion, and belongs to the technical field of computer vision. Background Art

[0002] my country has vast territorial waters, and the detection of ships at sea is particularly important. In the military field, the location detection of specific targets is conducive to tactical deployment and improving the coastal defense early warning capability; in the civilian area, the detection of certain illegal fishing vessels is helpful for the monitoring and management of shipping. SAR image ship detection is an important way of ship detection. However, due to its special imaging mechanism, SAR images have problems such as feature diversity, severe image noise, and low clarity. There are certain challenges in accurately detecting ship targets from SAR images.

[0003] Traditional SAR image ship detection methods are mainly based on the Constant False Alarm Rate (CFAR) algorithm. When the scene is relatively simple, the CFAR method can achieve good results. For small ships and complex marine scenes, there will be more false alarms and poor detection performance. With the rapid development of deep learning in the field of computer vision, a large number of deep learning-based detection methods have emerged. However, the current deep learning-based target detection algorithm has too much computing resource requirements and serious memory consumption, which makes the cost high, and is not conducive to deployment on embedded edge devices with limited computing performance and storage space to achieve real-time monitoring. Lightweight convolution kernels can effectively reduce the amount of computation and parameters of convolutional neural networks without redesigning the neural network architecture. Lightweight convolution provides a solution for achieving network lightweighting. Summary of the invention

[0004] The purpose of the present invention is to propose a lightweight SAR image ship detection method based on feature-guided fusion.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] A lightweight SAR image ship detection method based on feature-guided fusion, comprising the following steps:

[0007] Step 1: Image preprocessing: scaling the original image to a predetermined standard size to obtain a preprocessed image;

[0008] Step 2: Establish the target detection network: The target detection network includes the backbone network Backbone, the neck network Neck and the head network Head; the backbone network Backbone includes the sequentially cascaded downsampling module Focus, the lightweight downsampling module LDSM1, the attention enhancement module LACSP1, the lightweight downsampling module LDSM2, the attention enhancement module LACSP2, the lightweight downsampling module LDSM3, the attention enhancement module LACSP3, the lightweight downsampling module LDSM4, the pyramid pooling module SPP, and the attention enhancement module LACSP4; the neck network Neck includes the sequentially cascaded Conv1 module Block, upsampling module UpSample1 module, attention guidance fusion module AGFM1, cross-scale connection module CSP1, Conv2 module, upsampling module UpSample2 module, attention guidance fusion module AGFM2, cross-scale connection module CSP2, lightweight downsampling module LDSM5, attention guidance fusion module AGFM3, cross-scale connection module CSP3, lightweight downsampling module LDSM6, attention guidance fusion module AGFM4, cross-scale connection module CSP4; the head network Head includes Detect1 module, Detect2 module, and Detect3 module which are cascaded in sequence;

[0009] The corresponding output ends of the attention enhancement module LACSP2, the attention enhancement module LACSP3, and the attention enhancement module LACSP4 are respectively connected to the corresponding input ends of the attention guidance fusion module AGFM2, the attention guidance fusion module AGFM1, and the Conv1 module; the corresponding input ends of the attention guidance fusion module AGFM2 and the attention guidance fusion module AGFM1 are respectively connected to the corresponding output ends of the upsampling module UpSample2 and the upsampling module UpSample1; the corresponding input ends of the Detect1 module, the Detect2 module, and the Detect3 module are respectively connected to the corresponding output ends of the cross-scale connection module CSP2, the cross-scale connection module CSP3, and the cross-scale connection module CSP4;

[0010] Step 3: Model training: Input the preprocessed images into the target detection network, train the target detection network for a specified number of rounds, and select the model parameters with the best preset performance indicators as the parameters of the trained target detection network;

[0011] Step 4: Detect ship targets in the image: Input the SAR image into the trained target detection network, complete target detection, perform non-maximum suppression to remove overlapping target boxes, retain the most credible target box, and output the detection result.

[0012] Furthermore, the lightweight downsampling modules LDSM1 to LDSM6 have the same structure; the lightweight downsampling module LDSM1 includes a lightweight feature extraction branch and a convolution branch. The lightweight feature extraction branch uses a 3x3 depth convolution DWConv to extract feature 1. The convolution branch uses a convolution module Conv3 to reduce the number of channels. The output of the convolution module Conv3 is respectively subjected to the maximum pooling MaxPool module and the average pooling AvgPool module to extract feature 2 and feature 3. Feature 1 to feature 3 are merged using a splicing operation and then a channel shuffling CS module is used to increase the connection between channels to achieve information exchange between groups. By reorganizing the channel order of the feature map, a feature map F with rich feature representation is obtained. d .

[0013] Furthermore, the attention enhancement modules LACSP1 to LACSP4 have the same structure, which are used to extract and fuse input features and enrich the semantic information of features; the attention enhancement module LACSP1 divides the input features into two branches, one of which is processed by the convolution module Conv4 to reduce the channel dimension, and then processed by the lightweight Bottleneck module; the other branch only uses the convolution module Conv5 to reduce the channel dimension of the input features, and the processing results of the two branches are merged by the splicing module Concat, and then the channel dimension is restored by the convolution module Conv6, and then the feature is enhanced by the attention mechanism module BAM.

[0014] Furthermore, BAM includes a channel attention branch and a spatial attention branch. The input feature F passes through the spatial attention branch and the channel attention branch to obtain spatial attention and channel attention, and the two are added and then passed through the Sigmoid function to obtain BAM attention. The obtained BAM attention is multiplied with the original feature, and then added to the original feature through a residual connection to obtain the feature after attention enhancement.

[0015] Furthermore, the attention-guided fusion modules AGFM1 to AGFM4 have the same structure and are used to fuse the longitudinally transmitted features with the transversely transmitted features in the neck network Neck. The longitudinally transmitted features are processed by the channel attention module Channel attention, multiplied with the transversely transmitted features, and then concatenated with the longitudinally transmitted features. The fused features are then output after channel dimension reduction by the convolution module Conv.

[0016] By adopting the above technical solution, the present invention achieves the following technical effects:

[0017] The present invention realizes accurate detection of ships while realizing a lightweight network architecture, showing good practicality. A lightweight downsampling module LDSM is proposed to replace the standard convolution to extract features, which reduces the number of parameters and computations while ensuring detection performance. A lightweight attention enhanced CSP module LACSP is proposed to further lightweight the network, and the attention mechanism is used to learn dependent features from the channel dimension and the spatial dimension respectively, which enhances the representation of ship targets. In order to reduce the interference of complex backgrounds in SAR images, an attention guided fusion module AGFM is designed, which uses high-level features as attention to process low-level features and then fuses them, significantly improving the detection performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a framework diagram of the present invention.

[0019] Figure 2 It is a structural diagram of the lightweight downsampling module LDSM of the present invention.

[0020] Figure 3 It is a structural diagram of the lightweight attention enhancement CSP module LACSP of the present invention.

[0021] Figure 4 It is a structural diagram of the attention guidance fusion module AGFM designed in the present invention.

[0022] Figure 5 It is the input image of embodiment 1 of the present invention.

[0023] Figure 6 This is a diagram of the test results of Example 1 of the present invention. DETAILED DESCRIPTION

[0024] The following examples serve to illustrate the invention.

[0025] Example 1

[0026] Reference Figure 1 , a lightweight SAR image ship detection method based on feature-guided fusion, comprising the following steps:

[0027] Step 1: Image preprocessing: scaling the original image to a preset standard size, which in this embodiment is 640×640×3;

[0028] Step 2: Build the target detection network:

[0029] The target detection network includes a backbone network Backbone, a neck network Neck and a head network Head; the backbone network Backbone includes a sequentially cascaded downsampling module Focus, a lightweight downsampling module LDSM1, an attention enhancement module LACSP1, a lightweight downsampling module LDSM2, an attention enhancement module LACSP2, a lightweight downsampling module LDSM3, an attention enhancement module LACSP3, a lightweight downsampling module LDSM4, a pyramid pooling module SPP, and an attention enhancement module LACSP4; the neck network Neck includes a sequentially cascaded convolution module Conv1, Upsampling module UpSample1, attention-guided fusion module AGFM1, cross-scale connection module CSP1, convolution module Conv2, upsampling module UpSample2, attention-guided fusion module AGFM2, cross-scale connection module CSP2, lightweight downsampling module LDSM5, attention-guided fusion module AGFM3, cross-scale connection module CSP3, lightweight downsampling module LDSM6, attention-guided fusion module AGFM4, cross-scale connection module CSP4; head network Head includes Detect1 module, Detect2 module, and Detect3 module;

[0030] The corresponding output ends of the attention enhancement modules LACSP2-LACSP4 are respectively connected to the corresponding input ends of the attention guidance fusion module AGFM2, the attention guidance fusion module AGFM1, and the convolution module Conv1; the corresponding input ends of the attention guidance fusion module AGFM2 and the attention guidance fusion module AGFM1 are respectively connected to the corresponding output ends of the upsampling module UpSample2 and the upsampling module UpSample1; the corresponding input ends of the Detect1 module, the Detect2 module, and the Detect3 module are respectively connected to the corresponding output ends of the cross-scale connection modules CSP2-CSP4;

[0031] The backbone network is used to extract feature maps of different sizes. The first feature processing unit includes Focus, LDSM, and LACSP structures. The feature processing units of the second and third stages include LDSM and LACSP structures. The fourth feature processing unit includes LDSM, LACSP, and SPP structures. LDSM is used to downsample the input, and LACSP is used to extract and fuse features of the input to enrich the semantic information of the features. In the first stage, the Focus structure is used to compress and combine the information in the input feature map to extract a higher-level feature representation. In the feature processing of the fourth stage, the SPP pyramid pooling structure is used to pool feature maps of different sizes, thereby enhancing the model's perception of targets of different scales. The backbone network extracts four feature maps of different sizes.

[0032] The feature maps obtained in the second, third and fourth stages of the backbone network are sent to the neck network for processing. The PAN structure is adopted, including two paths: top-down and bottom-up. The PAN structure consists of Conv, CSP, UpSample, LDSM, and AGFM modules. The three feature maps first convey strong semantic features from top to bottom, adjust the number of channels through Conv, and upsample through UpSample to align the scale of the feature map at a higher level with the scale of the feature map at a lower level, and then fuse them through the AGFM module, and finally output the fused feature map. Then the three feature maps convey the positioning features from bottom to top, use LSDM to downsample the shallow features, and fuse the downsampled feature maps with the high-level features through AGFM to obtain the final fused features. The neck network fully integrates the semantic information of the deep features and the shallow features in the feature maps of three scales. The head network processes the feature maps of three different scales obtained by the neck network to detect small, medium and large scale ship targets respectively;

[0033] Input image X, first through Focus to compress and combine the information in the input feature map, then through LDSM1, LACSP1, LDSM2 and LACSP2 for feature extraction to generate feature map C1, feature map C1 through LDSM3 and LACSP3 to obtain feature map C2, feature map C2 through LDSM4, SPP and LACSP4 to obtain feature map C3. Feature maps C1, C2 and C3 generated by the backbone network enter the neck network for feature fusion. In the neck network, feature maps C1, C2 and C3 first convey strong semantic features from bottom to top, feature map C3 adjusts the number of channels through Conv1 to obtain P3, P3 is upsampled through UpSample1 to align the scale of the higher-level feature map with the scale of the lower-level feature map, and then fused with C2 through the AGFM1 module, and then through the CSP1 and Conv2 modules to obtain the fused feature map P2, P2 is upsampled through UpSample2, and then fused with C1 through the AGFM2 module to obtain the fused feature map P1. Then the feature maps P1, P2, and P3 convey the positioning features from top to bottom. P1 passes through the CSP2 module to obtain D1. D1 is downsampled using LDSM5, and then fused with P2 through AGFM3, and then passes through the CSP3 module to obtain D2. D2 is downsampled using LSDM6, and then fused with P3 through AGFM4, and then passes through the CSP4 module to obtain D3. The feature maps D1, D2, and D3 generated by the neck network enter the head network and detect small, medium, and large scale ship targets through the Detect1, Detect2, and Detect3 modules respectively;

[0034] In this embodiment, the sizes of the three tensors C1, C2, and C3 are 80×80×128, 40×40×256, and 20×20×512, respectively.

[0035] Step 3: Model training: Input the preprocessed images into the target detection network, train the target detection network for a specified number of rounds, and select the model parameters with the best preset performance indicators as the parameters of the trained target detection network;

[0036] The number of training rounds set in this embodiment is 200 rounds.

[0037] Step 4: Detect ship targets in the image: Input the SAR image into the trained target detection network, complete target detection, perform non-maximum suppression to remove overlapping target boxes, retain the most credible target box, and output the detection result.

[0038] In order to achieve network lightweight, a lightweight downsampling module LDSM is used for downsampling. LDSM is used to extract and fuse input features to enrich the semantic information of features. LDSM divides the input feature F into two branches. One branch is a lightweight feature extraction branch that uses 3x3DWConv to extract features, significantly reducing the number of parameters and operations. The other branch uses Conv3 to reduce the number of channels, and then divides the features into two parts from the channel dimension. The maximum value pooling (MaxPool) and the average value pooling (AvgPool) are used to change the size of the features, and then the splicing operation is used to merge the results of the three branches. Finally, the channel shuffle (CS) operation is used to increase the connection between channels to achieve information exchange between groups. By reorganizing the channel order of the feature map, the feature map F with rich feature representation is obtained. d :

[0039]

[0040] In the formula, F, F d They represent the original feature map and the downsampled feature map respectively, CS represents the channel shuffle operation, and Concat represents the concatenation operation. represents a 3x3 depth convolution, Represents a standard 1x1 convolution.

[0041] In order to further make the network lightweight and maintain good detection performance, the lightweight attention enhanced CSP module LACSP is used for feature extraction. LACSP divides the input features into two branches, one of which is subjected to Conv4 to reduce the channel dimension, and then passes through a lightweight Bottleneck module, which is composed of 1x1Conv and 3x3 DWConv. The other branch only performs Conv5 on the features to reduce the channel dimension. The results of the two branches are merged by splicing, and then restored to the channel dimension through Conv6, and then enhanced by the BAM attention mechanism. Dependent features are learned from the channel dimension and spatial dimension respectively to enhance the representation of ship targets.

[0042] In order to reduce the interference of complex background in SAR images on ship targets, the attention-guided fusion module AGFM is used in the neck network to perform the longitudinal transfer of features F in the neck network. v With lateral transfer F h The features of the vertical transmission are fused. v First, it passes through a channel attention module, and the result is the same as the horizontally transmitted feature F h Perform multiplication operation to obtain the processed horizontal features, then concatenate the processed horizontal features with the original vertical features, and then perform channel dimension reduction through convolution to obtain the fused features F′. The fused features are calculated as follows:

[0043]

[0044] Among them, F h ,F v They represent the features transmitted in the horizontal direction and the features transmitted in the vertical direction respectively, and M c ′ represents the channel attention mechanism, Concat represents the concatenation operation, and F′ represents the fused feature.

[0045] Finally, the feature maps of different scales are upsampled to the original image size using bilinear interpolation. The scale becomes 640×640. The detected ship targets are annotated in the image using rectangular boxes.

[0046] At present, the technical solution of the present invention has been piloted, and a user survey has been conducted in a small range. The survey results show that user satisfaction is high, and preparations for the transformation and promotion of the results have begun. At the same time, a survey on intellectual property risk warning has been carried out.

Claims

1. A lightweight SAR image ship detection method based on feature-guided fusion, characterized by: The following steps are involved: Step 1: Image preprocessing: scaling the original image to a predetermined standard size to obtain a preprocessed image; Step 2: Establish the target detection network: The target detection network includes the backbone network Backbone, the neck network Neck and the head network Head; the backbone network Backbone includes the sequentially cascaded downsampling module Focus, the lightweight downsampling module LDSM1, the attention enhancement module LACSP1, the lightweight downsampling module LDSM2, the attention enhancement module LACSP2, the lightweight downsampling module LDSM3, the attention enhancement module LACSP3, the lightweight downsampling module LDSM4, the pyramid pooling module SPP, and the attention enhancement module LACSP4; the neck network Neck includes the sequentially cascaded convolution modules Conv1, upsampling module UpSample1, attention-guided fusion module AGFM1, cross-scale connection module CSP1, convolution module Conv2, upsampling module UpSample2, attention-guided fusion module AGFM2, cross-scale connection module CSP2, lightweight downsampling module LDSM5, attention-guided fusion module AGFM3, cross-scale connection module CSP3, lightweight downsampling module LDSM6, attention-guided fusion module AGFM4, cross-scale connection module CSP4; the head network Head includes Detect1 module, Detect2 module, and Detect3 module; The corresponding output ends of the attention enhancement modules LACSP2-LACSP4 are respectively connected to the corresponding input ends of the attention guidance fusion module AGFM2, the attention guidance fusion module AGFM1, and the convolution module Conv1; the corresponding input ends of the attention guidance fusion module AGFM2 and the attention guidance fusion module AGFM1 are respectively connected to the corresponding output ends of the upsampling module UpSample2 and the upsampling module UpSample1; the corresponding input ends of the Detect1 module, the Detect2 module, and the Detect3 module are respectively connected to the corresponding output ends of the cross-scale connection modules CSP2-CSP4; Step 3: Model training: Input the preprocessed images into the target detection network, train the target detection network for a specified number of rounds, and select the model parameters with the best preset performance indicators as the parameters of the trained target detection network; Step 4: Detect ship targets in the image: Input the SAR image into the trained target detection network, complete target detection, perform non-maximum suppression to remove overlapping target boxes, retain the most credible target box, and output the detection result.

2. The method for lightweight SAR image ship detection based on feature-guided fusion according to claim 1 is characterized in that: The lightweight downsampling modules LDSM1-LDSM6 have the same structure; the lightweight downsampling module LDSM1 includes a lightweight feature extraction branch and a convolution branch, the lightweight feature extraction branch uses a 3x3 deep convolution module DWConv to extract the first feature, the convolution branch uses a convolution module Conv3 to reduce the number of channels, the output end of the convolution module Conv3 is respectively connected to the input end of the maximum pooling MaxPool module and the average pooling AvgPool module, the maximum pooling MaxPool module and the average pooling AvgPool module extract the second feature and the third feature respectively, the first to the third features are spliced ​​and merged, and then the channel shuffling module CS increases the connection between the channels, reorganizes the channel order of the feature map, and outputs the feature map F d .

3. The lightweight SAR image ship detection method based on feature-guided fusion according to claim 1 is characterized by: The attention enhancement modules LACSP1-LACSP4 have the same structure; in the attention enhancement module LACSP1, the input features are divided into two branches for processing, one branch is processed by the convolution module Conv4 after the channel dimension is reduced and then input into the lightweight Bottleneck module for processing; the other branch is only processed by the convolution module Conv5 after the channel dimension is reduced and the processing result of the lightweight Bottleneck module is merged with the splicing module Concat, and then the channel dimension is restored by the convolution module Conv6, and then the feature is enhanced by the attention mechanism module BAM.

4. The lightweight SAR image ship detection method based on feature-guided fusion according to claim 1 is characterized by: The attention-guided fusion modules AGFM1-AGFM4 have the same structure. The features input from the first input end are processed by the channel attention module Channelattention, multiplied by the features input from the second input end, and then concatenated with the features input from the first input end, and then output the fused features through the convolution module Conv.

5. The lightweight SAR image ship detection method based on feature-guided fusion according to claim 3 is characterized by: The attention mechanism module BAM includes a channel attention branch and a spatial attention branch. The input features are respectively passed through the spatial attention branch and the channel attention branch to obtain spatial attention and channel attention. The two are added and then passed through the Sigmoid function to obtain BAM attention. The obtained BAM attention is multiplied with the original feature, and then added to the original feature through a residual connection to obtain the feature after attention enhancement.

Images