Lightweight sar image ship detection method based on feature guided fusion

Abstract

The application discloses a kind of light SAR image ship detection methods based on feature guidance fusion, including image pre-processing, establishing target detection network, training target detection network and detecting the steps of ship target in image.The application extracts features by backbone network stage by stage, and fuses features by PAN network;The backbone network uses two light modules, LDSM and LACSP, for feature extraction, uses light convolution module to realize model lightening, and uses attention mechanism to enhance the representation of ship target. Feature fusion is carried out using attention guidance fusion module AGFM;High-level features are processed by attention to low-level features and fused with low-level features, which can effectively suppress the interference of complex background. The application can accurately detect the ship target in any given SAR image.

Description

TECHNICAL FIELD

[0001] The application relates to a lightweight SAR image ship detection method, in particular to a lightweight SAR image ship detection method based on feature guidance fusion, and belongs to the technical field of computer vision. BACKGROUND

[0002] China has a vast sea area, and the detection of marine ship targets is particularly important. In the military field, detecting the position of a specific target is beneficial to tactical deployment and improves the sea defense warning capability; in the civil field, detecting some illegal fishing vessels is helpful for the monitoring and management of sea transportation. SAR image ship detection is an important way of ship detection, however, due to the special imaging mechanism of SAR images, there are problems such as feature diversity, serious image noise and low clarity, and it is challenging to accurately detect ship targets from SAR images.

[0003] Traditional SAR image ship detection methods are mainly based on the constant false alarm rate (CFAR) detection algorithm. When the scene is relatively simple, the CFAR method can achieve good results. For small ships and complex marine scenes, there will be more false positives and poor detection performance. With the rapid development of deep learning in the field of computer vision, a large number of detection methods based on deep learning have emerged. However, the current target detection algorithm based on deep learning has too large a demand for computing resources and serious memory consumption, which makes the cost high, and is not conducive to deployment on embedded edge devices with limited computing performance and storage space to realize real-time monitoring. Lightweight convolution kernels can effectively reduce the computational amount and parameter amount of convolutional neural networks without the need to redesign the neural network architecture, and lightweight convolution provides a solution for network lightweight. SUMMARY

[0004] The purpose of the application is to provide a lightweight SAR image ship detection method based on feature guidance fusion.

[0005] To solve the above technical problems, the technical scheme adopted by the application is as follows:

[0006] A lightweight SAR image ship detection method based on feature guidance fusion comprises the following steps:

[0007] Step 1: image preprocessing: scale the original picture to a predetermined standard size to obtain a preprocessed picture;

[0008] Step 2: Establishing the target detection network: the target detection network includes a backbone network Backbone, a neck network Neck and a head network Head; the backbone network Backbone includes a down-sampling module Focus, a light down-sampling module LDSM1, an attention enhancement module LACSP1, a light down-sampling module LDSM2, an attention enhancement module LACSP2, a light down-sampling module LDSM3, an attention enhancement module LACSP3, a light down-sampling module LDSM4, a pyramid pooling module SPP and an attention enhancement module LACSP4 which are sequentially cascaded; the neck network Neck includes a Conv1 module, an up-sampling module UpSample1, an attention guidance fusion module AGFM1, a cross-scale connection module CSP1, a Conv2 module, an up-sampling module UpSample2, an attention guidance fusion module AGFM2, a cross-scale connection module CSP2, a light down-sampling module LDSM5, an attention guidance fusion module AGFM3, a cross-scale connection module CSP3, a light down-sampling module LDSM6, an attention guidance fusion module AGFM4 and a cross-scale connection module CSP4 which are sequentially cascaded; and the head network Head includes a Detect1 module, a Detect2 module and a Detect3 module which are sequentially cascaded.

[0009] The respective output ends of the attention enhancement module LACSP2, the attention enhancement module LACSP3 and the attention enhancement module LACSP4 are connected to the respective input ends of the attention guidance fusion module AGFM2, the attention guidance fusion module AGFM1 and the Conv1 module respectively; the respective input ends of the attention guidance fusion module AGFM2 and the attention guidance fusion module AGFM1 are connected to the respective output ends of the up-sampling module UpSample2 and the up-sampling module UpSample1 respectively; and the respective input ends of the Detect1 module, the Detect2 module and the Detect3 module are connected to the respective output ends of the cross-scale connection module CSP2, the cross-scale connection module CSP3 and the cross-scale connection module CSP4 respectively.

[0010] Step 3: Model training: inputting the pre-processed picture into the target detection network, performing specified rounds of training on the target detection network, and selecting the model parameters with the best preset performance index as the parameters of the trained target detection network.

[0011] Step 4: Detecting the ship target in the image: inputting the SAR image into the trained target detection network, completing target detection, performing non-maximum suppression to remove overlapping target boxes, retaining the most credible target box, and outputting the detection result.

[0012] Further, the lightweight downsampling modules LDSM1 to LDSM6 are of the same structure; the lightweight downsampling module LDSM1 comprises a lightweight feature extraction branch and a convolution branch, the lightweight feature extraction branch adopts a 3x3 deep convolution DWConv to extract feature 1, the convolution branch uses a convolution module Conv3 to reduce the number of channels, the output of the convolution module Conv3 is respectively extracted as feature 2 and feature 3 through a maximum value pooling MaxPool module and an average value pooling AvgPool module, feature 1 to feature 3 are combined through a splicing operation and then a channel shuffle CS module is used to increase the connection between channels, realize inter-group information exchange, and obtain a feature map F with rich feature representation by reorganizing the channel order of the feature map d .

[0013] Further, the attention enhancement modules LACSP1 to LACSP4 are of the same structure, and are used for feature extraction and fusion of input, and rich semantic information of features; the attention enhancement module LACSP1 divides input features into two branches, one of which is processed through a convolution module Conv4 to reduce the dimension of channels, and then processed through a lightweight bottleneck module; the other branch is only processed through a convolution module Conv5 to reduce the dimension of channels of input features, the processing results of the two branches are combined through a splicing module Concat, and then processed through a convolution module Conv6 to restore the dimension of channels, and then processed through an attention mechanism module BAM to enhance features.

[0014] Further, the BAM comprises a channel attention branch and a spatial attention branch, input features F are respectively processed through the spatial attention branch and the channel attention branch to obtain spatial attention and channel attention, the two are added and then processed through a Sigmoid function to obtain BAM attention. The obtained BAM attention is multiplied by the original features, and then added to the original features through a residual connection to obtain features after attention enhancement.

[0015] Further, the attention guidance fusion modules AGFM1 to AGFM4 are of the same structure, and are used for fusing features transmitted longitudinally and features transmitted transversely in the neck network Neck, the features transmitted longitudinally are processed through a channel attention module Channel attention, multiplied with the features transmitted transversely, then spliced with the features transmitted longitudinally, and then output fusion features through a convolution module Conv to reduce the dimension of channels.

[0016] By adopting the above technical solutions, the present application achieves the following technical effects:

[0017] The application realizes accurate detection of ships while realizing a lightweight network architecture, and exhibits good practicability. A lightweight down-sampling module LDSM is proposed to replace standard convolution to extract features, which reduces the parameter amount and operation amount while ensuring detection performance. A lightweight attention enhanced CSP module LACSP is proposed to further lighten the network, and the attention mechanism is used to learn dependent features from the channel dimension and the spatial dimension respectively, which enhances the representation of the ship target. In order to reduce the interference of the complex background in the SAR image, an attention guided fusion module AGFM is designed, which processes the low-level features by using the high-level features as attention and then fuses them, which significantly improves the detection performance of the model. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a framework diagram of the application.

[0019] Figure 2 is a structure diagram of the lightweight down-sampling module LDSM of the application.

[0020] Figure 3 is a structure diagram of the lightweight attention enhanced CSP module LACSP of the application.

[0021] Figure 4 is a structure diagram of the attention guided fusion module AGFM designed in the application.

[0022] Figure 5 is an input image of embodiment 1 of the application.

[0023] Figure 6 is a detection result diagram of the detection of embodiment 1 of the application. DETAILED DESCRIPTION

[0024] The following examples are used to illustrate the application.

[0025] Embodiment 1

[0026] Referring to Figure 1 A lightweight SAR image ship detection method based on feature guided fusion includes the following steps:

[0027] Step 1: image preprocessing: scale the original picture to a preset standard size, and in this embodiment, the size is 640x640x3;

[0028] Step 2: establish a target detection network:

[0029] The target detection network comprises a backbone network, a neck network and a head network; the backbone network comprises, in sequence, a downsampling module Focus, a light downsampling module LDSM1, an attention enhancement module LACSP1, a light downsampling module LDSM2, an attention enhancement module LACSP2, a light downsampling module LDSM3, an attention enhancement module LACSP3, a light downsampling module LDSM4, a pyramid pooling module SPP and an attention enhancement module LACSP4; the neck network comprises, in sequence, a convolution module Conv1, an upsampling module UpSample1, an attention guidance fusion module AGFM1, a cross-scale connection module CSP1, a convolution module Conv2, an upsampling module UpSample2, an attention guidance fusion module AGFM2, a cross-scale connection module CSP2, a light downsampling module LDSM5, an attention guidance fusion module AGFM3, a cross-scale connection module CSP3, a light downsampling module LDSM6, an attention guidance fusion module AGFM4 and a cross-scale connection module CSP4; and the head network comprises a Detect1 module, a Detect2 module and a Detect3 module.

[0030] Respective output ends of the attention enhancement modules LACSP2-LACSP4 are connected to respective input ends of the attention guidance fusion modules AGFM2, AGFM1 and the convolution module Conv1 respectively; respective input ends of the attention guidance fusion modules AGFM2 and AGFM1 are connected to respective output ends of the upsampling modules UpSample2 and UpSample1 respectively; and respective input ends of the Detect1 module, the Detect2 module and the Detect3 module are connected to respective output ends of the cross-scale connection modules CSP2-CSP4 respectively.

[0031] The backbone network is used for extracting feature maps of different sizes, the first feature processing unit comprises the Focus, the LDSM and the LACSP structure, the second and third stage feature processing units comprise the LDSM and the LACSP structure, and the fourth feature processing unit comprises the LDSM, the LACSP and the SPP structure; the LDSM is used for downsampling the input, and the LACSP is used for extracting and fusing the input to enrich the semantic information of the features. In the first stage, the Focus structure is used to compress and combine the information in the input feature map, thereby extracting a higher level feature representation. In the fourth stage feature processing, the SPP pyramid pooling structure is used to pool the feature maps of different sizes, thereby enhancing the perception ability of the model to targets of different scales. The backbone network extracts four feature maps of different sizes.

[0032] The feature maps obtained in the second, third and fourth stages of the backbone network are sent to the neck network for processing. A PAN structure including a top-down and bottom-up path is adopted. The PAN structure is composed of Conv, CSP, UpSample, LDSM and AGFM modules. The three feature maps first convey strong semantic features in a top-down manner, the channel number is adjusted through Conv, and the scale of the feature maps at a higher level is aligned with that of the feature maps at a lower level through UpSample, then they are fused through the AGFM module, and finally the fused feature maps are output. Then the three feature maps convey positioning features in a bottom-up manner, the shallow features are down-sampled using the LSDM, and the down-sampled feature maps are fused with the high-level features through the AGFM to obtain the finally fused features. The neck network fully fuses the semantic information of the deep features and the shallow features in the feature maps at three scales. The head network processes the three feature maps at different scales obtained by the neck network to detect small, medium and large ship targets respectively;

[0033] The input picture X is first compressed and combined through Focus to compress and combine the information in the input feature map, and then sequentially passes through LDSM1, LACSP1, LDSM2 and LACSP2 to extract features to generate feature map C1, feature map C1 is sequentially processed through LDSM3 and LACSP3 to obtain feature map C2, and feature map C2 is sequentially processed through LDSM4, SPP and LACSP4 to obtain feature map C3. The feature maps C1, C2 and C3 generated by the backbone network enter the neck network for feature fusion. In the neck network, the feature maps C1, C2 and C3 first convey strong semantic features in a top-down manner, the feature map C3 is adjusted in channel number through Conv1 to obtain P3, P3 is up-sampled through UpSample1 to align the scale of the feature maps at a higher level with that of the feature maps at a lower level, then it is fused with C2 through the AGFM1 module, and then it is processed through the CSP1 and Conv2 modules to obtain the fused feature map P2, P2 is up-sampled through UpSample2, and then it is fused with C1 through the AGFM2 module to obtain the fused feature map P1. Then the feature maps P1, P2 and P3 convey positioning features in a top-down manner, P1 passes through the CSP2 module to obtain D1, D1 is down-sampled using the LDSM5, then it is fused with P2 through the AGFM3, and then it is processed through the CSP3 module to obtain D2, D2 is down-sampled using the LSDM6, then it is fused with P3 through the AGFM4, and then it is processed through the CSP4 module to obtain D3. The feature maps D1, D2 and D3 generated by the neck network enter the head network and are detected through Detect1, Detect2 and Detect3 modules to detect small, medium and large ship targets respectively;

[0034] The sizes of the three tensors C1, C2 and C3 in the embodiment are 80x80x128, 40x40x256 and 20x20x512 respectively.

[0035] Step 3: Model training: input the preprocessed picture into the target detection network, train the target detection network for a specified number of rounds, and select the model parameters with the best preset performance indicators as the parameters of the trained target detection network.

[0036] In the embodiment, the training rounds are set to 200 rounds,

[0037] Step 4: Detecting the ship target in the image: input the SAR image into the trained target detection network, complete target detection, perform non-maximum suppression to remove overlapping target boxes, retain the most credible target box, and output the detection result.

[0038] In order to realize network lightweight, a light down-sampling module (LDSM) is used for down-sampling. The LDSM is used for feature extraction and fusion of input, and enriches the semantic information of features. The input feature F is divided into two branches by the LDSM. One branch is a light feature extraction branch, which uses 3x3 DWConv to extract features, significantly reducing the parameter quantity and operation quantity. The other branch uses Conv3 to reduce the channel number, and then divides the features from the channel dimension into two parts, which are subjected to MaxPool and AvgPool respectively to change the size of the features. Then, the results of the three branches are merged by using a splicing operation, and finally, a channel shuffle (CS) operation is used to increase the connection between channels, realize group information exchange, and obtain a feature map F with rich feature representation by reorganizing the channel order of the feature map. d :

[0039]

[0040] In the formula, F, F d represent the original feature map and the down-sampled feature map respectively, CS represents the channel shuffling operation, Concat represents the splicing operation, represents 3x3 deep convolution, represents 1x1 standard convolution.

[0041] To further reduce network weight while maintaining good detection performance, a lightweight attention-enhanced CSP module, LACSP, is used for feature extraction. LACSP divides the input features into two branches. One branch performs channel dimensionality reduction via Conv4, followed by a lightweight Bottleneck module, which consists of a 1x1 Conv and a 3x3 DWConv. The other branch only performs channel dimensionality reduction via Conv5. The results from the two branches are concatenated and then processed through Conv6 to restore the channel dimension. Finally, a BAM attention mechanism is used for feature enhancement, learning dependent features from both the channel and spatial dimensions to improve the representation of ship targets.

[0042] To reduce the interference of complex backgrounds on ship targets in SAR images, the attention-guided fusion module AGFM is used in the neck network to optimize the longitudinally propagated features F in the neck network. v With lateral transfer F h The features are fused, and the vertically transmitted feature F is used to perform feature fusion. v First, the data passes through a channel attention module, and the result is then compared with the laterally propagated feature F. h A multiplication operation is performed to obtain the processed horizontal features. These horizontal features are then concatenated with the original vertical features, and finally, convolution is used for channel dimensionality reduction to obtain the fused feature F′. The fused feature is calculated as follows:

[0043]

[0044] Among them, F h ,F v M represents the characteristics transmitted in the horizontal direction and the characteristics transmitted in the vertical direction, respectively. c F′ represents the channel attention mechanism, Concat represents the concatenation operation, and F′ represents the fused features.

[0045] Finally, feature maps at different scales were upsampled using bilinear interpolation to restore them to their original image size. The scale became 640×640. Detected ship targets were labeled in the image using rectangular boxes.

[0046] Currently, the technical solution of this invention has undergone pilot testing, and user surveys have been conducted on a small scale. The survey results show that user satisfaction is high, and preparations for the commercialization and promotion of the results have begun. Simultaneously, a survey on intellectual property risk warnings has been conducted.

Claims

1. A method for light-weight SAR image ship detection based on feature-guided fusion, characterized in that: The method comprises the following steps: Step 1: image preprocessing: scaling the original picture to a predetermined standard size to obtain a preprocessed picture; Step 2: establishing a target detection network: the target detection network comprises a backbone network, a neck network and a head network; the backbone network comprises a focus downsampling module, a light downsampling module LDSM1, an attention enhancement module LACSP1, a light downsampling module LDSM2, an attention enhancement module LACSP2, a light downsampling module LDSM3, an attention enhancement module LACSP3, a light downsampling module LDSM4, a pyramid pooling module SPP and an attention enhancement module LACSP4 which are sequentially cascaded; the neck network comprises a convolution module Conv1, an up-sampling module UpSample1, an attention guidance fusion module AGFM1, a cross-scale connection module CSP1, a convolution module Conv2, an up-sampling module UpSample2, an attention guidance fusion module AGFM2, a cross-scale connection module CSP2, a light downsampling module LDSM5, an attention guidance fusion module AGFM3, a cross-scale connection module CSP3, a light downsampling module LDSM6, an attention guidance fusion module AGFM4 and a cross-scale connection module CSP4 which are sequentially cascaded; and the head network comprises a Detect1 module, a Detect2 module and a Detect3 module; The respective output ends of the attention enhancement modules LACSP2-LACSP4 are connected to the respective input ends of the attention guidance fusion modules AGFM2, AGFM1 and the convolution module Conv1 respectively; the respective input ends of the attention guidance fusion modules AGFM2 and AGFM1 are connected to the respective output ends of the up-sampling modules UpSample2 and UpSample1 respectively; and the respective input ends of the Detect1 module, the Detect2 module and the Detect3 module are connected to the respective output ends of the cross-scale connection modules CSP2-CSP4 respectively; Step 3: model training: inputting the preprocessed picture into the target detection network, performing specified rounds of training on the target detection network, and selecting the model parameters with the best preset performance index as the parameters of the trained target detection network; Step 4: detecting a ship target in an image: inputting a SAR image into the trained target detection network, completing target detection, performing non-maximum suppression to remove overlapping target boxes, retaining the most credible target box, and outputting a detection result.

2. The lightweight SAR image ship detection method based on feature-guided fusion according to claim 1, characterized in that: The lightweight down-sampling modules LDSM1-LDSM6 are of the same structure; the lightweight down-sampling module LDSM1 comprises a lightweight feature extraction branch and a convolution branch, the lightweight feature extraction branch uses a 3x3 deep convolution module DWConv to extract first features, the convolution branch uses a convolution module Conv3 to reduce the number of channels, the output end of the convolution module Conv3 is connected to the input end of a maximum value pooling MaxPool module and an average value pooling AvgPool module respectively, the maximum value pooling MaxPool module and the average value pooling AvgPool module extract second features and third features respectively, after the first to third features are spliced and combined, a channel shuffle module CS is used to increase the connection between channels and reorganize the channel order of the feature map, and an output feature map F is output d . 3.The lightweight SAR image ship detection method based on feature-guided fusion according to claim 1, characterized in that: The attention enhancement modules LACSP1-LACSP4 are identical in structure; in the attention enhancement module LACSP1, input features are processed in two branches, one branch is input into a light bottleneck module after being reduced in dimension by a convolution module Conv4, and the other branch is only reduced in dimension by a convolution module Conv5 and then combined with the processing result of the light bottleneck module, and then the combined result is restored in dimension by a convolution module Conv6, and then the features are enhanced by an attention mechanism module BAM.

4. The lightweight SAR image ship detection method based on feature-guided fusion according to claim 1, characterized in that: The attention guidance fusion modules AGFM1-AGFM4 are identical in structure, the features input from the first input end are processed by a channel attention module Channelattention, multiplied by the features input from the second input end, spliced with the features input from the first input end, and then output fusion features by a convolution module Conv.

5. The lightweight SAR image ship detection method based on feature-guided fusion according to claim 3, characterized in that: The attention mechanism module BAM includes a channel attention branch and a spatial attention branch, input features are respectively processed by the spatial attention branch and the channel attention branch to obtain spatial attention and channel attention, the two are added and then passed through a Sigmoid function to obtain BAM attention; the obtained BAM attention is multiplied by the original features, and then added to the original features through a residual connection to obtain features enhanced by attention.

Images