A sparse SAR ship target detection method and device based on attention feature fusion

Through an improved Faster R-CNN network based on attention feature fusion, combined with AMP algorithm and wide residual network, the problems of missed detection and false detection in SAR ship target detection are solved, achieving higher detection accuracy and speed, and are applicable In real-time maritime monitoring.

CN117372676BActive Publication Date: 2024-11-19NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311249489.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-26
Publication Date
2024-11-19
Estimated Expiration
2043-09-26

AI Technical Summary

Technical Problem

The existing SAR ship target detection algorithm has problems of missed detection and misdetection when dealing with weak targets, and the detection speed of the two-stage target detection algorithm is slow, making it difficult to meet the needs of real-time maritime monitoring.

Method used

Using an improved Faster R-CNN object detection network based on attention feature fusion, SAR images are sparsely reconstructed through AMP algorithm, combining wide residual network and feature pyramids of attention mechanism, different scale features are extracted and candidate target boxes are generated. 使用ROIAlign层提高回归精度,减少噪声并增强检测能力。

Benefits of technology

It improves the accuracy of SAR ship target detection, reduces missed and misdetection, enhances the detection ability of weak targets, and improves network performance and detection speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117372676B_ABST
    Figure CN117372676B_ABST
Patent Text Reader

Abstract

The present invention discloses a sparse SAR ship target detection method and device based on attention feature fusion, which reconstructs the pre-acquired SAR image into a sparse SAR image based on the AMP algorithm; constructs a target detection network based on the improved Faster R-CNN; the feature extraction network uses a wide residual network WRN to extract feature maps of different scales of the sparse SAR ship image; the extracted feature maps of different scales are input into a feature pyramid structure based on the attention mechanism to achieve feature fusion of feature maps of different scales; the region generation network is used to generate candidate target boxes, which are considered to contain objects of interest; the fused feature maps are input into the ROIAlign layer to improve the regression accuracy; the sparse SAR image is used as the input of the improved Faster R-CNN target detection network for training to obtain the target detection result. The present invention improves the detection capability of weak targets and further improves the target detection accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of radar image processing and target detection, and in particular relates to a sparse SAR ship target detection method and device based on attention feature fusion. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active remote sensing system that works in the microwave band. The main advantages of SAR are that it can work all day and all weather, and the transmitted signal has strong penetration ability. SAR ship detection is crucial for maritime monitoring and has broad application prospects in both civil and military fields.

[0003] SAR images have great potential in target detection and recognition because of their special imaging mode and are sensitive to the height information of ground objects. The emergence of deep learning technology has greatly improved the accuracy and robustness of SAR target detection. Target detection algorithms based on deep learning are mainly divided into two categories: two-stage target detection algorithms and single-stage target detection algorithms. The two-stage target detection algorithm first generates a series of candidate frames as samples by the feature extraction network, and then performs classification and regression by the convolutional neural network. Its representative algorithms mainly include: R-CNN series, SPP-Net and Mask R-CNN. The single-stage target detection algorithm directly predicts the category and position of the target by extracting features in the neural network. Its representative algorithms mainly include: YOLO series, SSD series, RetinaNet. Compared with the single-stage target detection algorithm, the two-stage target detection algorithm has a slower detection speed, but can achieve better detection accuracy. Therefore, the present invention adopts the classic two-stage target detection algorithm-Faster R-CNN. Aiming at the characteristic of a high proportion of small and weak targets in the SAR ship data set, this paper proposes an improved Faster R-CNN target detection network based on attention feature fusion on the basis of the Faster R-CNN algorithm to realize SAR ship detection. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a sparse SAR ship target detection method and device based on attention feature fusion, so as to improve the accuracy of SAR ship target detection and avoid missed detection and false detection.

[0005] Technical solution: The sparse SAR ship target detection method based on attention feature fusion described in the present invention specifically includes the following steps:

[0006] (1) Reconstruct the pre-acquired SAR image into a sparse SAR image based on the AMP algorithm;

[0007] (2) constructing an object detection network based on the improved Faster R-CNN; the object detection network includes a feature extraction network, a feature pyramid based on an attention mechanism, a ROIAlign pooling layer, and a region generation network RPN;

[0008] (3) The feature extraction network uses a wide residual network (WRN) to extract feature maps of different scales in sparse SAR ship images;

[0009] (4) Inputting the feature maps of different scales extracted in step (3) into the feature pyramid structure based on the attention mechanism to achieve feature fusion of feature maps of different scales;

[0010] (5) The region generation network is used to generate candidate target boxes, which are considered to contain the object of interest;

[0011] (6) Input the fused feature map in step (5) into the ROIAlign layer to improve the regression accuracy;

[0012] (7) The sparse SAR image in step (1) is used as the input of the improved Faster R-CNN target detection network for training to obtain the target detection result.

[0013] Furthermore, the step (1) calculates the residual value between the input image and the sparsely reconstructed image, performs soft threshold iteration to reduce the residual value between the input image and the sparsely reconstructed image, and performs sparse reconstruction on the image. The implementation process is as follows:

[0014] (11) The updated residual value at step i+1 is:

[0015] V i+1 =Y-AX i+1 +b i+1 V i-1

[0016] Among them, Y represents the input measurement value, that is, the SAR ship image, represents the measurement matrix, X i+1 Represents the reconstructed sparse estimate, with initial value X0 = 0 and residual initial value V -1 =0, i=1,2,…,I max represents the number of iteration steps, b i+1 V i ∈R M×1 represents the Onsager correction term, which can accelerate the convergence of AMP.

[0017] (12) The sparse estimate X at step i+1 i+1 It is expressed as:

[0018] X i+1 =η(Xi +A H V i ; t )

[0019] Wherein, the threshold shrinkage function [η(r;λ)] j =sgn(r j )max{|r j |-λ,0}, denoising threshold α represents the tuning parameter;

[0020] (13) The iterative error of the i+1th step is:

[0021]

[0022] When the iterative error satisfies the condition Residual<ε, the loop ends and the sparse reconstruction result X of the output scene is:

[0023] X=X i+1

[0024] Otherwise, i=i+1, and continue to execute steps (11) to (13); ε represents the reconstruction error parameter, and the sparse reconstruction result X is the reconstructed sparse SAR image.

[0025] Furthermore, the wide residual network WRN adds a dropout layer; by adding more convolution kernels, the number of feature channels is increased, the conversion of channel dimension and spatial dimension is processed separately, and rich feature information in the image is captured.

[0026] Furthermore, the implementation process of step (4) is as follows:

[0027] The feature pyramid based on the attention mechanism consists of a set of multi-scale feature maps P = {P1, P2, ..., P n}, the attention weight is obtained by calculating the relationship between the feature map channels and assigned to the feature map to obtain a new multi-scale feature map group:

[0028] P′ i =hi(up(A i+1 ⊙P i+1 ))+gi(A i ⊙P i )

[0029] P″ i =A′ i ⊙P′ i +P i

[0030] Among them, A=sigmoid(W2(ReLU(W1(Z)))) is the attention weight, W1 and W2 are two learnable weight matrices, Z is the global context vector obtained by the fully connected layer, hi and gi are conversion functions composed of convolutional layers and activation functions, and up is the upsampling function.

[0031] Furthermore, the region generation network generates a series of candidate target boxes on the input image by applying a set of anchor boxes of different sizes and aspect ratios to each location, and the candidate target boxes are considered to possibly contain the object of interest.

[0032] Furthermore, the ROI Align layer uses a bilinear difference method to calculate corresponding pixel values.

[0033] Based on the same inventive concept, a device described in the present invention includes a memory and a processor, wherein:

[0034] A memory for storing computer programs that can be run on the processor;

[0035] The processor is used to execute the steps of the sparse SAR ship target detection method based on attention feature fusion as described above when running the computer program.

[0036] Based on the same inventive concept, the present invention provides a storage medium on which a computer program is stored, and when the computer program is executed by at least one processor, the steps of the sparse SAR ship target detection method based on attention feature fusion as described above are implemented.

[0037] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: the present invention adopts the AMP algorithm to perform sparse reconstruction on the image, thereby reducing noise, improving image quality, and improving the accuracy of subsequent target detection; the present invention proposes a feature pyramid structure based on the attention mechanism, which can selectively adjust the visual processing of the network, suppress useless information, and enable the network to extract more useful features of different scales, enhance the detection capability of weak and small ship targets, and improve network performance; improve target detection accuracy and avoid missed detection and false detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the present invention;

[0039] Figure 2 Schematic diagram of the structure of the basic residual network and the wide residual network, where (a) is the basic residual network structure; (b) is the residual network structure with bottleneck; (c) is the basic wide residual network structure; (d) is the wide residual network structure with dropout layer;

[0040] Figure 3 A schematic diagram of the network structure of the feature pyramid based on the attention mechanism proposed in the present invention;

[0041] Figure 4 It is a schematic diagram of the lightweight attention mechanism ECA module used in the present invention;

[0042] Figure 5 The accuracy curves of the sparse image dataset reconstructed by the present invention under the traditional Faster R-CNN network and the target detection network proposed by the present invention;

[0043] Figure 6 The detection results of SAR ship images under the traditional Faster R-CNN network and the target detection network proposed in this invention; among them, (a) is the detection result of far coastal targets; (b) is the detection result of offshore targets; (c) is the detection result of strong noise targets. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0045] like Figure 1 As shown, the present invention provides a sparse SAR ship target detection method based on attention feature fusion, and the specific implementation process is as follows:

[0046] Step 1: Reconstruct the pre-acquired SAR image into a sparse SAR image based on the Approximate Message Passing (AMP) algorithm.

[0047] The present invention calculates the residual value between the input image and the sparsely reconstructed image, performs soft threshold iteration to reduce the residual value between the input image and the sparsely reconstructed image, performs sparse reconstruction on the image, reduces noise, and improves image quality. Taking the i+1th iteration as an example, the iterative process of the AMP algorithm is:

[0048] (1.1) The updated residual value in step i+1 is:

[0049] V i+1 =Y-AX i+1 +b i+1 V i-1

[0050] Among them, Y represents the input measurement value, that is, the input original SAR ship image, represents the measurement matrix, b i+1 V i ∈R M×1 represents the Onsager correction term, which can accelerate the convergence of AMP. X i+1Represents the reconstructed sparse estimate, with initial value X0 = 0 and residual initial value V -1 =0, i=1,2,…,I max Indicates the number of iteration steps.

[0051] (1.2) The sparse estimate X of step i+1 i+1 It is expressed as:

[0052] X i+1 =η(X i +A H V i ; t )

[0053] Wherein, the threshold shrinkage function [η(r;λ)] j =sgn(r j )max{|r j |-λ,0}, denoising threshold α represents the tuning parameter.

[0054] (1.3) The iterative error of step i+1 is:

[0055]

[0056] When the iterative error satisfies the condition Residual<ε, the loop ends and the sparse reconstruction result X of the output scene is:

[0057] X=X i+1

[0058] Otherwise, i=i+1, and continue to execute the loop process in steps (1) to (3); ε represents the reconstruction error parameter, and its value is selected as 10 in the present invention. -8 The sparse reconstruction result X is the sparse SAR image reconstructed in the present invention.

[0059] Step 2: Build an object detection network based on the improved Faster R-CNN, including a feature extraction network, a feature pyramid based on the attention mechanism, a ROI Align pooling layer, and a region proposal network (RPN).

[0060] The feature extraction network in the original FasterR-CNN target detection network is changed to a wide residual network WRN. The wide residual network improves ResNet from the perspective of increasing the network width. On the basis of the original ResNet, the number of convolution kernels is widened, and a shallower model with wider layers is used to improve model performance. Figure 2As shown, (a) is the basic residual network structure, (b) is the residual network structure with bottleneck, (c) is the basic wide residual network structure, and (d) is the wide residual network structure with dropout layer, that is, the structure of the wide residual network adopted by the present invention. The residual network uses jump connection to alleviate the gradient vanishing problem caused by increasing the depth in the deep neural network. In order to solve the problem of mismatch between the input and the output of the previous layer in the channel and spatial dimensions, 1×1 convolution is used to transform the spatial dimension and channel dimension of the feature map to achieve dimensional matching between different modules. However, this will cause information loss and introduce noise, causing negative interference to the main information flow in the network. The wide residual network increases the number of feature channels by adding more convolution kernels, realizes the separate processing of channel dimension and spatial dimension conversion, and better captures the rich feature information in the image. The increase in width improves performance, and with the same parameters, the training speed of WRN is faster than ResNet.

[0061] A feature pyramid structure based on the attention mechanism is added to the original Faster R-CNN target detection network to achieve feature fusion of feature maps of different scales. The network structure of the feature pyramid based on the attention mechanism is as follows: Figure 3 As shown in the figure, the lightweight attention mechanism ECA module is used to enable the feature pyramid structure to capture important information of feature maps of different scales and better fuse feature maps of different scales.

[0062] The feature pyramid based on the attention mechanism consists of a set of multi-scale feature maps P = {P1, P2, ..., P n}, the attention weight is obtained by calculating the relationship between the feature map channels and assigned to the feature map to obtain a new multi-scale feature map group.

[0063] P′ i =hi(up(A i+1 ⊙P i+1 ))+gi(A i ⊙P i )

[0064] P″ i =A′ i ⊙P′ i +P i

[0065] Among them, A=sigmoid(W2(ReLU(W1(Z)))) is the attention weight, W1 and W2 are two learnable weight matrices, Z is the global context vector obtained by the fully connected layer, hi and gi are conversion functions composed of convolutional layers and activation functions, and up is the upsampling function.

[0066] The feature maps of different scales obtained by the feature extraction network in the target detection network are input into the feature pyramid based on the attention mechanism. The feature channels are weighted by the attention mechanism, and then the feature maps of different scales are fused to highlight the important information of the feature maps of different scales and fuse them. The attention mechanism uses a lightweight attention mechanism ECA module, such as Figure 4 As shown in the figure. The ECA module improves SENet by learning the correlation between channels and adaptively adjusting the weights of channels to capture local cross-channel interaction information, that is, each channel participates in attention prediction together with its adjacent channels. This attention mechanism ensures both model efficiency and computational effectiveness. It calculates the weight information at the channel position when performing deep and shallow feature fusion. Based on the calculated weight distribution, the feature pyramid can pay more attention to the useful information in the feature map and improve the accuracy of target detection.

[0067] The region generation network is an important component for generating candidate target boxes. It generates a series of candidate target boxes on the input image by applying a set of anchor boxes of different sizes and aspect ratios to each location. These candidate target boxes are considered to contain the object of interest.

[0068] In addition, the ROI pooling layer in the original Faster R-CNN target detection network is changed to the ROIAlign layer. Since the ROI pooling layer uses the nearest neighbor method, which may cause image distortion, the ROIAlign layer is used, which uses the bilinear difference method to calculate the corresponding pixel values ​​to improve the regression accuracy.

[0069] Step 3: SAR ship image target detection.

[0070] The sparse SAR image obtained in step 1 is used as the input of the proposed improved target detection network for training, and the SAR image dataset to be detected is input into the network to obtain the target detection result.

[0071] The following uses actual data to verify the sparse SAR ship target detection method based on attention feature fusion proposed in the present invention by taking the public RSDD-SAR ship dataset as an example.

[0072] Table 1 shows the target detection experimental results of the sparse SAR ship image dataset reconstructed by the present invention using the improved Faster R-CNN network.

[0073] Table 1 Experimental results

[0074]

[0075]

[0076] Figure 5 The accuracy curves of the sparse image dataset reconstructed by the present invention on the Faster R-CNN target detection network and the improved Faster R-CNN target detection network proposed in the present invention. Training sets with sample numbers of 1500, 2000, 3000, 4200 and 5000 were used here for training on the Faster R-CNN network and the improved Faster R-CNN network proposed in this paper. The results show that compared with the traditional Faster R-CNN, the improved Faster R-CNN network proposed in the present invention has higher accuracy and better performance when the number of samples is small.

[0077] Figure 6 The detection results of SAR ship images under the traditional FasterR-CNN network and the target detection network proposed in this invention, where (a) is the detection result of far coastal targets; (b) is the detection result of offshore targets; (c) is the detection result of strong noise targets. Compared with the traditional FasterR-CNN network, the target detection network proposed in this invention improves the detection accuracy and reduces the missed detection and false detection of SAR target detection.

[0078] Based on the same inventive concept, the present invention also provides a device, including a memory and a processor, wherein: the memory is used to store a computer program that can be run on the processor; the processor is used to execute the steps of the sparse SAR ship target detection method based on attention feature fusion as described above when running the computer program.

[0079] Based on the same inventive concept, the present invention also provides a storage medium, on which a computer program is stored, and when the computer program is executed by at least one processor, the steps of the sparse SAR ship target detection method based on attention feature fusion as described above are implemented.

Claims

1. A sparse SAR ship target detection method based on attention feature fusion, characterized in that: The steps include: (1) Reconstruct the pre-acquired SAR image into a sparse SAR image based on the AMP algorithm; (2) constructing an object detection network based on the improved Faster R-CNN; the object detection network includes a feature extraction network, a feature pyramid based on an attention mechanism, a ROIAlign pooling layer, and a region generation network RPN; (3) The feature extraction network uses a wide residual network (WRN) to extract feature maps of different scales in sparse SAR ship images; (4) Inputting the feature maps of different scales extracted in step (3) into the feature pyramid structure based on the attention mechanism to achieve feature fusion of feature maps of different scales; (5) The region generation network is used to generate candidate target boxes, which are considered to contain the object of interest; (6) Input the fused feature map in step (5) into the ROIAlign layer to improve the regression accuracy; (7) Using the sparse SAR image in step (1) as input for training the improved Faster R-CNN target detection network to obtain the target detection result; The step (1) calculates the residual value between the input image and the sparsely reconstructed image, performs soft threshold iteration to reduce the residual value between the input image and the sparsely reconstructed image, and performs sparse reconstruction on the image. The implementation process is as follows: (11) The updated residual value at step i+1 is: V i+1 =Y-AX i+1 +b i+1 V i-1 Among them, Y represents the input measurement value, that is, the SAR ship image, represents the measurement matrix, X i+1 Represents the reconstructed sparse estimate, with initial value X0 = 0 and residual initial value V -1 =0, i=1,2,…,I max represents the number of iteration steps, b i+1 V i ∈R M×1 represents the Onsager correction term, which can accelerate the convergence of AMP. (12) The sparse estimate X at step i+1 i+1 It is expressed as: X i+1 =η(X i +A H V i ;l t ) Wherein, the threshold shrinkage function [η(r;λ)] j =sgn(r j )max{|r j |-λ,0}, denoising threshold α represents the tuning parameter; (13) The iterative error of the i+1th step is: When the iterative error satisfies the condition Residual<ε, the loop ends and the sparse reconstruction result X of the output scene is: X=X i+1 Otherwise, i=i+1, and continue to execute steps (11) to (13); ε represents the reconstruction error parameter, and the sparse reconstruction result X is the reconstructed sparse SAR image.

2. The sparse SAR ship target detection method based on attention feature fusion according to claim 1 is characterized in that: The wide residual network WRN adds a dropout layer; by adding more convolution kernels, the number of feature channels is increased, the conversion of channel dimension and spatial dimension is processed separately, and the feature information in the image is captured.

3. The sparse SAR ship target detection method based on attention feature fusion according to claim 1 is characterized in that: The implementation process of step (4) is as follows: The feature pyramid based on the attention mechanism consists of a set of multi-scale feature maps P = {P1, P2, ..., P n }, the attention weight is obtained by calculating the relationship between the feature map channels and assigned to the feature map to obtain a new multi-scale feature map group: P′ i =hi(up(A i+1 ⊙R i+1 ))+gi(A i ⊙P i ) P″ i =A′ i OP' i +P i Among them, A=sigmoid(W2(ReLU(W1(Z)))) is the attention weight, W1 and W2 are two learnable weight matrices, Z is the global context vector obtained by the fully connected layer, hi and gi are conversion functions composed of convolutional layers and activation functions, and up is the upsampling function.

4. The sparse SAR ship target detection method based on attention feature fusion according to claim 1 is characterized in that: The region generation network generates a series of candidate object boxes on the input image by applying a set of anchor boxes of different sizes and aspect ratios to each location, which are considered to be likely to contain objects of interest.

5. The sparse SAR ship target detection method based on attention feature fusion according to claim 1 is characterized in that: The ROI Align layer uses a bilinear interpolation method to calculate corresponding pixel values.

6. A device, characterized in that: comprising a memory and a processor, wherein: A memory for storing computer programs that can be run on the processor; A processor is used to execute the steps of the sparse SAR ship target detection method based on attention feature fusion as described in any one of claims 1 to 5 when running the computer program.

7. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by at least one processor, implements the steps of the sparse SAR ship target detection method based on attention feature fusion as described in any one of claims 1 to 5.