SAR vessel small target detection method based on feature texture enhancement

Abstract

The invention discloses an SAR ship small target detection method based on feature texture enhancement, and the method comprises the steps: firstly obtaining a ship SAR image data set, and dividing the ship SAR image data set into a training set and a test set; secondly, designing a joint representation enhancement module and a feature enhancement attention module CSAT, and respectively adding the CSAT after the last three convolution stages of the backbone network Backbone of the YOLO structure; then, in a feature fusion part in the YOLO structure, a cross-scale channel feature splicing module is changed into a self-attention feature fusion module FFM; and finally, obtaining an SAR vessel small target model after the above improvement, outputting a detection result, and training and testing by a training set and a test set through an improved loss function NWD. According to the method, the interference of other targets on the shore on ship detection is reduced, the ship identification rate in a complex sea scene is relatively high, and the problem that small targets are difficult to match in training sample matching is solved.

Description

Technical Field

[0001] The invention belongs to the field of synthetic aperture radar image processing, and in particular relates to a SAR ship detection method based on feature texture enhancement. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave remote sensing device with wide-range observation capabilities all day and all weather. Polarization information is an important attribute of electromagnetic waves, and the variable polarization characteristic of the target is also one of the important inherent attributes of the target itself, which can be used in target detection and identification. It retains the complete electromagnetic scattering characteristics of the target and has become an important tool for modern radar imaging technology. It has been widely used in military reconnaissance, topographic mapping, environmental and natural disaster monitoring, and surface ship target detection.

[0003] With the development of imaging technology, the size of SAR images is getting larger and larger, and the background information is becoming richer and richer. The traditional method represented by the constant false alarm rate has become increasingly inadequate, and at the same time, there are higher requirements for detection speed. With the development of deep learning, SAR image target detection technology based on deep learning has made great progress, but it still encounters many problems, especially in small targets.

[0004] With the development of deep learning technology, the SAR ship small target detection method based on deep learning has achieved good results, but there are still problems. For example, the loss function, the difficulty of matching positive samples, the lack of target information, and the difficulty of extraction have led to problems in the performance of the detection model. Summary of the invention

[0005] Aiming at the above problems existing in SAR ship detection and the characteristics of the marine environment, such as multi-scale, multi-orientation, complex background and much interference, the present invention proposes a SAR ship small target detection method based on ship feature texture enhancement on the basis of the single-stage YOLO detection model. The overall structure is shown in the attached figure. Figure 1 .

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions, a SAR ship small target detection method based on feature texture enhancement, comprising the following steps:

[0007] Step 1: Obtain a ship SAR image dataset: Divide the existing ship SAR image dataset into a training set and a test set for subsequent training and testing.

[0008] Step 2: Design the joint representation enhancement module and the feature enhancement attention module CSAT, and add CSAT after the last three convolution stages of the backbone network Backbone of the YOLO structure.

[0009] The backbone feature extraction network of YOLO is improved, that is, the backbone Backbone network is improved. Aiming at the strong scattering characteristics of ships, a joint representation enhancement module and a characteristic enhancement attention module CSAT (Characteristic Strengthen Attention Module) are designed. While enhancing the information extraction of small target ships, the noise information of the background is suppressed, and SiLU and Hard Swish are used as activation functions.

[0010] Step 3: In the feature fusion part of the YOLO structure, the cross-scale channel feature splicing module is replaced with a self-attention feature fusion module FFM that autonomously learns the importance of information from different channels and performs weighted splicing.

[0011] The feature fusion part of the YOLO structure has also been improved. The previous cross-scale channel feature splicing module has been improved to a self-attention feature fusion module FFM (Feature Fusion Module) that can autonomously learn the importance of different channel information and perform weighted splicing. At the same time, in the small target layer of the feature fusion part, the traditional convolution is changed to a snake convolution (Dynamic Snake Convolution) to adapt to the difficulty of detecting irregular ships.

[0012] Step 4. The YOLO structure improved by steps 3 and 4 is a SAR ship small target model. The detection result is output. The SAR ship small target model is trained by the training set through the improved loss function NWD, and the training and fitting of the model are accelerated through the improved loss function NWD.

[0013] Step 5: Input the test data set into the trained SAR ship small target detection model based on ship feature texture enhancement to obtain the detection and recognition results. Analyze the results and compare the performance of the algorithms.

[0014] Compared with the existing detection technology, the present invention has the following significant advantages: (1) When the method is used to identify ships in near-coast scenes and far-coast scenes, the ship's features can be effectively extracted through the ship's scattering features, while reducing the interference of other shore targets on the ship detection, and having a good recognition rate for ships in complex sea scenes; (2) The method retains the target's detailed texture information to a large extent through an improved feature fusion module, thereby improving the detection rate of small targets; (3) Through the NWD loss, the problem of difficulty in matching small targets in training samples is effectively improved, thereby improving the recall rate of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1: is a structural diagram of a SAR ship small target detection model in an embodiment of the present invention;

[0016] Figure 2 It is the C3 model for feature extraction in the original YOLO model;

[0017] Figure 3 It is a feature enhancement attention CSAT module in an embodiment of the present invention;

[0018] Figure 4 It is the self-attention feature fusion FFM module in the embodiment of the present invention;

[0019] Figure 5 This is the dynamic snake convolution in the embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention proposes a SAR ship small target detection method based on feature texture enhancement, and the steps are as follows: obtain a SAR image data set, and divide it into a training set, a validation set, and a test set; establish an improved lightweight target detection model, design a feature enhancement attention module CSAT for ship scattering, use SiLU and HardSwish as activation functions, enhance ship features, and suppress noise; improve the feature fusion part, improve the channel feature splicing module into an FFM module that can autonomously learn the importance of different channel information and perform weighted splicing, and at the same time, in the small target layer of the feature fusion part, change the traditional convolution to a snake convolution; train the SAR ship small target detection model designed in the above steps through an improved loss function α-NWD; input the test data set into the trained SAR ship small target detection model to obtain the detection and recognition results. The improved SAR ship small target detection model can more accurately identify small target ships in SAR images and has higher detection accuracy.

[0021] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0022] Step 1: Obtain SAR image datasets: Divide the existing HRSID and LS-SSDD datasets into training sets, validation sets, and test sets for subsequent testing and verification.

[0023] Step 2: Establish an improved lightweight target detection model, as shown in the attached Figure 1 That is, the backbone network of Resnet is improved, and a joint representation enhancement module and feature enhancement attention module CSAT are designed according to the strong scattering characteristics of ships. While enhancing the information extraction of small target ships, the noise information of the background is suppressed, and SiLU and HardSwish are used as activation functions.

[0024] In the original YOLO backbone network, there are a large number of C3 modules. The C3 module consists of three convolutions and a residual edge, as shown in the following figure. Figure 2 The Bottleneck part of the residual edge is replaced by the feature enhancement attention module CSAT designed according to the sidelobe scattering characteristics of ships on SAR images. Figure 3 This module increases the attention to the strongly scattering ship target by extracting features in the horizontal and vertical directions of the feature map, and fits more accurate feature information in the feature extraction; for other targets, especially shore targets with similar size and shape to the ship target, this operation weakens their influence on the feature map, effectively suppresses the interference of irrelevant information, and enables the model to better focus on the ship target.

[0025] At the same time, in order to improve the model's utilization of channel information, a channel shuffle operation is used in the feature enhancement attention module CSAT module, that is, the two adjacent channels of the feature map in the channel dimension are exchanged. Through such an operation, the ability to fit the complex associations between different channels can be improved, and the ship target can be better focused during the network training process, while making the model more robust. In short, by adding the CSAT module, the model's ability to detect offshore targets is significantly improved. The specific implementation process is as follows:

[0026] Step 2.1: Split the input feature map into channel dimensions:

[0027] C i =Split(C)i=1,2

[0028] Among them, C represents the input feature map, Split represents the average splitting of the feature in the channel dimension, C1 and C2 represent the results after splitting, which are used to calculate the subsequent attention in the horizontal and vertical directions.

[0029] Step 2.2: Calculate the average values ​​of C1 and C2 in the horizontal and vertical directions respectively. Finally, we get the features in two directions. Here, mean means averaging the features, δ means batch normalization, and H means using the HardSwish activation function to activate the features:

[0030] C′ i =H(δ(mean(C i )))i=1,2

[0031] Step 2.3: Concatenate, reduce the dimension, shuffle the channels of the attention layers C′1 and C′2 in the two directions, and finally decompose them again. Concat represents the channel feature concatenation operation, F1 represents the channel dimension reduction operation using a 1*1 convolution kernel, and F SIndicates the channel feature shuffle operation, and Split means the average splitting of the feature in the channel dimension:

[0032] C″ i =Split(δ(H(F S (F1(Concat(C′1,C′2))))))i=1,2

[0033] Step 2.4: Perform channel attention enhancement (CA) on the 2 feature maps C″1 and C″2 obtained in the previous step, and re-superimpose them on the feature maps C′1 and C′2, connect the channel features of the two in series, and activate them. Among them, S represents the activation feature using the SiLu activation function:

[0034] F=S(Concat(C′1*(F1(CA(C″′1))),C′2*(F1(CA(C″′2)))))

[0035] Step 2.5: Add the residual of F and the input feature map C to obtain the fused feature map P:

[0036] P=C+F

[0037] Step 3: The FPN feature fusion part of YOLO was also improved. The previous cross-scale channel feature splicing module was improved to an FFM module that can autonomously learn the importance of different channel information and perform weighted splicing. At the same time, in the small target layer of the feature fusion part, the traditional convolution was changed to a snake convolution to adapt to the irregular and small characteristics of ships.

[0038] As attached Figure 4 By adding a learnable parameter tensor W, it is used to learn and process the importance of feature maps from different scales, and re-join them according to the learned weights. Through this design, the original simple channel dimension superposition concat is transformed into a self-attention feature fusion module FFM. With the continuous training and learning of the model, the parameter tensor W will pay more attention to important detail information, and the weight will be larger, so that there will be more and more accurate focus points on different scale information during the fusion process. With a better grasp of detailed texture information, the model's attention to small target detection and the detection accuracy of small target ships by the SAR ship small target detection method are improved.

[0039] Step 3.1: Initialize a tensor with the number of channels equal to the sum of the input feature map channels, normalize it to indicate the attention paid to each layer of channels, and then split it according to the channels. Where W represents the newly initialized tensor, and W′ i Represents the newly generated tensor:

[0040] W′ i=Split(δ(W))i=1,2

[0041] Step 3.2: Multiply the input feature maps of different scales with the newly generated tensor, and finally reconstruct them as output. D1 and D2 represent the input feature maps:

[0042] C out =Concat(W′1*D1,W′2*D2)

[0043] As attached Figure 5 In the YOLO structure, the output layer with the smallest output dimension of the feature fusion part is changed to a snake convolution. When the target is narrow and long and unconnected, the dynamic snake convolution is easier to fit the target than the traditional convolution and the hole convolution. Figure 2 The traditional convolution in the residual edge of the C3 module in the image is changed to a dynamic snake convolution. The ships on the shore are often parallel to the river bank and present a slender shape. Standard convolution or hole convolution cannot adjust the focus area according to the target features, and may extract non-ship error information on the shore. Although the deformable convolution can learn the region of interest more adaptively, for small and slender targets, the deformable convolution cannot guarantee the connectivity of the focus area, which is not beneficial for ship detection. After the dynamic snake convolution, the range of the offset is constrained to ensure the continuity of the extracted area. The above problems are solved very well. After improvement, the detection recall rate of ships in the nearshore area is greatly improved.

[0044] Step 4: Train the SAR ship small target model based on ship feature texture enhancement designed in the above steps, and accelerate the training and fitting of the model through the NWD loss function.

[0045] It is characterized in that the IoU metric in the loss function is replaced by the NWD metric, by modeling the bounding box as a two-dimensional Gaussian distribution, and using the Wasserstein distance to measure the similarity of the derived Gaussian distribution.

[0046] Step 4.1: For the horizontal bounding box R = (C x ,C y ,w,h), where (C x ,C y ) is the coordinate of the center point, w and h are the width and height of the box respectively. R is modeled as a Gaussian distribution N(μ,σ), where

[0047] Step 4.2: Calculate the Wasserstein distance of the horizontal bounding boxes R1, R2:

[0048]

[0049] Step 4.3: Normalize the distance index to compare the similarity between the two regression frames. The final formula is:

[0050]

[0051] Where H represents a constant related to the dataset and is set to 2 in the experiment.

[0052] The further setting of this embodiment is to construct the training environment and training parameters of this embodiment during training. Use Ubuntu 18.04 version operating system and Pytorch 13.1 deep learning framework. The platform implemented in this example is CPU: Intel (R) Core (TM) i7-8700 CPU @ 3.20GHz, GPU: NVIDIA GeForce RTX 3090, 24g memory, data set is HRSID, and the specific experimental settings are: the number of training rounds epoch is 300, the batch size Batch-size is 16, the stochastic gradient descent SGD optimizer is used, the learning rate lr is set to 0.001, the IoU threshold for NMS is set to 0.5, and the Conf confidence threshold is set to 0.5.

[0053] Step 5: Input the test set to test and evaluate the SAR ship small target detection network model.

[0054] Step 5.1: Input the test data set into the trained SAR ship small target detection model based on ship feature texture enhancement to obtain the detection and recognition results. Analyze the results and compare the performance of the algorithms.

[0055] Step 5.2: Construct the calculation model performance index to evaluate the detection performance of the model. The index is constructed as follows:

[0056]

[0057] mAP = ∫0 1 P(R)dR

[0058] Among them, TP represents the number of actual positive predictions as positive; FP represents the number of actual negative predictions as positive; FN represents the number of actual positive predictions as negative; Precision represents the precision rate, and Recall represents the recall rate.

[0059] Step 5.3: In order to verify the effectiveness of the improved model, the single-stage target detection algorithms YOLOv5 to YOLOv7 and the two-stage target detection algorithm Faster-RCNN were selected for comparative experiments. In order to unify the indicators, this example selected the commonly used evaluation indicators Precision, Recall, and F1 for SAR target detection tasks. The specific comparison results are shown in Table 1:

[0060] Table 1 Experimental results

[0061]

[0062]

[0063] It can be found from Table 1 that the proposed SAR ship small target detection method based on feature texture enhancement has good performance in the detection of sea surface ship targets in complex environments, and also has significant advantages in indicators. SAR image nearshore ship targets are easily affected by background clutter and coastal buildings. At the same time, SAR images also have large scale changes, and a large number of ships are small and difficult to detect, resulting in poor detection of nearshore ship targets in SAR images, high false alarm rate and missed detection rate, etc. The method proposed in this chapter is specially designed and studied according to the characteristics of ship imaging and small size. Experiments have proved that it also has higher accuracy in complex backgrounds and nearshore scenes, which is better than other common detection models.

Claims

1. A SAR ship small target detection method based on feature texture enhancement, characterized in that: The following steps are involved: Step 1: Obtain a ship SAR image dataset and divide it into a training set and a test set; Step 2: Design the joint representation enhancement module and the feature enhancement attention module CSAT, and add CSAT after the last three convolution stages of the backbone network of the YOLO structure; Step 3: In the feature fusion part of the YOLO structure, the cross-scale channel feature splicing module is replaced with a self-attention feature fusion module FFM that autonomously learns the importance of different channel information and performs weighted splicing; Step 4: After the improvement of steps 3 and 4, the YOLO structure is a SAR ship small target model, and the detection result is output. The SAR ship small target model is trained by the training set through the improved loss function NWD; Step 5: Input the test data set into the trained SAR ship small target detection model to obtain the detection and recognition results.

2. The SAR ship small target detection method based on feature texture enhancement according to claim 1 is characterized in that: The specific process of the joint representation enhancement module and the feature enhancement attention module CSAT is as follows: Step 2.1: Split the input feature map into channel dimensions: C i =Split(C) i=1,2 Among them, C represents the input feature map, Split represents the average splitting of the feature in the channel dimension, and C1 and C2 represent the results after splitting; Step 2.2: Calculate the average values ​​of C1 and C2 in the horizontal and vertical directions respectively; finally, the features in two directions are obtained. The specific process is as follows: C′ i =H(δ(mean(C i ))) i=1,2 Among them, mean means averaging the features, δ means batch normalization operation, and H means using HardSwish activation function to activate the features; Step 2.3: Concatenate, reduce the dimension, shuffle the channels of the attention layers C′1 and C′2 in the two directions, and finally decompose them again, specifically: C″ i =Split(δ(H(F S (F1(Concat(C′1,C′2)))))) i=1,2 Among them, Concat represents the channel feature concatenation operation, F1 represents the channel dimension reduction operation using 1*1 convolution kernel, and F S Indicates the channel feature shuffling operation; Step 2.4: Perform channel attention enhancement CA on C″1 and C″2, and re-superimpose them on the feature maps C′1 and C′2, concatenate the channel features of the two, and activate them, specifically: F=S(Concat(C′1*(F1(CA(C″′1))),C′2*(F1(CA(C″′2))))) Among them, S means using SiLu activation function to activate the feature; Step 2.5: Add the residual of F and the input feature map C to obtain the fused feature map P: P=C+F.

3. The SAR ship small target detection method based on feature texture enhancement according to claim 2 is characterized in that: The specific implementation process of the self-attention feature fusion module FFM is as follows: Step 3.1: Initialize a tensor with the number of channels equal to the sum of the input feature map channels, normalize it, and then split it by channel: IN i ′=Split(δ(W)) i=1,2 Where W represents the newly initialized tensor, and W′ i Represents the newly generated tensor; Step 3.2: Multiply the input feature maps of different scales with the newly generated tensor and finally reconstruct them as output: C out =Concat(W′1*D1,W′2*D2) Where D1 and D2 represent input feature maps.

4. The SAR ship small target detection method based on feature texture enhancement according to claim 3 is characterized in that: The step 3 also includes changing the convolution into a snake convolution in the output layer with the smallest output dimension of the feature fusion part in the YOLO structure.

5. The SAR ship small target detection method based on feature texture enhancement according to claim 4 is characterized in that: In step 4, the loss function for training the SAR ship small target model is: replacing the IoU metric in the loss function with the bulldozer distance metric NWD, and using the similarity of the Gaussian distribution derived from the bulldozer distance metric.

Images