A sar ship small target detection method based on feature texture enhancement

Abstract

The application discloses a SAR ship small target detection method based on feature texture enhancement, which comprises the following steps: firstly, acquiring a ship SAR image dataset and dividing it into a training set and a test set; secondly, designing a joint representation enhancement module and a feature enhancement attention module CSAT, and adding the CSAT after the last three convolution stages of a backbone network of a YOLO structure; then, in the feature fusion part of the YOLO structure, replacing a cross-scale channel feature splicing module with a self-attention feature fusion module FFM; finally, obtaining a SAR ship small target model after the above improvement, outputting a detection result, and training and testing the model by using an improved loss function NWD and the training set and the test set. The application reduces the interference of other targets on the shore on ship detection, has a good recognition rate for ships in a complex sea scene, and improves the problem of difficulty in matching small targets in training samples.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of synthetic aperture radar image processing, and particularly relates to a SAR ship detection method based on feature texture enhancement. BACKGROUND

[0002] Synthetic aperture radar (SAR) is an active microwave remote sensing device with all-weather and all-day observation capability. Polarization information is an important attribute of electromagnetic waves, and the target depolarization characteristic is also one of the important attributes inherent to the target itself, which can be used in target detection and identification fields. It retains the complete target electromagnetic scattering characteristics and becomes an important tool for modern radar imaging technology, and has been widely used in military reconnaissance, terrain mapping, environmental and natural disaster monitoring, and sea surface ship target detection fields.

[0003] With the development of image technology, the size of SAR images is getting larger and larger, and the background information is also getting richer. The traditional method represented by constant false alarm rate has been more and more difficult to meet the requirements, and there is also a higher requirement for the detection speed. With the development of deep learning, the SAR image target detection technology based on deep learning has made great progress, but still encounters many problems, especially in small target aspect.

[0004] With the development of deep learning technology, the SAR ship small target detection method based on deep learning has achieved good results, but there are still problems. For example, the loss function, the difficulty in matching positive samples, the lack of target information, and the difficulty in extraction, etc. lead to problems in the performance of the detection model. SUMMARY

[0005] The present application is aimed at the above-mentioned problems existing in SAR ship detection, and the characteristics of marine environment such as multi-scale, multi-direction, complex background and multiple interference. Based on the single-stage YOLO detection model, a SAR ship small target detection method based on ship feature texture enhancement is proposed, and the overall structure is as shown in the accompanying Figure One .

[0006] The present application realizes the above-mentioned purpose through the following technical scheme, a SAR ship small target detection method based on feature texture enhancement, comprising the following steps:

[0007] Step 1, obtaining ship SAR image data set: dividing the existing ship SAR image data set into training set and test set for subsequent training and testing.

[0008] Step 2, designing a joint representation enhancement module and feature enhancement attention module CSAT, and adding CSAT after the last three convolution stages of the backbone network of YOLO structure.

[0009] The backbone feature extraction network of YOLO is improved, that is, the backbone network is improved, a combined representation enhancement module and a characteristic strengthening attention module CSAT (Characteristic Strengthen Attention Module) are designed according to the strong scattering characteristics of the ship, the information extraction of small target ships is enhanced, and the noise information of the back shadow is suppressed, and SiLU and HardSwish are used as activation functions.

[0010] Step 3, in the feature fusion part of the YOLO structure, the cross-scale channel feature splicing module is replaced by a self-attention feature fusion module FFM (Feature Fusion Module) which can learn the importance of different channel information and perform weighted splicing.

[0011] The feature fusion part of the YOLO structure is also improved, the previous cross-scale channel feature splicing module is improved into a self-attention feature fusion module FFM (Feature Fusion Module) which can learn the importance of different channel information and perform weighted splicing. At the same time, in the small target layer of the feature fusion part, the traditional convolution is replaced by a snake convolution (Dynamic Snake Convolution), which adapts to the detection difficulty of irregular ships.

[0012] Step 4, the YOLO structure improved in steps 3 and 4 is a SAR ship small target model, and the detection result is output, the SAR ship small target model is trained by the training set through the improved loss function NWD, and the training and fitting of the model are accelerated through the improved loss function NWD.

[0013] Step 5, the test data set is input into the trained SAR ship small target detection model based on ship feature texture enhancement, and the detection and recognition result is obtained. The result is analyzed, and the performance of the algorithm is compared.

[0014] Compared with the existing detection technology, the present application has the following advantages: (1) when the method is applied to identify ships in nearshore scenes and far offshore scenes, the ship scattering characteristics can effectively extract the features of the ships while reducing the interference of other targets on the shore on the ship detection, and the ship has a good recognition rate in a complex sea scene; (2) the method improves the feature fusion module, which greatly retains the detail texture information of the target and improves the small target detection rate; (3) through the NWD loss, the problem of difficult matching of small targets in the training sample is effectively improved, and the recall rate of small targets is improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1A SAR ship small target detection model structure diagram in the embodiment of the application;

[0016] Figure 2 A C3 model for feature extraction in the original YOLO model;

[0017] Figure 3 A feature enhancement attention CSAT module in the embodiment of the application;

[0018] Figure 4 A self-attention feature fusion FFM module in the embodiment of the application;

[0019] Figure 5 A dynamic snake convolution in the embodiment of the application. DETAILED DESCRIPTION

[0020] The application provides a SAR ship small target detection method based on feature texture enhancement, and the steps are as follows: acquiring a SAR image data set and performing training set, verification set and test set division; establishing an improved lightweight target detection model, designing a feature enhancement attention module CSAT for ship scattering, using SiLU and HardSwish as activation functions to enhance ship features and suppress noise; improving the feature fusion part, improving the channel feature splicing module into an FFM module that can autonomously learn the importance of different channel information and perform weighted splicing, and replacing the traditional convolution with a snake convolution in the small target layer of the feature fusion part; training the SAR ship small target detection model designed in the above steps through an improved loss function alpha-NWD; and inputting a test data set into the trained SAR ship small target detection model to obtain a detection and recognition result. The improved SAR ship small target detection model can more accurately recognize small target ships in a SAR image and has higher detection precision.

[0021] The application will be further described in detail below with reference to the drawings.

[0022] Step 1: acquiring a SAR image data set: dividing existing HRSID and LS-SSDD data sets into training sets, verification sets and test sets for subsequent testing and verification.

[0023] Step 2: establishing an improved lightweight target detection model, as shown in FIG. 2. Figure 1 That is, the backbone network of Resnet is improved, a joint representation enhancement module and a feature enhancement attention module CSAT are designed for the strong scattering characteristics of ships, the information extraction of small target ships is enhanced, and the noise information of the back shadow is suppressed, and SiLU and HardSwish are used as activation functions.

[0024] In the original YOLO backbone network, there are a large number of C3 modules. C3 module has 3 convolutional structures, and there is a residual edge, as shown in the following figure: Figure 2 The Bottleneck part of the residual edge is replaced with a feature enhancement attention module CSAT designed according to the sidelobe scattering characteristics of ships on the SAR image. Specifically, as shown in the following figure: Figure 3 This module improves the attention degree of the strong scattering ship target through feature extraction in the horizontal and vertical directions of the feature map, and fits more accurate feature information in the feature extraction. For other targets, especially the shore targets similar in size and shape to the ship target, this operation weakens their influence on the feature map, effectively suppressing irrelevant information interference, so that the model can better focus on the ship target.

[0025] Meanwhile, in order to improve the utilization of channel information by the model, a channel shuffling operation is used in the feature enhancement attention module CSAT, that is, the adjacent two channels of the feature map in the channel dimension are exchanged. Through such operation, the complex correlation between different channels can be improved, and the ship target can be better focused during network training, and the model is more robust. In summary, by adding the CSAT module, the detection ability of the model for offshore targets is significantly improved. The specific implementation process is as follows:

[0026] Step 2.1: split the input feature map in the channel dimension:

[0027] C i =Split(C)i=1,2

[0028] Where C represents the input feature map, Split represents the average splitting of the feature in the channel dimension, and C1 and C2 represent the results of the splitting, which are used to calculate the attention in the horizontal and vertical directions.

[0029] Step 2.2: calculate the average value of C1 and C2 in the horizontal and vertical directions respectively. Finally, two features in the two directions are obtained. Where mean represents the average value of the feature, δ represents the batch normalization operation, and H represents the activation of the feature using the HardSwish activation function:

[0030] C′ i =H(δ(mean(C i )))i=1,2

[0031] Step 2.3: concatenate, reduce dimension, and shuffle the channels of the two direction attention layers C'1 and C'2, and finally decompose them. Where Concat represents the channel concatenation operation, F1 represents the channel dimension reduction operation using a 1*1 convolution kernel, and F SIndicates the channel feature shuffling operation, and Split indicates the average splitting of the feature in the channel dimension:

[0032] C″ i =Split(δ(H(F S (F1(Concat(C′1,C′2))))))i=1,2

[0033] Step 2.4: Perform channel attention enhancement (CA) on the 2 feature maps C″1 and C″2 obtained in the previous step, and re-superimpose them on the feature maps C′1 and C′2. The channel features of the two are concatenated and activated. Where S represents the activation function of the feature:

[0034] F=S(Concat(C′1*(F1(CA(C″′1))),C′2*(F1(CA(C″′2)))))

[0035] Step 2.5: Add the residuals of F and the input feature map C to obtain the fused feature map P:

[0036] P=C+F

[0037] Step 3: Improvements were also made to YOLO's FPN feature fusion process. The previous cross-scale channel feature concatenation module was upgraded to an FFM module that autonomously learns the importance of information from different channels and performs weighted concatenation. Furthermore, in the small object layer of the feature fusion process, the traditional convolution algorithm was replaced with a snake-like convolution to accommodate the irregular and small characteristics of ships.

[0038] As attached Figure 4 By adding a learnable parameter tensor W to learn and process the importance of feature maps from different scales and reconcatenating them according to the learned weights, this design transforms the original simple channel-wise concatenation into a self-attention feature fusion module (FFM). As the model continuously trains and learns, the parameter tensor W pays more attention to important details and gives them greater weight, thus providing more and more accurate focus on information at different scales during the fusion process. This allows for a better grasp of detailed texture information, improving the model's focus on small target detection and the accuracy of the SAR ship small target detection method for small targets.

[0039] Step 3.1: Initialize a tensor with the number of channels equal to the sum of the input feature map channels and normalize it to represent the attention of each layer channel, and then split it according to the channel. Where W represents the newly initialized tensor, and W′ i Represents the newly generated tensor:

[0040] W′ i=Split(δ(W))i=1,2

[0041] Step 3.2: Multiply the input feature maps of different scales with the newly generated tensor and finally reconstruct them as the output. Where D1 and D2 represent the input feature maps:

[0042] C out =Concat(W′1*D1,W′2*D2)

[0043] As attached Figure 5 In the YOLO structure, the output layer with the smallest output dimension of the feature fusion part is changed to a snake convolution. In the case of narrow and unconnected targets, dynamic snake convolution is easier to fit the target than traditional convolution and dilated convolution. Figure 2 The traditional convolution in the residual edge of the C3 module in the

[15] is replaced with a dynamic snake convolution. Ships near the shore are often parallel to the riverbank and appear elongated. Standard or dilated convolutions cannot adjust the region of interest based on target features and may also extract erroneous information about non-ships on the shore. While deformable convolutions can more feature-adaptively learn regions of interest, they cannot ensure the connectivity of the region of interest for small, elongated targets, making them less effective for ship detection. Dynamic snake convolution, however, constrains the offset range to ensure a continuous region of interest. This effectively addresses these issues. This improvement significantly improves the recall rate of ship detection near the shore.

[0044] Step 4: Train the SAR ship small target model based on ship feature texture enhancement designed in the above steps, and accelerate the training and fitting of the model through the NWD loss function.

[0045] It is characterized by replacing the IoU metric in the loss function with the NWD metric, modeling the bounding box as a two-dimensional Gaussian distribution, and using the Wasserstein distance to measure the similarity of the derived Gaussian distribution.

[0046] Step 4.1: For the horizontal bounding box R = (C x ,C y ,w,h), where (C x ,C y ) is the center point coordinate, w and h are the width and height of the box respectively. Model R as a Gaussian distribution N(μ,σ), where

[0047] Step 4.2: Calculate the Wasserstein distance of the horizontal bounding boxes R1 and R2:

[0048]

[0049] Step 4.3: The distance index is normalized to compare the similarity of two regression boxes, and the final formula is:

[0050]

[0051] where H represents a constant related to the data set, which is set to 2 in the experiment.

[0052] Further settings of the embodiment, during training, the training environment and training parameters of the embodiment are constructed. Ubuntu 18.04 version operating system and Pytorch 13.1 deep learning framework are used. The platform implemented in this example is CPU: Intel(R) Core(TM) i7-8700 CPU @ 3.20 GHz, GPU: NVIDIA GeForce RTX 3090, 24g memory, and the data set is HRSID. The specific experimental settings are: the number of training rounds epoch is 300, the batch size is 16, the stochastic gradient descent SGD optimizer is used, the learning rate lr is set to 0.001, the IoU threshold for NMS is set to 0.5, and the Conf confidence threshold is set to 0.5.

[0053] Step 5, input the test set, and test and evaluate the SAR ship small target detection network model.

[0054] Step 5.1: Input the test data set into the trained SAR ship small target detection model based on ship feature texture enhancement to obtain the detection and recognition results. Analyze the results and compare the performance of the algorithm.

[0055] Step 5.2: Build a performance index for the calculation model to evaluate the detection performance of the model, and the index is constructed as follows:

[0056]

[0057] mAP = ∫0 1 P(R)dR

[0058] where TP represents the number of actual positive classes predicted as positive classes; FP represents the number of actual negative classes predicted as positive classes; FN represents the number of actual positive classes predicted as negative classes; Precision represents the precision, and Recall represents the recall.

[0059] Step 5.3: In order to verify the effectiveness of the improved model, single-stage target detection algorithms YOLOv5 to YOLOv7 and double-stage target detection algorithm Faster-RCNN are selected for comparative experiments. In order to unify the indicators, this example selects the commonly used evaluation indicators Precision, Recall, and F1 for SAR target detection tasks. The specific comparison results are shown in Table 1:

[0060] Table 1 experimental results

[0061]

[0062]

[0063] It can be found from Table 1 that the proposed SAR ship small target detection method based on feature texture enhancement has good performance in the detection of sea ship targets in complex environment, and has significant advantages in indicators. The SAR image nearshore ship target is easily affected by background clutter and shore buildings, and the SAR image also has large scale variation, a large number of small ship detection, resulting in SAR image nearshore ship target detection effect is poor, false alarm rate and missed detection rate and other problems, while the method proposed in this chapter is specially designed according to the characteristics of ship imaging and small volume, and the experiment proves that it has higher precision in complex background and nearshore scene, which is better than other common detection models.

Claims

1. A SAR ship small target detection method based on feature texture enhancement, characterized in that: The following steps are involved: Step 1: Obtain a ship SAR image dataset and divide it into a training set and a test set; Step 2: Design the joint representation enhancement module and the feature enhancement attention module CSAT, and add CSAT after the last three convolution stages of the backbone network of the YOLO structure; The specific process of the joint representation enhancement module and the feature enhancement attention module CSAT is as follows: Step 2.1: Split the input feature map into channel dimensions: C i =Split(C) i=1,2 Among them, C represents the input feature map, Split represents the average splitting of the feature in the channel dimension, and C1 and C2 represent the results after splitting; Step 2.2: Calculate the average values ​​of C1 and C2 in the horizontal and vertical directions respectively; finally, the features in the two directions are obtained. The specific process is as follows: C′ i =H(δ(mean(C i ))) i=1,2 Among them, mean means averaging the features, δ means batch normalization operation, and H means using HardSwish activation function to activate the features; Step 2.3: Concatenate the attention layers C′1 and C′2 in the two directions, reduce their dimensionality, shuffle their channels, and finally decompose them again. Specifically: C″ i =Split(δ(H(F S (F1(Concat(C′1,C′2))))))i=1,2 Among them, Concat represents the channel feature concatenation operation, F1 represents the channel dimension reduction operation using 1*1 convolution kernel, and F S Indicates the channel feature shuffling operation; Step 2.4: Perform channel attention enhancement CA on C″1 and C″2, and re-superimpose them on the feature maps C′1 and C′2. The channel features of the two are concatenated and activated. Specifically: F=S(Concat(C′1*(F1(CA(C″′1))),C′2*(F1(CA(C″′2))))) Among them, S represents the activation feature using SiLu activation function; Step 2.5: Add the residuals of F and the input feature map C to obtain the fused feature map P: P=C+F Step 3: In the feature fusion part of the YOLO structure, the cross-scale channel feature splicing module is replaced with a self-attention feature fusion module (FFM) that autonomously learns the importance of different channel information and performs weighted splicing. Step 4: After the improvement of steps 3 and 4, the YOLO structure is a SAR ship small target model, and the detection results are output. The SAR ship small target model is trained with the training set through the improved loss function NWD; The loss function for training the SAR ship small target model is: replace the IoU metric in the loss function with the bulldozer distance metric (NWD), and use the similarity of the Gaussian distribution derived from the bulldozer distance metric; Step 5: Input the test data set into the trained SAR ship small target detection model to obtain the detection and recognition results.

2. The SAR ship small target detection method based on feature texture enhancement according to claim 1 is characterized in that: The specific implementation process of the self-attention feature fusion module FFM is as follows: Step 3.1: Initialize a tensor with the number of channels equal to the sum of the input feature map channels, normalize it, and then split it by channel: IN' i =Split(δ(W))i=1,2 Where W represents the newly initialized tensor, and W′ i Represents the newly generated tensor; Step 3.2: Multiply the input feature maps of different scales with the newly generated tensor and finally reconstruct them as output: C out =Concat(W′1*D1,W′2*D2) Where D1 and D2 represent input feature maps.

3. The SAR ship small target detection method based on feature texture enhancement according to claim 2 is characterized in that: The step 3 also includes changing the convolution to a snake convolution in the output layer with the smallest output dimension of the feature fusion part in the YOLO structure.

Images