A SAR image ship small target detection method based on YOLOV4-Tiny

Abstract

The application relates to a SAR image ship small target detection method based on YOLOV4-Tiny, and belongs to the technical field of computer vision. First, 8 times down-sampling feature maps are added to enhance the extraction capability of small target features; second, a new feature extraction module Module is designed by referring to Inception and SPP networks to obtain higher-level semantic information and improve the receptive field of the shallow network; then, an SE channel attention mechanism is introduced to enhance important features and improve the robustness of the network; finally, the most suitable parameters of the SE channel attention mechanism are selected through a comparative experiment. Compared with the original YOLOV4-Tiny target detection method, the method can detect more ship small targets in the SAR image ship small target detection of the public data set SSDD, and the confidence is higher, so that the real-time detection requirement is met.

Description

TECHNICAL FIELD

[0001] The application relates to a SAR image ship small target detection method based on YOLOV4-Tiny, and belongs to the technical field of computer vision. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) is an active earth observation system, which works in the microwave band, and the absorption of atmospheric molecules to electromagnetic waves is basically in the visible light and infrared band, so SAR is not easily affected by the atmosphere and has the characteristics of all-day and all-weather. China is a big country of the sea, and the accurate detection of ships has important strategic significance in both military and civilian fields.

[0003] The SAR image public data set SSDD mainly includes two scenes of near sea and far sea. In the near sea image, the buildings on the coast cause great interference to the detection of ships, and the network is prone to misjudgment. In the far sea image, there is often a lot of noise, and the scale of the ship target is very small. The current mainstream detection algorithm has the problem of poor detection effect when detecting SAR ship images. In order to improve the detection accuracy, the mainstream detection algorithm often uses a large number of convolution operations and a large number of channels. When detecting SAR images, with the convolution, the size of the image is constantly reduced. For a target with a pixel size less than 32x32, when the feature map is reduced by 32 times, the information of the target does not exist in the network. And too deep channel number often increases the possibility of overfitting, because the features of the ship target are relatively simple, too many channel numbers will greatly increase the possibility of network misjudgment. In addition, these algorithms have high requirements for hardware resources during training, and require more training time, and it is also difficult to meet the real-time detection demand, and the parameter amount of the model is also very large. On the other hand, some simplified detection algorithms are simply obtained by reducing the channel number and using fewer convolutions of the complete target detection algorithm. Such detection algorithms do not require high hardware resources for training, and the detection speed is greatly improved compared with the complete algorithm. Although the features of SAR ship images are relatively simple and single, a too complex algorithm model is not needed when detecting SAR ship images. However, simply performing a small amount of convolution cannot effectively extract the feature information of small targets, and when processing the far sea scene in the data set, since the image contains a large number of small-scale ships and contains a large amount of noise, it is easy to cause false detection and missed detection, and the confidence is also low when detecting large targets, because the receptive field and semantic information of the simplified detection algorithm are limited, and the feature information in the network is seriously affected by noise interference. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a SAR image ship small target detection method based on YOLOV4-Tiny, so as to solve the problems of missed detection, false detection, low confidence, slow detection speed and large model parameter quantity in detecting SAR image ship small targets.

[0005] The technical solution of the present application is: a SAR image ship small target detection method based on YOLOV4-Tiny, first, 8 times down-sampling feature maps are added to enhance the extraction ability of small target features; second, a new feature extraction module Module is designed by referring to Inception and SPP network to obtain higher semantic information and improve the receptive field of the shallow network; then, the SE channel attention mechanism is introduced to enhance important features and improve the robustness of the network; finally, the most suitable parameters of the SE channel attention mechanism are selected through comparative experiments.

[0006] The specific steps are:

[0007] Step 1: 8 times down-sampling feature maps are added to enhance the extraction ability of small target features;

[0008] An SAR ship image with a size of 416x416 is input, and after the backbone network performs third down-sampling on the image, a 52x52 size feature map is output, and the related calculation formula is:

[0009]

[0010] In the formula, n represents the size of the input feature map, f represents the size of the convolution kernel for convolution operation, p represents the size of the image surrounding padding when performing convolution operation, s represents the distance of convolution kernel moving each time, Floor represents the down rounding, and O represents the size of the output feature map.

[0011] Step 2: a new feature extraction module Module is designed by referring to Inception and SPP network to obtain higher semantic information and improve the receptive field of the shallow network;

[0012] The implementation principle of Module is: for the input features, two parts will be experienced, the left part uses 1x1 convolution to further integrate feature information, which maintains the reusability of the features; the right part first uses 1x1 convolution to reduce the channel, then is sent into three branches, the 5x5 and 7x7 pooling kernels can effectively expand the receptive field of the network, and the 3x3 convolution can further extract deeper semantic information, finally the features are fused, which can effectively improve the detection effect of small targets. This module can extract multi-scale features, enrich semantic information, and avoid adding too many complex structures to cause the decline of network precision and speed.

[0013] Step3: Introduce SE channel attention mechanism to enhance important features and improve network robustness;

[0014] For SAR ship images with more noise, the attention mechanism can make the network pay more attention to ship information. For the input feature map, first is the compression operation F sq , which is used to perform GAP operation on the input feature h x w x c to obtain a 1 x 1 x c vector. The relevant calculation formula is:

[0015]

[0016] In the formula, h and w represent the height and width of the input feature respectively, i and j represent the coordinates of a certain channel pixel point in the input feature, and u c represents the value of the pixel point.

[0017] Second is the excitation operation F ex , which considers the information between all channels and enhances the network expression ability through two fully connected and one nonlinear activation operations, and then outputs a 1 x 1 x c weight vector through a Sigmoid operation. The relevant calculation formula is:

[0018] F ex = sigma (W2delta (W1F sq )) (3)

[0019] In the formula, F sq represents the compressed vector, W1 and W2 represent the weight parameters of the two fully connected operations respectively, delta represents the activation function, and sigma represents the Sigmoid operation.

[0020] Finally, the F scale operation is performed, which multiplies the input feature with a 1 x 1 x c weight vector to determine the importance of the input feature channel.

[0021] Step4: Select the most suitable parameters of SE channel attention mechanism through comparative experiments;

[0022] In the SE channel attention mechanism, different compression ratios during the two fully connected operations will result in different model sizes and detection effects. Therefore, three sets of comparative experiments are set up in this step to select the most suitable parameters for SAR image ship detection.

[0023] The beneficial effects of the present application are: compared with the original YOLOV4-Tiny target detection method, the present application can detect more small ship targets in the SAR image ship small target detection in the public data set SSDD, with higher confidence and lower miss detection rate, meeting the real-time detection requirements. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a network structure diagram of the application;

[0025] Figure 2 is a network structure diagram of the Module in the embodiment of the application;

[0026] Figure 3 is a network structure diagram of the SE channel attention mechanism in the embodiment of the application;

[0027] Figure 4 is a detailed structure diagram of the SE channel attention mechanism in the embodiment of the application;

[0028] Figure 5 is a comparison diagram of the detection effect of the original version and the embodiment of the application. DETAILED DESCRIPTION

[0029] The application will be further described below in conjunction with the drawings and specific embodiments.

[0030] A SAR image ship small target detection method based on YOLOV4-Tiny, first, an 8 times down-sampling feature map is added to enhance the extraction ability of small target features, improve the detailed information of small targets in the network, and lay a good foundation for subsequent feature fusion; secondly, a new feature extraction module Module is designed by referring to Inception and SPP network to obtain higher semantic information and improve the receptive field of the shallow network, which mainly uses different scales and large convolution kernels to process the input features, can obtain a larger receptive field through large-scale convolution kernels in the shallow layer to improve the accuracy of detecting small targets, and can obtain more rich feature information in the high layer; in addition, the SE channel attention mechanism introduced can well enhance important features, suppress noise information, enrich the expressiveness of the model, and improve the robustness of the network as a whole by considering the feature information of the whole channel through twice full connection and once nonlinear activation; finally, the most suitable parameters of the SE channel attention mechanism are selected through comparison experiments. Compared with the original YOLOV4-Tiny target detection method, the method of the application can detect more ship small targets when detecting ship small targets in the public data set SSDD, has higher confidence, has lower missed detection rate, and meets the real-time detection requirement.

[0031] The specific steps are as follows:

[0032] Step 1: an 8 times down-sampling feature map is added to enhance the extraction ability of small target features, which can effectively reduce the loss of detailed information caused by the convolution operation of the network on the ship small target, reduce the occurrence of missed detection, and improve the positioning ability of the network on the ship target;

[0033] An SAR ship image with a size of 416x416 is input, and a feature map with a size of 52x52 is output after the backbone network is thirdly down-sampled, and the relevant calculation formula is:

[0034]

[0035] In the formula, n represents the size of the input feature map, f represents the size of the convolution kernel for the convolution operation, p represents the size of the image surrounding padding when the convolution operation is performed, s represents the distance of the convolution kernel moving each time, Floor represents the down rounding, and O represents the size of the output feature map.

[0036] The 8 times down-sampling feature map has an important advantage in the whole target detection process. With the convolution operation in the neural network, the feature map gradually decreases, and the semantic information becomes richer and the detail information becomes less. Since the SAR image ship contains many small targets, with the convolution, the information of small targets in the network gradually decreases or even does not contain the information of small targets. This will make the network have a high probability of missing detection and false detection when making a prediction. Therefore, with the addition of the 8 times down-sampling feature map, the detail information of small targets is well increased, and combined with the semantic information from the high layer, the network can be more conducive to detecting small ship targets in the image.

[0037] Step 2: A new feature extraction module Module is designed by Inception and SPP network to obtain higher semantic information and improve the receptive field of the shallow network.

[0038] The implementation principle of Module is that for the input features, two parts will be experienced, the left part uses 1x1 convolution to further integrate feature information, which maintains the reusability of the features; the right part first uses 1x1 convolution to reduce the channel, then is sent into three branches, the 5x5 and 7x7 pooling kernels can effectively expand the receptive field of the network, and the 3x3 convolution can further extract deeper semantic information, finally the features are fused, which can effectively improve the detection effect of small targets. The module can extract multi-scale features, enrich semantic information, and avoid adding too many complex structures to cause the decline of network precision and speed.

[0039] After adding the 8 times down-sampling feature map, the detail information of small target is supplemented. In order to further improve the detection performance of small target, the newly designed Module module is added. First, the left branch can further integrate feature information and maintain the reusability of features. Second, the large-scale convolution kernel on the right can bring a larger receptive field, which makes up for the defect of small receptive field of low-level network and is beneficial to enrich the small target information of the whole network. Then, the outputs of the four parts are spliced together in the channel dimension, which can jointly improve the detection effect of small targets. Finally, the Module can also play an important role at the high level. Different scale convolution kernels can extract different features to enrich the feature information of the whole network, which will also improve the detection of large targets. Unlike simple stacking convolution operations, the Module designed in the patent avoids the decline of network precision and speed caused by adding too many complex structures.

[0040] Step3: Introducing SE channel attention mechanism to enhance important features and improve the robustness of the network;

[0041] For SAR ship images with more noise, the attention mechanism can make the network pay more attention to ship information. For the input feature map, first is the compression operation F sq , which is used to perform GAP operation on the input feature h x w x c to get a 1 x 1 x c vector. The related calculation formula is:

[0042]

[0043] In the formula, h and w represent the height and width of the input feature respectively, i and j represent the coordinates of a certain channel pixel point in the input feature, and u c represents the value of the pixel point.

[0044] Second is the excitation operation F ex , which considers the information between all channels and enhances the network expression ability through two fully connected and one nonlinear activation operations, and then outputs a 1 x 1 x c weight vector through a Sigmoid operation. The related calculation formula is:

[0045] F ex = σ(W2δ(W1F sq )) (3)

[0046] In the formula, F sq represents the compressed vector, W1 and W2 represent the weight parameters of the two fully connected operations respectively, δ represents the activation function, and σ represents the Sigmoid operation.

[0047] Finally, F scaleOperation, input the feature of h x w x c with 1 x 1 x c weight vector, and the input feature channel importance is calibrated.

[0048] Because the SAR ship image generally contains more noise, these interference pixels will affect the judgment of the network on the ship target, therefore, the SE channel attention mechanism can well suppress the interference information, and through twice full connection, the features of all channels can be considered, and the final detection accuracy is better than other channel attention mechanisms.

[0049] Step 4: Select the most suitable parameters of SE channel attention mechanism through comparative experiment;

[0050] In the SE channel attention mechanism, different compression ratios will lead to different model sizes and different detection effects when performing twice full connection operation, therefore, in this step, three groups of comparative experiments are set to select the most suitable parameters for SAR image ship detection.

[0051] The data set used is the public data set SSDD, which contains 1160 images, a total of 2456 ships, and an average of 2.12 ship targets per image.

[0052] Specifically:

[0053] For example Figure 1As shown, the approximate steps are input image, two consecutive convolution operations, due to the step size of 2, the size of the feature map will become 104x104 before entering the CSP-Tiny structure, CSP-Tiny is a structure with excellent feature extraction capability and is very concise, for the input CSP-Tiny feature, first take the back half of the feature, then connect two convolution operations with a step size of 1, concatenate the results of the two convolution operations in the channel dimension, restore the number of channels, and then integrate the features through a 1x1 convolution, and finally concatenate the integrated results with the input feature again in the channel feature to obtain a feature with double the input channel. After that, the first max pooling downsampling is connected, the feature map size becomes 52x52, then a CSP-Tiny structure is connected again, the second max pooling downsampling is connected, the feature map size becomes 26x26, the third max pooling downsampling is connected, the feature map size becomes 13x13, and the feature extraction step of the backbone network ends. Then the Neck layer fuses the extracted features, as shown in the figure, the entire backbone network outputs three different sizes of feature maps, the 13x13 feature map of the highest layer is first enriched by Module, then the SE channel attention mechanism is used to enhance important features and suppress irrelevant information, and then after two convolutions, one direction is the Head1 layer, which is used to output the prediction result; the other direction flows to the lower layer to pass on rich semantic information. When flowing to the lower layer, there will be upsampling and adding SE channel attention mechanism, so the size of the filtered feature map becomes 26x26, which is concatenated with the feature output by the third CSP-Tiny. The feature output by the third CSP-Tiny is also first enriched by Module to enrich feature information and expand the receptive field, and after concatenation, one direction is the Head2 layer, which is used to output the prediction result, and the other direction is to continue to the lower layer, and finally fuse with the 52x52 size feature map to flow to the Head3 layer to output the prediction result.

[0054] Figure 2 The Module structure in the network is shown, for the input feature, it will go through two parts, the left part uses 1x1 convolution to further integrate feature information, maintaining the reusability of the feature; the right part first uses 1x1 convolution to reduce the channel, then sends it to three branches respectively, the 5x5 and 7x7 pooling kernels can effectively expand the receptive field of the network, and the 3x3 convolution can further extract deeper semantic information, finally the features are fused, slightly increasing the width of the network to enrich the expressiveness of the network, which can effectively improve the detection effect of small targets. This module can extract multi-scale features and enrich semantic information while avoiding the addition of too many complex structures to reduce network precision and speed.

[0055] Figure 3 and Figure 4 The SE channel attention mechanism is shown, which first is a compression operation F sq , which is used to compress the input feature F sq to a 1x1xc vector; second is an excitation operation F ex , which considers all the information between the channels and enhances the network expression ability through twice full connection and once nonlinear activation operation, and then outputs a 1x1xc weight vector through once Sigmoid operation; and finally is a F scale operation, which multiplies the input feature with a 1x1xc weight vector to calibrate the importance of the input feature channel. Under the action of the loss function, the network will pay attention to important feature information, and the color depth represents the degree of network attention to the feature. Figure 4 The ratio in the formula can set the multiple of channel reduction, which can be used to control the complexity of the model. Through three sets of comparative experiments, the most suitable detection algorithm for the version of the patent is selected.

[0056] Figure 5 The figure shows the comparison between the patent and the unimproved version, column a is the real label, column b is the unimproved version, and column c is the version of the patent. The first row is in the offshore range, and the size of the ship is large, but due to the proximity to the port, the surrounding buildings and densely parked ships will bring great interference to the detection. It can be seen that the unimproved detection algorithm easily misses the ship target and without exception considers multiple ships as the same one. This is because the unimproved version has insufficient feature extraction capability and a small receptive field, which is easy to mistake the surrounding environment as part of the ship and cause misjudgment. The second and third rows are open sea images. It can be seen that the image contains a large number of small-scale ships, and some noise interference in the second row. This will bring greater interference to the detection of small-scale ships than the noise in the image of large-scale ships. In the open sea image, the unimproved version misses 4 ships, while the version of the patent does not miss any. This is because the unimproved version does not protect the feature information of the low-level small target, and with the progress of convolution, the network is likely to lose the information of the small target ship. The version of the patent not only increases the 8 times down-sampling feature map, but also adds the Module module to expand the receptive field of the low layer, which proves that the improvement point of the patent is feasible. The fourth row is an offshore image, and the ship size is smaller than the first row. There are still densely parked ships, and under the condition of containing more noise interference, the version of the patent has higher confidence and detects more densely parked ships. Therefore, in general, from the aspects of detection accuracy, missing rate, confidence, and the fit of the detection frame and the real frame, the patent is superior to the unimproved version.

[0057] The specific embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present application.

Claims

1. A SAR image ship small target detection method based on YOLOV4-Tiny, characterized in that: Step1: the backbone network increases 8 times down-sampling feature maps to enhance the extraction ability of small target features; Step2: a new feature extraction module Module is designed by referring to Inception and SPP network to obtain higher-level semantic information and improve the receptive field of the shallow network, after the backbone network outputs each feature map, the Module is used to enrich the feature information and expand the receptive field; Step3: the SE channel attention mechanism is introduced to enhance important features and improve the robustness of the network; After the highest layer feature map is processed by the Module, the SE channel attention mechanism is used to enhance important features and suppress irrelevant information, for the features flowing to the low layer direction, after being up-sampled and adding the SE channel attention mechanism, they are spliced with the feature maps output by the bottom layer Module; Step4: the most suitable parameters of the SE channel attention mechanism are selected through comparative experiments; The Step 2 is specifically: for the input features, divided into left and right two parts, the left part uses Convolution further integrates feature information, the right part first uses Convolution reduces channels, then respectively sent into three branches, And The pooling kernel can effectively expand the receptive field of the network, and The convolution can further extract deeper semantic information, and finally fuse these features.

2. The YOLOV4-Tiny-based SAR image ship small target detection method according to claim 1, characterized in that, The Step1 is specifically: Input a SAR ship image with the size of After the backbone network performs the third down-sampling on the image, a feature map with the size of is output, and the relevant calculation formula is as follows: （1）； In the formula, denotes the size of the input feature map, denotes the size of the convolution kernel for performing the convolution operation, denotes the size of the padding around the image when performing the convolution operation, denotes the distance of the convolution kernel moving each time, denotes the floor function, denotes the size of the output feature map.

3. The YOLOV4-Tiny-based SAR image ship small target detection method according to claim 1, characterized in that, The Step3 is specifically: Step3.1: For the input feature map, first compression operation , to the input feature perform GAP operation to get vector, the relevant formula is: （2）； wherein, , respectively represent the height and width of the input feature, , represents the coordinate of a pixel point in a certain channel of the input feature, represents the value of the pixel point; Step 3.2: Excitation operation , all the information between channels is considered and the network expression ability is enhanced through twice full connection and once nonlinear activation operation, and then output through once sigmoid operation weight vector, the relevant calculation formula is: （3）； In the formula, denotes the vector after compression, , denote the weight parameters of the two full connection operations respectively, denotes the activation function, denotes the Sigmoid operation; Step3.3: Operation, input as The characteristics are multiplied by The weight vector, and the input feature channel importance is calibrated.

4. The YOLOV4-Tiny-based SAR image ship small target detection method according to claim 1, characterized in that, The Step4 is specifically: three groups of comparative experiments are set to select the most suitable parameters for SAR image ship small target detection.

Images