Ship target detection method in SAR images based on multi-level feature fusion and hybrid attention

Abstract

The present invention discloses a method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention, comprising: constructing a YOLO-SP network model for ship target detection in SAR images, wherein the YOLO-SP network model includes a backbone feature extraction network, a multi-level feature fusion network, and a detection head connected in sequence; training the constructed YOLO-SP network model using a training data set, calculating a loss function, and inversely updating the model parameters to obtain a trained YOLO-SP network model; performing data standardization preprocessing on the SAR image to be detected to obtain a preprocessed pixel matrix; inputting the preprocessed pixel matrix into the trained YOLO-SP network model, performing ship target detection on the SAR image to be detected, and obtaining target category information, location information, and confidence. The present invention enhances the extraction of target feature information and improves the detection accuracy of ship targets through a multi-level feature extraction strategy and a hybrid attention mechanism.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a SAR image ship target detection method based on multi-level feature fusion and mixed attention. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave remote sensing imaging radar. Unaffected by time of day and weather, it can monitor targets around the clock and in all weather conditions. It also boasts high precision and strong anti-interference capabilities. It is widely used in military and civilian applications, such as environmental protection, disaster monitoring, ocean observation, resource conservation, land cover analysis, precision agriculture, urban area monitoring, and geographic mapping. SAR, typically mounted on aircraft or satellites, can acquire high-resolution SAR imagery. Detecting ship targets in SAR images is crucial for both military and civilian applications.

[0003] SAR image ship detection methods are divided into traditional methods and deep learning-based methods. Traditional SAR image ship target detection algorithms can generally be divided into three stages: image preprocessing, candidate region extraction, and target detection and recognition. Generally, SAR images contain cross-shaped coherent speckle noise interference and sea clutter interference caused by the sea surface environment. Therefore, SAR images need to be filtered to suppress coherent speckle. Secondly, in order to reduce false alarms caused by some strong scattering targets on land and ports, sea-land segmentation methods are needed to remove land false alarms. Preprocessed SAR images can better meet the needs of subsequent task processing. Candidate region extraction is to extract areas that may be ship targets from large-scale SAR images as candidate regions. Target detection and recognition uses manually designed feature extraction classifiers to detect the obtained candidate regions, and finally outputs the detected ship results.

[0004] Deep learning-based methods primarily employ convolutional neural network models. Mainstream deep learning-based object detection algorithms fall into two categories: two-stage and single-stage. Two-stage algorithms first generate candidate bounding boxes on the SAR image and then perform classification and regression on them to determine the location and category of the object in the image. Single-stage algorithms skip the candidate regions and directly feed the SAR image into a convolutional neural network for feature extraction, completing the task of object detection.

[0005] Detecting ship targets in SAR images is a challenging task. It faces challenges such as the inherent speckle noise in SAR images, which blurs the edges of ship targets and makes it difficult to detect nearshore and small-scale ship targets. Traditional SAR image ship detection relies primarily on manual extraction of ship features from images to distinguish between ship targets and other background objects. However, this method suffers from insufficient robustness and timeliness. While deep learning-based SAR image ship detection methods offer improved detection results, they can suffer from false positives on land and missed detections of small-scale ships, resulting in low detection accuracy. Summary of the Invention

[0006] To address the above-mentioned problems in the prior art, the present invention provides a method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] The present invention provides a method for detecting ship targets in SAR images based on multi-level feature fusion and hybrid attention, comprising:

[0008] S1: Construct a YOLO-SP network model for ship target detection in SAR images. The YOLO-SP network model includes a backbone feature extraction network, a multi-level feature fusion network, and a detection head connected in sequence. The backbone feature extraction network is used to extract features from the preprocessed SAR image and output multiple feature maps of different scales. The multi-level feature fusion network is used to fuse the feature maps of different scales to obtain multiple predicted feature maps of different scales. The detection head is used to perform classification and regression operations on the predicted feature maps to obtain category information, location information, and confidence of the ship target in the SAR image.

[0009] S2: Use the training data set to train the constructed YOLO-SP network model, calculate the loss function, and reversely update the model parameters to obtain the trained YOLO-SP network model;

[0010] S3: Perform data standardization preprocessing on the SAR image to be detected to obtain a preprocessed pixel matrix;

[0011] S4: Input the preprocessed pixel matrix into the trained YOLO-SP network model to perform ship target detection on the SAR image to be detected, and obtain the target category information, location information and confidence.

[0012] In one embodiment of the present invention, the backbone feature extraction network includes a Focus module, a convolutional layer, a CSP-Tiny module, a maximum pooling layer, a CSP-Tiny module, a maximum pooling layer, a CSP-Tiny module, a maximum pooling layer and a convolutional layer connected in sequence, wherein the Focus module is used to slice the input pixel matrix to obtain multiple downsampled feature maps with complementary information and obtain a 2x downsampled feature map through splicing and convolution operations; the CSP-Tiny module is used to extract information from the input feature map; and the convolutional layers are all 3×3 convolutional layers.

[0013] In one embodiment of the present invention, the Focus module includes a slicing unit, a splicing unit and a convolutional layer, wherein the slicing unit is used to slice the pixel matrix after SAR image preprocessing, obtain a value for every other pixel in the pixel matrix, and obtain multiple data-complementary pixel matrices; the splicing unit is used to splice the multiple data-complementary pixel matrices, and the convolutional layer is used to extract image features from the spliced pixel matrix and output a feature map.

[0014] In one embodiment of the present invention, the multi-level feature fusion network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a first hybrid attention module, a first 1×1 convolutional layer, a first 3×3 convolutional layer, a first upsampling module, a second hybrid attention module, a first splicing module, a second 3×3 convolutional layer, a second upsampling module, a third hybrid attention module, a second splicing module and a third 3×3 convolutional layer, wherein,

[0015] The first feature extraction module, the second feature extraction module and the third feature extraction module are respectively used to input feature maps of different scales from the backbone feature extraction network;

[0016] The first feature extraction module, the first hybrid attention module, the first 1×1 convolutional layer, and the first 3×3 convolutional layer are connected in sequence, the first feature extraction module is used to input a feature map of a first scale, and the first 3×3 convolutional layer is used to output a predicted feature map of the first scale;

[0017] The first upsampling module, the second hybrid attention module, the first splicing module and the second 3×3 convolutional layer are sequentially connected to the output end of the first 3×3 convolutional layer, the output end of the second feature extraction module is connected to the input end of the first splicing module, the output of the second feature extraction module and the output of the second hybrid attention module are feature spliced in the first splicing module, the second feature extraction module is used to input the feature map of the second scale, and the second 3×3 convolutional layer is used to output the predicted feature map of the second scale;

[0018] The second upsampling module, the third hybrid attention module, the second splicing module and the third 3×3 convolutional layer are connected in sequence to the output end of the second 3×3 convolutional layer, the output end of the third feature extraction module is connected to the input end of the second splicing module, the output of the third feature extraction module and the output of the third hybrid attention module are feature spliced in the second splicing module, the third feature extraction module is used to input the feature map of the third scale, and the third 3×3 convolutional layer is used to output the predicted feature map of the third scale.

[0019] In one embodiment of the present invention, the first feature extraction module, the second feature extraction module and the third feature extraction module have the same structure and each includes two parts. The first part uses a 1×1 convolution layer to further integrate feature information; the second part first uses a 1×1 convolution layer to reduce the channel, and then sends it to three branches respectively: a 3×3 convolution layer, a 3×3 pooling layer and a 5×5 pooling layer, wherein the pooling kernels of the 3×3 pooling layer and the 5×5 pooling layer expand the receptive field of the network, and the 3×3 convolution layer can store semantic information. Finally, the features output by the 3×3 convolution layer, the 3×3 pooling layer and the 5×5 pooling layer and the features output by the 1×1 convolution layer of the first part are spliced and output.

[0020] In one embodiment of the present invention, the first hybrid attention module, the second hybrid attention module and the third hybrid attention module have the same structure, and are composed of a cascade of channel attention units and spatial attention units, and are used to extract the ship area of interest while suppressing the background area.

[0021] In one embodiment of the present invention, the channel attention unit includes a first global average pooling layer, a first global maximum pooling layer, a multi-layer perceptron and a first sigmoid function layer, wherein the output expression of the channel attention unit is:

[0022] Mc(F)=sigmoid(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0023] Among them, AvgPool is the global average pooling, MaxPool is the global maximum pooling, MLP represents the multi-layer perceptron; sigmoid represents the sigmoid function, and F represents the output feature of the channel attention unit.

[0024] In one embodiment of the present invention, the spatial attention module includes a second global average pooling layer, a second global maximum pooling layer, a splicing unit, a 7×7 convolution layer, and a second sigmoid function layer. The output expression of the spatial attention module is:

[0025] Ms(G)=sigmoid(conv([AvgPool(G);MaxPool(G)])

[0026] Among them, conv is a 7×7 convolution, and G represents the input feature of the spatial attention module.

[0027] In one embodiment of the present invention, the loss function includes border position loss, confidence loss and category loss, wherein,

[0028] The expression of the border position loss is:

[0029]

[0030] Among them, IOU is the intersection-union ratio of the real box and the predicted box, D1 is the distance between the center points of the real box and the predicted box, D2 is the diagonal distance of the smallest box that encloses the real box and the predicted box, and v is the parameter for aspect ratio consistency:

[0031]

[0032] Among them, w gt is the width of the real frame, h gt is the height of the real box, w is the width of the predicted box, and h is the height of the predicted box;

[0033] The expression of the confidence loss is:

[0034]

[0035] in, Indicates whether the j-th prediction box in the i-th row predicts an object, S 2 To traverse the entire prediction feature map, B is to traverse all prediction boxes, is the probability score of the target being contained in the j-th real box in the i-th row, is the probability score of the target in the j-th prediction box in the i-th row, λ noship is a 0-1 variable, Is a 0-1 variable indicating whether the j-th prediction box in the i-th row predicts an object;

[0036] The expression of the category loss is:

[0037]

[0038] in, Indicates the probability that the j-th prediction box in the i-th row is category c, Indicates the probability that the jth ground-truth box in the i-th row is category c.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The present invention provides a SAR image ship target detection method based on multi-level feature fusion and hybrid attention. Through the multi-level feature extraction strategy and hybrid attention mechanism, the extraction of ship target feature information is enhanced, the detection accuracy of ship targets is improved, and the problem of poor detection effect of near-shore ships and small-scale ships is solved.

[0041] 2. The multi-level feature fusion network of the present invention integrates information from feature layers of different scales. In the shallow feature layer, it can expand the receptive field, enrich the semantic information, and further enhance the detection capability of small-scale ship targets. In the deep feature layer, it can extract multi-scale features and integrate more feature information. The hybrid attention module can strengthen the screening of ship target feature information and reduce the interference of background targets, effectively solving the problem of poor detection of near-shore ships and small-scale ships.

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of a method for ship target detection in SAR images based on multi-level feature fusion and mixed attention provided by an embodiment of the present invention;

[0044] Figure 2 Schematic diagram of the structure of a backbone feature extraction network provided by an embodiment of the present invention;

[0045] Figure 3 1 is a schematic diagram of a processing process of a Focus module provided by an embodiment of the present invention;

[0046] Figure 4 Schematic diagram of a multi-level feature fusion network provided by an embodiment of the present invention;

[0047] Figure 5 is a structural diagram of a feature extraction module provided by an embodiment of the present invention;

[0048] Figure 6 is a structural diagram of a hybrid attention module provided by an embodiment of the present invention;

[0049] Figure 7 A SAR image containing a ship target is provided in an embodiment of the present invention;

[0050] Figure 8 yes Figure 7 The ship target detection label map of the SAR image shown;

[0051] Figure 9 yes Figure 7The image shown shows the ship detection results of the SAR image using the existing deep learning method;

[0052] Figure 10 yes Figure 7 The SAR image shown is a diagram showing the ship detection result using the method of the present invention. DETAILED DESCRIPTION

[0053] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of a SAR image ship target detection method based on multi-level feature fusion and hybrid attention proposed in accordance with the present invention, in combination with the accompanying drawings and specific implementation methods.

[0054] The aforementioned and other technical contents, features, and effects of the present invention are clearly presented in the following detailed description of the specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the intended purpose can be obtained. However, the accompanying drawings are provided for reference and illustration purposes only and are not intended to limit the technical solutions of the present invention.

[0055] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations are intended to cover non-exclusive inclusion, such that an article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the article or device comprising the element.

[0056] See Figure 1 , Figure 1 This is a flow chart of a method for detecting ship targets in SAR images based on multi-level feature fusion and mixed attention, provided by an embodiment of the present invention. The target detection method includes:

[0057] S1: Construct a YOLO-SP network model for SAR image ship target detection, wherein the YOLO-SP network model includes a backbone feature extraction network, a multi-level feature fusion network and a detection head connected in sequence, wherein the backbone feature extraction network is used to extract features from the preprocessed SAR image and output feature maps of multiple scales; the multi-level feature fusion network is used to perform feature fusion on the feature maps of different scales to obtain multiple predicted feature maps of different scales; the detection head is used to perform classification and regression operations on the predicted feature maps to obtain category information, position information and confidence of the ship target in the SAR image.

[0058] See Figure 2 , Figure 2 This is a schematic diagram of the structure of a backbone feature extraction network provided by an embodiment of the present invention. The backbone feature extraction network includes a Focus module, a convolutional layer, a CSP-Tiny module, a maximum pooling layer, a CSP-Tiny module, a maximum pooling layer, a CSP-Tiny module, a maximum pooling layer, and a convolutional layer connected in sequence. The Focus module is used to slice the input SAR image pixel matrix to obtain multiple downsampled feature maps with complementary information and obtain a 2x downsampled feature map through splicing and convolution operations; the CSP-Tiny module is used to extract information from the input feature map; and the convolutional layers in this embodiment are all 3×3 convolutional layers.

[0059] The backbone feature extraction network of this embodiment consists of a Focus module, a 3×3 convolutional layer, a CSP-Tiny module, and a maximum pooling layer, which can output 8x, 16x, and 32x downsampled feature maps, respectively, which is conducive to detecting multi-scale ship targets.

[0060] Further, see Figure 3 , Figure 3 The figure is a schematic diagram of the processing process of a Focus module provided in an embodiment of the present invention. In this embodiment, the Focus module includes a slicing unit, a stitching unit, and a convolutional layer. The slicing unit is used to slice the pixel matrix of the SAR image, obtaining a value for every other pixel in the pixel matrix to obtain multiple pixel matrices with complementary data; the stitching unit is used to stitch the multiple pixel matrices with complementary data; and the convolutional layer is used to extract image features from the stitched pixel matrix and output a feature map.

[0061] It should be noted that in this embodiment, before inputting the SAR image into the backbone feature extraction network, the original SAR image needs to be preprocessed. Specifically, in the field of image recognition, the input is a picture, and the preprocessing converts the picture into a three-dimensional pixel matrix, where the length and width represent the image size, and the depth represents the image color channel. The depth of the SAR image is 1. Because the convolutional neural network uses a gradient descent algorithm for feature extraction, it is necessary to normalize the pixel matrix, normalizing the original pixel values from 0-255 to between 0-1. This can accelerate the convergence of the convolutional neural network model and reduce the training time of the detection model.

[0062] In this embodiment, the preprocessed pixel matrix is fed into the Focus module for slicing. Specifically, a value is obtained for every other pixel in the pixel matrix, similar to neighboring subsampling. This results in four complementary pixel matrices with similar appearances but no information loss. This concentrates the width and height information in the channel space, expanding the input channel by a factor of four. Finally, the resulting new pixel matrix undergoes a convolution operation, ultimately yielding a doubly downsampled feature map with no information loss.

[0063] The newly arrived doubled downsampled feature map is input into the 3×3 convolutional layer, the CSP-Tiny module, and the maximum pooling layer, as shown in Figure 2 As shown, the CSP-Tiny module of this embodiment is divided into two independent parts. The first part does not perform any processing on the input feature map; the second part performs a Spilt operation on the input feature map, takes the second half of the input feature channel, and reduces the number of channels to half of the original; in the second part, the features are concatenated (Concat), the number of channels is restored, and 1×1 convolution is used to integrate the features; finally, the outputs of the first and second parts are concatenated again to obtain features that are doubled compared to the input channels. Finally, the backbone feature extraction network outputs 8x downsampled feature maps, 16x downsampled feature maps, and 32x downsampled feature maps. Figure 3 feature maps of different scales for subsequent processing.

[0064] Further, see Figure 4 , Figure 4It is a structural diagram of a multi-level feature fusion network provided by an embodiment of the present invention. The multi-level feature fusion network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a first hybrid attention module, a first 1×1 convolution layer, a first 3×3 convolution layer, a first upsampling module, a second hybrid attention module, a first splicing module, a second 3×3 convolution layer, a second upsampling module, a third hybrid attention module, a second splicing module and a third 3×3 convolution layer, wherein the first feature extraction module, the second feature extraction module and the third feature extraction module are respectively used to input feature maps of different scales from the backbone feature extraction network; the first feature extraction module, the first hybrid attention module, the first 1×1 convolution layer and the first 3×3 convolution layer are connected in sequence, the first feature extraction module is used to input feature maps of the first scale, and the first 3×3 convolution layer is used to output predicted feature maps of the first scale; the first upsampling module, the second hybrid attention module, the first splicing module The block and the second 3×3 convolutional layer are connected to the output end of the first 3×3 convolutional layer in sequence, the output end of the second feature extraction module is connected to the input end of the first splicing module, the output of the second feature extraction module and the output of the second hybrid attention module are feature spliced in the first splicing module, the second feature extraction module is used to input the feature map of the second scale, and the second 3×3 convolutional layer is used to output the predicted feature map of the second scale; the second upsampling module, the third hybrid attention module, the second splicing module and the third 3×3 convolutional layer are connected to the output end of the second 3×3 convolutional layer in sequence, the output end of the third feature extraction module is connected to the input end of the second splicing module, the output of the third feature extraction module and the output of the third hybrid attention module are feature spliced in the second splicing module, the third feature extraction module is used to input the feature map of the third scale, and the third 3×3 convolutional layer is used to output the predicted feature map of the third scale.

[0065] See Figure 5 , Figure 5It is a structural diagram of a feature extraction module (Mod) provided by an embodiment of the present invention. The feature extraction modules of this embodiment (the first feature extraction module, the second feature extraction module and the third feature extraction module have the same structure) are used to further extract multi-scale semantic information and send the feature maps of different scales obtained by the backbone feature extraction network to the corresponding feature extraction module. The feature extraction module will go through two parts for the input features. The first part uses a 1×1 convolution layer to further integrate the feature information, maintaining the reusability of the features; the second part first uses a 1×1 convolution reduction channel, and then sends it to three branches: 3×3 convolution layer, 3×3 pooling layer and 5×5 pooling layer. The pooling kernels of the 3×3 pooling layer and the 5×5 pooling layer can effectively expand the receptive field of the network, and the 3×3 convolution layer can also further extract deeper semantic information. Finally, the features output by the 3×3 convolution layer, the 3×3 pooling layer and the 5×5 pooling layer and the features output by the first part 1×1 convolution layer are spliced and output, which can effectively improve the detection effect of small targets. This module can extract multi-scale features and enrich semantic information while avoiding the reduction of network accuracy and speed by adding too many complex structures.

[0066] The hybrid attention module (CBAM) is used to reduce the interference of background information on ship information. It is composed of a cascade of channel attention units and spatial attention units. It extracts the ship area of interest, emphasizes it, and assigns it a larger weight, while suppressing the background area.

[0067] The channel attention unit includes the first global average pooling layer, the first global maximum pooling layer, a multi-layer perceptron, and the first sigmoid function layer. Specifically, the input feature layer F of size H×W×C is subjected to global maximum pooling and global average pooling in the spatial dimension to obtain two feature maps of size 1×1×C. These two feature maps are sent to a two-layer and parameter-shared multi-layer perceptron (MLP). The number of neurons in the first layer of the multi-layer perceptron is C / r (r is the reduction rate), the activation function is ReLU, and the number of neurons in the second layer is C. Subsequently, the two features output by the multi-layer perceptron are added element by element, and then activated by the sigmoid function to generate the channel attention feature Mc, which focuses on obtaining the category information of the ship target. The expression is as follows:

[0068] Mc(F)=sigmoid(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0069] Among them, AvgPool is the global average pooling, MaxPool is the global maximum pooling, MLP represents the multi-layer perceptron; sigmoid represents the sigmoid function;

[0070] The spatial attention module includes a second global average pooling layer, a second global maximum pooling layer, a splicing unit, a 7×7 convolution layer, and a second sigmoid function layer. The second global average pooling layer and the second global maximum pooling layer are both connected to the first sigmoid function layer of the channel attention unit. Specifically, the channel attention feature Mc output by the channel attention unit is subjected to global maximum pooling and global average pooling in the channel dimension to obtain two feature maps of size H×W×1. These two feature maps are spliced in the channel dimension, and the spliced result is subjected to a 7×7 convolution operation. It is then activated by the sigmoid function to generate the channel attention feature layer Ms, focusing on obtaining the position information of the ship target. The expression is as follows:

[0071] Ms(G)=sigmoid(conv([AvgPool(G);MaxPool(G)])

[0072] Among them, conv is a 7×7 convolution, and G represents the input feature of the spatial attention module.

[0073] In the multi-level feature fusion network of this embodiment, the 32-fold down-sampling feature layer is subjected to a 3×3 convolution layer and an upsampling operation and is integrated with the 16-fold down-sampling feature layer to form a new 16-fold down-sampling feature layer; the 16-fold down-sampling feature layer is then subjected to a 3×3 convolution layer and an upsampling operation and is integrated with the 8-fold down-sampling feature layer to form a new 8-fold down-sampling feature layer. Finally, the feature layers of these three scales are output for processing by the detection head.

[0074] The three feature maps are input into the detection head and classification and regression operations are performed on the feature maps to obtain the category information, location information and confidence of the ship target.

[0075] S2: Use the training data set to train the constructed YOLO-SP network model, calculate the loss function, and reversely update the model parameters to obtain the trained YOLO-SP network model.

[0076] It should be noted that after the structure of the YOLO-SP network model is constructed, the model needs to be trained using a training data set to increase the detection accuracy of the model. In this embodiment, the training data set used is the SSDD public data set. Similarly, before inputting into the YOLO-SP network model, the SAR images in the training data set need to be preprocessed to obtain the corresponding pixel matrix.

[0077] During the training process, three prediction boxes of different sizes are generated for each pixel in the feature layers of the three scales. The loss function is calculated for the positive samples containing ship targets, and the parameters of the model are updated in reverse so that the model can better detect ship targets.

[0078] The loss function of this embodiment includes three parts: border position loss, confidence loss and category loss, among which,

[0079] The expression of the border position loss is

[0080]

[0081] Among them, IOU is the intersection-union ratio of the real box and the predicted box, D1 is the distance between the center point of the real box and the predicted box, D2 is the diagonal distance of the smallest box that encloses the real box and the predicted box, and v is the parameter of aspect ratio consistency:

[0082]

[0083] Among them, w gt is the width of the real frame, h gt is the height of the real box, w is the width of the predicted box, and h is the height of the predicted box.

[0084] The expression of confidence loss is:

[0085]

[0086] in, It is a 0-1 variable indicating whether the j-th prediction box in the i-th row predicts an object. If yes, it is 1, otherwise it is 0. 2 To traverse the entire prediction feature map, B is to traverse all prediction boxes, is the probability score of the target being contained in the j-th real box in the i-th row, is the probability score of the target in the j-th prediction box in the i-th row, λ noship is a 0-1 variable, It is a 0-1 variable indicating whether the j-th prediction box in the i-th row predicts an object. If yes, it is 0, otherwise it is 1.

[0087] Furthermore, the expression of category loss is:

[0088]

[0089] in, Indicates the probability that the j-th prediction box in the i-th row is category c, Indicates the probability that the jth ground-truth box in the i-th row is category c.

[0090] It should be noted that during the training process, the obtained prediction information is post-processed, such as NMS (Non-Maximum Suppression) algorithm, etc., a rectangular box is marked on the original image, the visualization result is output, and it is compared with the true label to calculate the evaluation indicators Precision, Recall and AP.

[0091] Usually, many detection frames are generated during the detection process, many of which are detecting the same target. However, in the end, only one detection frame is needed for each target. NMS selects the detection frame with the highest confidence and then calculates the corresponding IOU value with the remaining frames. When the IOU value exceeds the set threshold, the frame is suppressed to prevent multiple detection frames from appearing for one target.

[0092] S3: Perform data standardization preprocessing on the SAR image to be detected to obtain a preprocessed pixel matrix.

[0093] Similarly, the SAR image to be detected is converted into a three-dimensional pixel matrix, and then the pixel matrix needs to be normalized to normalize the original pixel values ​​from 0 to 255 to between 0 and 1.

[0094] S4: Input the preprocessed pixel matrix into the trained YOLO-SP network model to perform ship target detection on the SAR image to be detected, and obtain the target category information, location information and confidence.

[0095] Specifically, during the prediction process, classification and regression operations are performed on the feature layers of the three scales to output the center point position of the ship target, the width and height of the detection box, the category and confidence of the detected target.

[0096] The following is a simulation experiment to further illustrate the effect of the SAR image ship target detection method according to the embodiment of the present invention.

[0097] (1) Simulation conditions: The graphics card used in the simulation experiment is NVIDIA GeForce RTX 2060 12G, the CPU is AMD Ryzen 5 1600, the RAM is 16G, the operating system is Ubuntu 18.04, the Python version is 3.8, Pytorch 1.4 is used, and CUDA 10.2 and Cudnn 8.0.2 are configured.

[0098] The dataset used in the simulation experiments is the publicly available SSDD dataset, collected by RadarSat-2, TerraSAR-X, and Sentinel-1 satellites. It contains 1,160 images of 2,456 ships, with an average of 2.12 ship targets per image. To fully utilize the dataset and better learn features, the dataset is divided into training, validation, and test sets in a ratio of 7:2:1.

[0099] In the simulation experiments, the weight decay regularization coefficient was 0.0001 and the momentum parameter was 0.9. The initial learning rate was 0.001, and training was performed from scratch for 500 epochs. The batch size was 16, the IoU threshold was 0.5, and the K-means clustering algorithm was used to preset the prediction boxes: [10, 12], [21, 19], [14, 29], [33, 16], [30, 30], [19, 48], [44, 23], [47, 38], [37, 64]. During training, the optimal model was saved until training was complete, and the last trained model was also saved for subsequent training.

[0100] The relevant evaluation indicators used in the simulation experiment and their definitions are as follows:

[0101] Precision: The proportion of correct positive predictions among the identified targets, defined as follows:

[0102]

[0103] Recall: It is the ratio of correctly identified objects to the total number of objects and is defined as follows:

[0104]

[0105] Average Precision (AP): The area under the PR curve formed by precision and recall, defined as follows:

[0106] AP = ∫0 1 P(R)dR

[0107] Among them, TP means that the positive sample is correctly classified, FP means that the negative sample is misclassified, FN means that the positive sample is misclassified, P means precision, and R means recall.

[0108] (2) Simulation experiment content: Use the present invention and the existing deep learning ship detection algorithm (YOLOv4-Tiny) to complete the detection of ship targets on the SSDD public dataset, and calculate the relevant evaluation indicators. Figure 7 The SAR image containing the ship target is displayed. Figure 8The label map of ship detection in SAR images is shown. Figure 9 The results of ship detection using existing deep learning methods in SAR images are shown. Figure 10 The experimental results are shown in Table 1.

[0109] Table 1. Comparison of simulation results of ship target detection in SAR images

[0110] Evaluation indicators Precision Recall AP This method 96.11％ 97.58％ 96.73％ YOLOv4-Tiny 91.26％ 89.94％ 89.26％

[0111] (3) Experimental results analysis: As can be seen from Table 1, the target detection method of the present invention can achieve an accuracy of 96.11% on the SSDD public dataset, a recall rate of 97.58%, and an average precision of 96.73%. Compared with the existing deep learning ship detection algorithm, the accuracy is improved by 4.85%, the recall rate is improved by 7.64%, and the average precision is improved by 7.47%. Figure 9 and Figure 10 It can be seen intuitively that this method has excellent detection effect on near-shore ship targets and small-scale ship targets.

[0112] The present invention is based on a SAR image ship target detection method based on multi-level feature fusion and hybrid attention. Through a multi-level feature extraction strategy and a hybrid attention mechanism, the extraction of ship target feature information is enhanced, the detection accuracy of ship targets is improved, and the problem of poor detection of nearshore ships and small-scale ships is solved. The multi-level feature fusion network of the present invention integrates information from feature layers of different scales. In the shallow feature layer, it can expand the receptive field, enrich semantic information, and further enhance the detection capability of small-scale ship targets; in the deep feature layer, it can extract multi-scale features and fuse more feature information. The hybrid attention module can strengthen the screening of ship target feature information and reduce the interference of background targets, effectively solving the problem of poor detection of nearshore ships and small-scale ships.

[0113] In the several embodiments provided herein, it should be understood that the apparatus and method disclosed herein can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the module division is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not implemented.

[0114] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or hardware plus software functional modules.

[0115] Another embodiment of the present invention provides a storage medium storing a computer program for executing the steps of the method for ship target detection in SAR images based on multi-level feature fusion and mixed attention described in the above-mentioned embodiment. Another aspect of the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor invokes the computer program in the memory, the processor executes the steps of the method for automatically generating a packaged bonding program described in the above-mentioned embodiment. Specifically, the integrated module implemented in the form of a software functional module can be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for causing an electronic device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the method described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0116] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention, characterized in that: include: S1: Construct a YOLO-SP network model for ship target detection in SAR images. The YOLO-SP network model includes a backbone feature extraction network, a multi-level feature fusion network, and a detection head connected in sequence. The backbone feature extraction network is used to extract features from the preprocessed SAR image and output multiple feature maps of different scales. The multi-level feature fusion network is used to fuse the feature maps of different scales to obtain multiple predicted feature maps of different scales. The detection head is used to perform classification and regression operations on the predicted feature maps to obtain category information, location information, and confidence of the ship target in the SAR image. S2: Use the training data set to train the constructed YOLO-SP network model, calculate the loss function, and reversely update the model parameters to obtain the trained YOLO-SP network model; S3: Perform data standardization preprocessing on the SAR image to be detected to obtain a preprocessed pixel matrix; S4: Input the preprocessed pixel matrix into the trained YOLO-SP network model to perform ship target detection on the SAR image to be detected, and obtain the target's category information, location information and confidence level; The backbone feature extraction network includes a Focus module, a convolutional layer, a CSP-Tiny module, a maximum pooling layer, a CSP-Tiny module, a maximum pooling layer, a CSP-Tiny module, a maximum pooling layer and a convolutional layer connected in sequence, wherein the Focus module is used to slice the input pixel matrix to obtain multiple downsampled feature maps with complementary information and obtain a 2x downsampled feature map through splicing and convolution operations; the CSP-Tiny module is used to extract information from the input feature map; and the convolutional layers are all 3×3 convolutional layers.

2. The method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention according to claim 1, characterized in that: The Focus module includes a slicing unit, a splicing unit and a convolutional layer, wherein the slicing unit is used to slice the pixel matrix after SAR image preprocessing, obtain a value for every other pixel in the pixel matrix, and obtain multiple pixel matrices with complementary data; the splicing unit is used to splice the multiple pixel matrices with complementary data, and the convolutional layer is used to extract image features from the spliced pixel matrix and output a feature map.

3. The method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention according to claim 1, characterized in that: The multi-level feature fusion network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a first hybrid attention module, a first 1×1 convolutional layer, a first 3×3 convolutional layer, a first upsampling module, a second hybrid attention module, a first splicing module, a second 3×3 convolutional layer, a second upsampling module, a third hybrid attention module, a second splicing module and a third 3×3 convolutional layer, wherein, The first feature extraction module, the second feature extraction module and the third feature extraction module are respectively used to input feature maps of different scales from the backbone feature extraction network; The first feature extraction module, the first hybrid attention module, the first 1×1 convolutional layer, and the first 3×3 convolutional layer are connected in sequence, the first feature extraction module is used to input a feature map of a first scale, and the first 3×3 convolutional layer is used to output a predicted feature map of the first scale; The first upsampling module, the second hybrid attention module, the first splicing module and the second 3×3 convolutional layer are sequentially connected to the output end of the first 3×3 convolutional layer, the output end of the second feature extraction module is connected to the input end of the first splicing module, the output of the second feature extraction module and the output of the second hybrid attention module are feature spliced in the first splicing module, the second feature extraction module is used to input the feature map of the second scale, and the second 3×3 convolutional layer is used to output the predicted feature map of the second scale; The second upsampling module, the third hybrid attention module, the second splicing module and the third 3×3 convolutional layer are connected in sequence to the output end of the second 3×3 convolutional layer, the output end of the third feature extraction module is connected to the input end of the second splicing module, the output of the third feature extraction module and the output of the third hybrid attention module are feature spliced in the second splicing module, the third feature extraction module is used to input the feature map of the third scale, and the third 3×3 convolutional layer is used to output the predicted feature map of the third scale.

4. The method for ship target detection in SAR images based on multi-level feature fusion and mixed attention according to claim 3 is characterized in that: The first feature extraction module, the second feature extraction module and the third feature extraction module have the same structure and each includes two parts. The first part uses a 1×1 convolutional layer to further integrate feature information. The second part first uses a 1×1 convolutional layer to reduce the channel, and then sends it to three branches respectively: a 3×3 convolutional layer, a 3×3 pooling layer, and a 5×5 pooling layer. The pooling kernels of the 3×3 pooling layer and the 5×5 pooling layer expand the receptive field of the network, and the 3×3 convolutional layer can store semantic information. Finally, the features output by the 3×3 convolutional layer, the 3×3 pooling layer, and the 5×5 pooling layer and the features output by the 1×1 convolutional layer in the first part are spliced and output.

5. The method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention according to claim 4, characterized in that: The first hybrid attention module, the second hybrid attention module and the third hybrid attention module have the same structure, and are composed of a cascade of channel attention units and spatial attention units, and are used to extract the ship area of interest while suppressing the background area.

6. The method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention according to claim 5, characterized in that: The channel attention unit includes a first global average pooling layer, a first global maximum pooling layer, a multi-layer perceptron and a first sigmoid function layer, wherein the output expression of the channel attention unit is: in, AvgPool is the global average pooling, MaxPool is the global maximum pooling, MLP represents a multilayer perceptron; sigmoid express sigmoid function, F Represents the input features of the channel attention unit.

7. The method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention according to claim 6, characterized in that: The spatial attention module includes a second global average pooling layer, a second global maximum pooling layer, a splicing unit, a 7×7 convolution layer and a second sigmoid function layer. The output expression of the spatial attention module is: in, conv is a 7×7 convolution, and G represents the input feature of the spatial attention module.

8. The method for ship target detection in SAR images based on multi-level feature fusion and hybrid attention according to any one of claims 1 to 7, characterized in that: The loss function includes border position loss, confidence loss and category loss, where: The expression of the border position loss is: in, IOU is the intersection-over-union ratio of the real box and the predicted box, D 1 represents the distance between the center point of the real box and the predicted box, D 2 represents the diagonal distance of the smallest box enclosing the real box and the predicted box, v Parameters representing aspect ratio consistency: in, is the width of the real frame, is the height of the real frame, w is the width of the prediction box, h is the height of the prediction box; The expression of the confidence loss is: in, Indicates the i Rank j Whether a prediction box predicts an object, To traverse the entire prediction feature map, B is to traverse all prediction boxes, For the i Rank j The probability score of the target in the real box, For the i Rank j The probability score of the target contained in the predicted box, is a 0-1 variable, Is a 0-1 variable, indicating the i Rank j Whether a prediction box predicts an object; The expression of the category loss is: in, Indicates the i Row No. j The probability that the predicted box is category c, Indicates the i Row No. j The probability that the ground-truth box is of category c.

Images