SAR image ship target detection method based on multi-attention fusion

Abstract

The invention belongs to the technical field of image processing, and particularly discloses an SAR image ship target detection method based on multi-attention fusion. The method and the device are used for solving the problem of how to accurately detect a ship target in an SAR image in a complex environment. Comprising the following steps: (1) constructing a data set; (2) constructing an SAR image ship target detection network model based on multi-attention fusion, comprising a network main branch, a multi-level feature enhancement module composed of a global feature enhancement branch and a local feature enhancement branch, a multi-attention fusion module composed of channel attention and space attention, and a mixed loss function composed of shape segmentation loss and boundary weighted cross entropy loss. And (3) training the SAR image ship target detection network model, and carrying out ship target detection on the SAR image. According to the invention, the optimal balance between accuracy and efficiency can be achieved, and ship targets of different scales in a complex environment can be effectively detected.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a SAR image ship target detection method based on multi-attention fusion. Background Art

[0002] Synthetic Aperture Radar (SAR) sensors have the ability of all-weather and long-range detection and are widely used in the perception of unmanned ships and port traffic management. Therefore, multi-scale ship target detection based on SAR sensor imaging has become a hot research topic. In general, traditional ship detection methods mainly focus on modeling background clutter and then adjust the detection threshold based on experience to distinguish between background and ship targets to obtain a constant false alarm rate. However, such methods rely on manually designed features and classifiers, and have poor detection performance under complex background challenges.

[0003] In recent years, thanks to the rapid development and performance improvement of artificial intelligence, ship detection based on deep learning has gradually emerged and become mainstream. SAR ship perception algorithms based on deep learning are mainly divided into SAR detection and classification, SAR instance segmentation and SAR semantic segmentation. SAR detection and classification algorithms usually use anchor frames to detect ships in SAR images. Their advantages are fast detection speed and low computational complexity, which are suitable for scenarios with high real-time requirements. However, SAR detection and classification algorithms can only provide the location point information of the target and lack pixel-level description of the target. In order to achieve more precise detection, some researchers have adopted SAR instance segmentation algorithms. SAR instance segmentation algorithms can segment each target instance at the pixel level according to the detected target position, but their computational complexity is high and the inference speed is slow. Unlike SAR instance segmentation algorithms, SAR semantic segmentation algorithms classify each pixel in SAR images and provide global pixel-level classification results. The advantage of SAR semantic segmentation algorithms is that they can classify all target pixels of the same category under the same label, thereby reducing the complexity of the model while providing pixel-level information.

[0004] However, there are still some problems to be solved in the complex environment of the ocean background. First, the above algorithms are mostly aimed at large ships. However, there are usually many types of ships in SAR images, such as merchant ships, fishing boats, yachts, etc., and their scales are also different, which puts higher requirements on the versatility and versatility of the published algorithms; secondly, some ships are small in size and have less feature information, making them difficult to detect. In addition, moored ship targets are easily confused and merged with the complex near-port background, further increasing the difficulty of the detection task. Summary of the invention

[0005] The purpose of the present invention is to provide a SAR image ship target detection method based on multi-attention fusion, which effectively solves the problem of how to accurately detect ship targets in synthetic aperture radar images in a complex marine environment.

[0006] To solve the above technical problems, the technical solution adopted by the present invention is: a SAR image ship target detection method based on multi-attention fusion, comprising the following steps: S1, constructing a SAR image ship target dataset, and dividing the SAR image ship target dataset into a training set, a test set and a validation set according to a preset ratio.

[0007] S2. Construct a SAR image ship target detection network model based on multi-attention fusion, including a network main branch, a multi-level feature enhancement module, a multi-attention fusion module and a hybrid loss function. The network main branch includes an encoder and a decoder. The encoder is composed of a convolutional neural network and a hierarchical transformer, and the decoder is a multi-layer perceptron. The multi-level feature enhancement module includes a global feature enhancement branch and a local feature enhancement branch. The global feature enhancement branch is composed of multiple convolutions and global maximum pooling operations, and the local feature enhancement branch is composed of overlapping patch embedding, layer normalization, and multi-head attention. The multi-attention fusion module is composed of channel attention and spatial attention. The hybrid loss function is composed of shape segmentation loss and boundary weighted cross entropy loss.

[0008] S3. Train the SAR image ship target detection network model, iterate and optimize the network parameters through the stochastic gradient descent method, use the training set and the validation set to train the network model, and set the pre-training hyperparameters and loss function weights. The pre-training hyperparameters include the maximum number of iterations, the initial learning rate, the weight decay rate and the batch size.

[0009] S4. Use the trained SAR image ship target detection network model to perform ship target detection on the SAR images in the test set to obtain the ship target detection results.

[0010] Further, in step S2, the convolutional neural network of the encoder includes a convolution block consisting of two convolutions with a convolution kernel size of 3×3, a stride of 2×2 and no padding, and a convolution block consisting of a convolution kernel size of 3×3, a stride of 1×1 and padding, and the convolutional neural network is used to extract preliminary features.

[0011] The encoder's hierarchical transformer consists of four hierarchical transformer blocks. In each hierarchical transformer block, the feature map processed by the previous layer is first downsampled using a convolution kernel size of 7×7 and a stride of 4×4. Then, a 4×4 convolution operation is used to generate overlapping patch embeddings. After layer normalization, the feature map is split into multiple heads, and each head performs self-attention calculation separately; then the features are further processed through a feedforward neural network, which includes two fully connected layers and a nonlinear activation function ReLU.

[0012] Furthermore, in step S2, the global feature enhancement branch of the multi-level feature enhancement module is used to extract the location information of the ship target and the semantic information related to the classification. First, two 1×1 convolution operations are applied to the input feature map, and then the global maximum pooling operation is used to obtain the weight of each channel.

[0013] The local feature enhancement branch of the multi-level feature enhancement module is used to enhance the structural information of the extracted ship target. The local feature enhancement branch divides the feature map into blocks of uniform size and uses a transformer to enhance the features of each block. The transformer divides the input feature map into overlapping patches for embedding, and then performs layer normalization and inputs it into the multi-head attention for enhancement. Each attention head performs self-attention calculation separately, and then the divided feature maps are reorganized.

[0014] Finally, the weights obtained from the global feature enhancement branch are multiplied by the features obtained from the local feature enhancement branch and then added to the input feature map.

[0015] Furthermore, in step S2, the channel attention of the multi-attention fusion module first performs a global average pooling operation on the input feature map, and then uses a one-dimensional convolution with a convolution kernel size of K to generate channel attention weights. After that, the ReLU activation function is applied to adjust the channel attention weights, and the adjusted channel attention weights are multiplied with the original feature map to generate a feature map after channel attention weighting.

[0016] The output of the channel attention is input into the spatial attention. The spatial attention uses two 1×1 convolution operations to convert the input feature map into query features and numerical features. The query features are then processed with global average pooling, one-dimensional convolution, reshape, and activation functions to generate spatial attention weights. The numerical features are convolved and reshaped, and then multiplied with the spatial attention weights to obtain a weighted feature map. The weighted feature map is then reshaped, activated, and multiplied with the input feature map to obtain a feature map enhanced by attention.

[0017] Furthermore, in step S2, the total loss of the SAR image ship target detection network model is the sum of the shape segmentation loss and the boundary weighted cross entropy loss.

[0018] Shape segmentation loss DiceLoss: Among them, y i represents the i-th pixel of the true label, represents the i-th pixel predicted by the model, N is the total number of sample pixels, and ∈ is a small regularization term to avoid division by zero error.

[0019]

[0020] Among them, y is the true label, is the model prediction, CE represents the cross entropy loss function, mean represents the mean function, and Loss edge represents the boundary weighted cross entropy loss, and bw is the boundary attention weight.

[0021] Furthermore, the calculation formula of the convolution kernel size K is: In the formula, express The nearest odd number, C represents the number of channels of the input feature map, and γ and b are coefficients.

[0022] Furthermore, the input image of the convolutional neural network of the encoder is a three-channel RGB image. The encoder extracts the input image layer by layer, and finally inputs the features output by the hierarchical transformer into the multi-level feature enhancement module.

[0023] The output of the multi-level feature enhancement module is input into the multi-layer perceptron of the decoder for linear transformation. The multi-layer perceptron of the decoder includes two fully connected layers for mapping features of different scales to the same feature space dimension.

[0024] The feature map processed by the multi-layer perceptron of the decoder is input into the multi-attention fusion module, and the multi-attention fusion module outputs the feature map after attention enhancement for subsequent ship target detection.

[0025] Compared with the prior art, the beneficial technical effects of the present invention are: (1) The present invention introduces a multi-level feature enhancement module and a multi-attention fusion module to design a stable and efficient neural network driven by multi-level complementary enhancement and multi-level fusion, which is conducive to effectively detecting ship targets of different scales in complex environmental backgrounds, achieving an optimal balance between accuracy and efficiency, and thus achieving more competitive ship detection results.

[0026] (2) The present invention fully utilizes the unity of convolution and multi-head self-attention mechanism by designing global feature enhancement branches and local feature enhancement branches, extracts features from global and local feature maps respectively, and performs adaptive fusion to enhance the feature mining and representation capabilities of the model, which is beneficial to prevent the loss of small target features during the convolution process.

[0027] (3) The present invention designs a multi-attention fusion module, which organically integrates channel attention and spatial attention. While adaptively focusing on value targets, it effectively suppresses irrelevant interference information, which is beneficial to improving the network's ability to suppress complex background interference factors.

[0028] (4) The present invention further improves the model's attention to target intersection information and improves the accuracy of inter-class boundaries by designing a more balanced and comprehensive hybrid loss function. The present invention is conducive to solving the problem of small ship feature loss and inability to accurately detect in complex sea areas, improving the accuracy of ship perception, and effectively guiding the perception of unmanned ships and port traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is the overall architecture diagram of the SAR image ship target detection network based on multi-attention fusion of the present invention. DETAILED DESCRIPTION

[0030] Example 1: A SAR image ship target detection method based on multi-attention fusion, comprising the following steps: S1, constructing a SAR image ship target dataset. In this embodiment, two public datasets SSDD and HRSID are used for training and verification respectively, and each dataset is divided into a training set, a test set and a verification set in a ratio of 8:1:1.

[0031] S2. Construct a network model for ship target detection in SAR images based on multi-attention fusion.

[0032] The process includes the following sub-steps: S21, image preprocessing: resetting the input image to a size of 512×512.

[0033] S22. Design the main branch of the network: The main branch of the network includes an encoder and a decoder. The decoder uses a multi-layer perceptron. Since there are ships of different sizes in the synthetic aperture radar and the background of the port environment is relatively complex, there is a lot of interference information, which will cause the characteristics of small ships to be obscured. Most existing methods are suitable for large ships, and it is easy to cause the characteristics of small ships to be lost during the extraction process, which puts higher requirements on the encoder of the network.

[0034] In this embodiment, if Figure 1As shown, the encoder is composed of a convolutional neural network and a hierarchical transformer (hierarchical Transformer). The convolutional neural network includes two convolution blocks with a convolution kernel size of 3×3, a stride of 2×2 and no padding, and a convolution block with a convolution kernel size of 3×3, a stride of 1×1 and moderate padding. The convolutional neural network is used to extract preliminary features.

[0035] The hierarchical transformer consists of four hierarchical transformer blocks. In each hierarchical transformer block, the feature map processed by the previous layer is first downsampled using a convolution kernel size of 7×7 and a stride of 4×4, and then a 4×4 convolution operation is used to generate overlapping patch embeddings. Then, after layer normalization, the feature map is split into multiple heads, and self-attention calculations are performed on each head. The features are then further processed through a feedforward neural network, which includes two fully connected layers and a nonlinear activation function ReLU.

[0036] The input image of the encoder's convolutional neural network is a three-channel RGB image. The encoder extracts the input image layer by layer, and finally inputs the features output by the hierarchical transformer into the multi-level feature enhancement module.

[0037] S23. Design of a multi-level feature enhancement module: In the synthetic aperture radar image recognition task, it is inevitable that some ships will occupy a small proportion of pixels, which is easy to lose features and lead to missed detection. The key to solving this problem lies in how to mine local detail features and global context information. In addition, generally speaking, feature information is divided into global features and local features. Global features contain high-level semantic information and are more suitable for positioning and classification tasks, but pay less attention to the local features of the target; on the contrary, although local features contain subtle structural feature information of the target, they lack the global context information required for classification tasks. Therefore, in order to combine the advantages of both, the applicant designs a multi-level feature enhancement module. The multi-level feature enhancement module includes a global feature enhancement branch and a local feature enhancement branch. The global feature enhancement branch consists of multiple convolutions and global maximum pooling operations. The global feature enhancement branch is used to extract the location information of the ship target and the semantic information related to the classification. The local feature enhancement branch consists of overlapping patch embedding, layer normalization, and multi-head attention. The purpose of the local feature enhancement branch is to enhance the structural information of the extracted ship target.

[0038] Specifically, the global feature enhancement branch first applies two 1×1 convolution operations to the input feature map to enhance the global feature extraction, and then uses the global maximum pooling operation to condense the features to obtain the weight of each channel.

[0039] The local feature enhancement branch can achieve multi-attention by using several different attention heads for feature extraction, thereby fully accommodating different aspects of local features. The local feature enhancement branch divides the feature map into blocks of uniform size and uses a transformer to enhance the features of each block. The transformer divides the input feature map into overlapping patches for embedding, and then performs layer normalization before inputting it into the multi-head attention for enhancement. Each attention head performs self-attention calculations separately, and then reorganizes the divided feature maps.

[0040] The internal structure of the transformer block is as follows: in each transformer block, the feature map is first downsampled using a convolution kernel size of 7×7 and a stride of 4×4; then, after performing layer normalization, multiple attention heads are used to perform self-attention calculations on the feature maps; and then a feedforward neural network (including a 3×3 convolution and a multi-layer perceptron) is used to further process the features.

[0041] Finally, the weights obtained by the global feature enhancement branch are multiplied by the features obtained by the local feature enhancement branch, and then added to the input feature map to enhance the feature map while retaining the original features.

[0042] The output of the multi-level feature enhancement module is input into the multi-layer perceptron of the decoder for linear transformation. The multi-layer perceptron of the decoder includes two fully connected layers for mapping features of different scales to the same feature space dimension.

[0043] S24. Design of multi-attention fusion module: Due to the complexity of the interweaving of the nearshore background, the direct summation of the feature map after the multi-layer perceptron may introduce interference information irrelevant to the target. In order to minimize or even eliminate the interference of complex background interference information and enable the model to focus more on the task target, the applicant designed a multi-attention fusion module. The feature map processed by the multi-layer perceptron of the decoder is input into the multi-attention fusion module, and the multi-attention fusion module outputs the feature map after attention enhancement for subsequent ship target detection.

[0044] The multi-attention fusion module consists of channel attention and spatial attention. In order to enhance the ability to capture local details and correlations between specific locations, channel attention is first introduced. Channel attention first performs a global average pooling operation on the input feature map to obtain a global feature representation of each channel. Then, the features after global average pooling are processed by a one-dimensional convolution to generate channel attention weights. This convolution operation uses a convolution layer with a convolution kernel size of K to achieve local cross-channel information interaction between each channel and its K neighboring channels while avoiding information loss caused by dimensionality reduction. Then, the ReLU activation function is applied to the convolution weights to adjust the weights. Finally, the adjusted weights are multiplied element-by-element with the original feature map to generate a feature map weighted by channel attention. This process enhances the expressiveness of the model by focusing on important channel features.

[0045] The calculation formula for the convolution kernel size K is: In the formula, express The nearest odd number, C represents the number of channels of the input feature map, and γ and b are coefficients.

[0046] The output of channel attention is input into spatial attention. Spatial attention first performs two 1×1 convolution operations on the input feature map. The first convolution operation converts the input feature map into query features, and the second convolution operation converts the input feature map into numerical features. The convolution kernel size of these two convolution operations is 1×1 and reduces the number of channels of the feature map to half of the original for subsequent processing.

[0047] The query features are then globally average pooled to obtain a global feature representation for each channel. This process compresses the spatial dimensions (height and width) of the feature map into a single global feature to capture the overall information of the feature map. A one-dimensional convolution with a kernel size of 1×1 is applied to the globally pooled query features to learn the spatial attention weights. The spatial attention weights are then generated by varying the shape and activation function of the weights (including Softmax and Sigmoid). This process helps to further adjust and optimize the important information in the feature map.

[0048] The numerical features are convolved and reshaped, and then multiplied with the spatial attention weights to obtain a weighted feature map. The weighted feature map is then reshaped, activated, and multiplied with the input feature map to obtain an attention-enhanced feature map.

[0049] S25. Design a hybrid loss function: The hybrid loss function consists of shape segmentation loss and boundary weighted cross entropy loss, that is, the total loss of the SAR image ship target detection network model is the sum of the shape segmentation loss and the boundary weighted cross entropy loss.

[0050] Since SAR images usually contain fewer ship targets and only a small number of pixels represent these targets, this embodiment uses shape segmentation loss (Dice loss). Dice loss is a pixel-level loss function for segmentation tasks, which helps to alleviate the negative impact of unbalanced datasets, where the number of positive pixels (ship targets) is significantly less than the number of negative pixels (background). Dice loss is used to measure the similarity between model predictions and true labels, so it is suitable for tasks where accurate positioning of targets is crucial. Shape segmentation loss DiceLoss: Among them, y i represents the i-th pixel of the true label, represents the i-th pixel predicted by the model, N is the total number of sample pixels, and ∈ is a small regularization term to avoid division by zero error.

[0051] Although Dice loss can effectively mine foreground areas and guarantee a low false negative rate, it pays less attention to the boundaries of ships. To this end, this embodiment designs a boundary-weighted cross-entropy loss function, which is strategically combined with the Dice loss function to ensure that the SAR image ship target detection network model not only focuses on the detection shape, but also pays enough attention to the boundaries of ships. In traditional cross-entropy loss, each pixel contributes equally to the overall loss. However, pixels close to the ship boundary are more important for model training because they are located in the transition area between the ship and the background. Boundary-weighted cross-entropy loss introduces pixel-level weights to give pixels close to the ship boundary greater importance. This strategic weighting ensures that the loss function gives priority to the edge parts of the ship during training. Therefore, the model is better at accurately modeling the boundaries of ships, thereby more effectively solving the challenges of ship overlap.

[0052]

[0053] Among them, y is the true label, is the model prediction, CE represents the cross entropy loss function, mean represents the mean function, and Loss edge represents the boundary weighted cross entropy loss, and bw is the boundary attention weight.

[0054] S3, training the SAR image ship target detection network model, iterating and optimizing the network parameters by stochastic gradient descent method, using the training set and validation set to train the network model, and setting the pre-training hyperparameters and loss function weights, the pre-training hyperparameters include the maximum number of iterations, initial learning rate, weight decay rate and batch size. In this embodiment, the maximum number of iterations is 100, and the initial learning rate is 1×10 -6 , the weight decay rate is 1×10 -6 , the batch size is 8.

[0055] S4. Use the trained SAR image ship target detection network model to perform ship target detection on the SAR images in the test set to obtain the ship target detection results.

[0056] S5. Test the SAR image ship target detection network model to verify the ship target detection effect.

[0057] (1) Tested on the SSDD dataset. The trained SAR image ship target detection network model is used to predict the images in the test set divided from the SSDD dataset, and the detection accuracy, quality and speed of the SAR image ship target detection network model are evaluated using the Precision, mIoU, mPA, Accuracy and FPS indicators. Where TP, FP, FN and TN represent the number of true positive points, false positive points, false negative points and true negative points respectively, N' represents the total number of categories, c i’ is the number of pixels correctly predicted by the model as category i' (i.e., the number of pixels whose true category is i' and whose prediction is also i'), T i’ is the true number of pixels of category i'.

[0058] At the same time, the commonly used neural networks PSPNet+ and HRNet in the field of deep learning, as well as the neural networks ToCo, Mask2Former, Swin_Unet and Segformer applied in the field of SAR detection are introduced and compared with the SAR image ship target detection network of this embodiment to further verify the effectiveness of this embodiment.

[0059] Table 1 Performance comparison of the network provided in Example 1 and other commonly used neural networks on the SSDD dataset

[0060] network Precision QUR mPA Accuracy FPS PSPNet+ 86.07 81.86 99.48 85.52 126.0 HRNet 77.80 79.35 85.58 99.37 48.6 ToCo 87.65 85.52 89.65 99.59 67.4 Mask2Former 88.72 89.69 92.35 99.67 28.9 Swin_Unet 91.21 88.61 94.18 99.74 58.2 Segformer 84.18 85.65 89.78 99.64 92.4 Example 1 89.94 90.66 94.83 99.89 83.1

[0061] As can be seen from Table 1, the SAR image ship target detection network model provided in this embodiment has achieved the best mIoU (90.66%), mPA (94.83%) and Precision (99.89%), which is better than Mask2Former, Swin_Unet and other models. It can be seen from the ship detection results on the SSDD dataset that Swin_Unet has poor detection effect in complex environments, and has a high missed detection rate for densely distributed small targets. On the contrary, the method proposed in the present invention can effectively suppress interference and accurately and efficiently detect ship targets under the complex background of the port, which is mainly due to the multi-attention fusion mechanism. The multi-attention fusion mechanism can effectively suppress the interference of complex backgrounds by organically integrating multiple attention mechanisms, so that the model's attention is focused on the detection target. At the same time, the method provided by the present invention is also very good at detecting ships of different scales, which is mainly due to the fact that the backbone model integrates multiple feature maps of different scales, which helps the model understand the characteristics of targets at different scales, thereby improving the perception effect of ships at different scales.

[0062] (2) Tested on the HRSID dataset. The trained SAR image ship target detection network model is used to predict the images in the test set divided from the HRSID dataset, and the Precision, mIoU, mPA, Accuracy and FPS indicators are used to evaluate the detection accuracy, quality and speed of the SAR image ship target detection network model.

[0063] As can be seen from Table 2, compared with other commonly used neural networks, the network provided by this embodiment has achieved the best Precision (86.24%) and mIoU (87.72%) on the HRSID dataset, which is better than models such as Swin_Unet and SegFormer, and has also achieved comprehensive competitive results in indicators such as mPA, Accuracy, and FPS. It can be seen from the ship detection results of HRNet, Swin_Unet, SegFormer and this embodiment on the HRSID dataset that compared with HRNet and Swin_Unet, this embodiment has significantly better recognition performance for small ships. A key reason is that the multi-level feature enhancement module enhances the feature map through the global feature extraction branch and the local feature extraction branch, which can effectively enhance the model's extraction effect on target features, which is beneficial to the model's detection of small ships. Compared with SegFormer, although this embodiment reduces the FPS index, other indicators are improved. On the one hand, the method proposed in the present invention combines the advantages of convolutional neural networks and transformers, which saves the speed of feature extraction while ensuring the quality of extracted features; on the other hand, the multi-attention fusion module effectively reduces the interference of complex environments, thereby improving the detection accuracy. In summary, the method proposed in the present invention achieves a good balance between ensuring the detection speed and accurate detection of small targets.

[0064] Table 2 Performance comparison of the network provided in Example 1 and other commonly used neural networks on the HRSID dataset

[0065] network Input image size Precision QUR mPA Accuracy FPS PSPNet+ 473×473 67.90 66.52 69.06 99.76 92.5 HRNet 480×480 75.19 75.12 80.21 99.82 27.5 ToCo 448×448 82.36 78.65 84.78 99.32 38.4 Mask2Former 512×512 85.92 78.91 86.84 99.43 19.0 Swin_Unet 224×224 84.68 85.89 92.27 99.89 40.6 SegFormer 512×512 85.62 86.87 90.12 99.62 79.5 Example 1 512×512 86.24 87.72 91.74 99.87 65.6

[0066] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A SAR image ship target detection method based on multi-attention fusion, characterized in that: The following steps are involved: S1. Construct the SAR image ship target dataset and divide the SAR image ship target dataset into training set, test set and verification set according to the preset ratio; S2. Construct a SAR image ship target detection network model based on multi-attention fusion, including a network main branch, a multi-level feature enhancement module, a multi-attention fusion module and a hybrid loss function. The network main branch includes an encoder and a decoder. The encoder is composed of a convolutional neural network and a hierarchical transformer, and the decoder is a multi-layer perceptron. The multi-level feature enhancement module includes a global feature enhancement branch and a local feature enhancement branch. The global feature enhancement branch is composed of multiple convolutions and global maximum pooling operations, and the local feature enhancement branch is composed of overlapping patch embedding, layer normalization, and multi-head attention. The multi-attention fusion module is composed of channel attention and spatial attention. The hybrid loss function is composed of shape segmentation loss and boundary weighted cross entropy loss. S3, training the SAR image ship target detection network model, iterating and optimizing the network parameters by the stochastic gradient descent method, using the training set and the validation set to train the network model, and setting the pre-training hyperparameters and loss function weights, wherein the pre-training hyperparameters include the maximum number of iterations, the initial learning rate, the weight decay rate, and the batch size; S4. Use the trained SAR image ship object detection network model to detect ship targets on the SAR image in the test center to obtain ship target detection results.

2. The SAR image ship target detection method based on multi-attention fusion according to claim 1 is characterized in that: In step S2, the convolutional neural network of the encoder includes a convolutional block consisting of two convolutions with a convolution kernel size of 3×3, a stride of 2×2 and no padding, and a convolution with a convolution kernel size of 3×3, a stride of 1×1 and padding, and the convolutional neural network is used to extract preliminary features; The encoder's hierarchical transformer consists of four hierarchical transformer blocks. In each hierarchical transformer block, the feature map processed by the previous layer is first downsampled using a convolution kernel size of 7×7 and a stride of 4×4. Then, a 4×4 convolution operation is used to generate overlapping patch embeddings. After layer normalization, the feature map is split into multiple heads, and each head performs self-attention calculation separately; then the features are further processed through a feedforward neural network, which includes two fully connected layers and a nonlinear activation function ReLU.

3. The SAR image ship target detection method based on multi-attention fusion according to claim 2 is characterized in that: In step S2, the global feature enhancement branch of the multi-level feature enhancement module is used to extract the location information of the ship target and the semantic information related to classification. First, two 1×1 convolution operations are applied to the input feature map, and then the weight of each channel is obtained by using the global maximum pooling operation; The local feature enhancement branch of the multi-level feature enhancement module is used to enhance the structural information of the extracted ship target. The local feature enhancement branch divides the feature map into blocks of uniform size and uses a transformer to enhance the features of each block. The transformer divides the input feature map into overlapping patches for embedding, and then performs layer normalization and inputs it into the multi-head attention for enhancement. Each attention head performs self-attention calculation separately, and then the divided feature maps are reorganized; Finally, the weights obtained from the global feature enhancement branch are multiplied by the features obtained from the local feature enhancement branch and then added to the input feature map.

4. The SAR image ship target detection method based on multi-attention fusion according to claim 3 is characterized in that: In step S2, the channel attention of the multi-attention fusion module first performs a global average pooling operation on the input feature map, and then uses a one-dimensional convolution with a convolution kernel size of K to generate a channel attention weight. Then, the ReLU activation function is applied to adjust the channel attention weight, and the adjusted channel attention weight is multiplied with the original feature map to generate a feature map after channel attention weighting; The output of channel attention is input into spatial attention. Spatial attention uses two 1×1 convolution operations to convert the input feature map into query features and numerical features. The query features are then processed with global average pooling, one-dimensional convolution, shape change, and activation function to generate spatial attention weights. The numerical features are convolved and reshaped, and then multiplied with the spatial attention weights to obtain a weighted feature map; the weighted feature map is then reshaped, activated, and multiplied with the input feature map to obtain an attention-enhanced feature map.

5. The SAR image ship target detection method based on multi-attention fusion according to claim 4 is characterized in that: In step S2, the total loss of the SAR image ship target detection network model is the sum of the shape segmentation loss and the boundary weighted cross entropy loss; Shape segmentation loss Among them, y i represents the i-th pixel of the true label, represents the i-th pixel predicted by the model, N is the total number of sample pixels, and ∈ is a small regularization term to avoid division by zero error; Among them, y is the true label, is the model prediction, CE represents the cross entropy loss function, mean represents the mean function, and Loss edge represents the boundary weighted cross entropy loss, and bw is the boundary attention weight.

6. The SAR image ship object detection method based on multi-attention fusion according to claim 4, characterized in that, The calculation formula for the convolution kernel size K is: In the formula, express The nearest odd number, C represents the number of channels of the input feature map, and γ and b are coefficients.

7. The SAR image ship target detection method based on multi-attention fusion according to claim 5 or 6, characterized in that: The input image of the encoder's convolutional neural network is a three-channel RGB image. The encoder extracts the input image layer by layer, and finally inputs the features output by the hierarchical transformer into the multi-level feature enhancement module; The output of the multi-level feature enhancement module is input into the multi-layer perceptron of the decoder for linear transformation. The multi-layer perceptron of the decoder includes two fully connected layers for mapping features of different scales to the same feature space dimension. The feature map processed by the multi-layer perceptron of the decoder is input into the multi-attention fusion module, and the multi-attention fusion module outputs the feature map after attention enhancement for subsequent ship target detection.

Images