Ship remote sensing image detection method based on multi-dimensional attention mechanism

Abstract

The invention discloses a ship remote sensing image detection method based on a multi-dimensional attention mechanism, and the method comprises the steps: constructing a remote sensing image detection model based on YOLOv10, replacing an SPPF module in a Backbone network with an SPPFCMSA module, and enabling the SPPF module to be fused with an attention mechanism CMSA to form the SPPFCMSA module; a PSA module in the Backbone network is replaced by an MSGE module; the method comprises the following steps: replacing the last two C2f modules in a Neck network with a C2fSev2 module, wherein the C2fSev2 module is formed by introducing an improved SELayer2 module into a BottleNeck of the C2f module; and training a remote sensing image detection model by using the data set, and performing remote sensing image ship detection by using the trained remote sensing image detection model. The method has the advantages of high detection precision and small model calculation amount, and can meet real-time remote sensing image detection requirements.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a ship remote sensing image detection method based on a multi-dimensional attention mechanism. Background Art

[0002] With the rapid development of remote sensing technology and deep learning, remote sensing image detection plays an important role in the fields of ocean monitoring, military reconnaissance and environmental protection. Ship detection, as one of the key tasks, aims to accurately identify and locate ship targets from massive remote sensing images. However, due to the complexity of the marine environment, the diversity of ship targets and the interference of the background in high-resolution remote sensing images, ship detection faces many challenges. The complex sea surface is affected by weather changes, lighting conditions and waves, presenting a diverse and dynamic texture background. These factors make it easy for ship targets to be confused with the sea surface background. In addition, there are many types of ships with different shapes, sizes and colors. Different types of ships may show significant visual differences under different shooting conditions, which brings additional difficulties to the detection algorithm. At the same time, ships often occupy very few pixel areas in remote sensing images, which is a typical small target detection problem. They are easily affected by noise interference and resolution limitations, which makes their features easily submerged in the deep feature map of the convolutional neural network (CNN), resulting in low target detection accuracy.

[0003] "SENetV2: Aggregated dense layer for channelwise and global representations" proposed SE Block (Squeeze-and-Excitation Module), which can adaptively adjust the features in the channel dimension and enhance the network's ability to learn global features. This design aims to improve the network's feature expression ability so that it can better capture inter-channel and global information. However, there are still certain limitations in practical applications, and there are obvious deficiencies in the processing of small targets and local features. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a ship remote sensing image detection method based on a multi-dimensional attention mechanism with high detection accuracy.

[0005] Technical solution: The ship remote sensing image detection method based on the multi-dimensional attention mechanism of the present invention comprises:

[0006] (1) Obtain a labeled remote sensing scene ship target detection dataset;

[0007] (2) Build a remote sensing image detection model based on YOLOv10, including:

[0008] Replace the SPPF module in the Backbone network of the YOLOv10 model with the SPPFCMSA module. The SPPFCMSA module is formed by integrating the SPPF module with the attention mechanism CMSA.

[0009] Replace the PSA module in the Backbone network of the YOLOv10 model with the MSGE module;

[0010] Replace the last two C2f modules in the Neck network in the YOLOv10 model with C2fSev2 modules, which are formed by introducing an improved SELayerv2 module in the BottleNeck of the C2f module.

[0011] (3) Use the dataset to train the remote sensing image detection model, use the trained remote sensing image detection model to perform remote sensing image ship detection, and evaluate the performance of the remote sensing image detection model.

[0012] Furthermore, the dataset uses the remote sensing ship dataset HRSC2016.

[0013] Furthermore, in the remote sensing image detection model, the Backbone network includes a first Conv module, a second Conv module, a third Conv module, a first C2f module, a second C2f module, a third C2f module, a first C2fCIB module, an SPPFCMSA module and an MSGE module; the Backbone network includes a fourth Conv module, a fourth C2f module, a second C2fCIB module, a first C2fSev2 module and a second C2fSev2 module; the Head network includes three detection heads;

[0014] The original image is input into the first Conv module, the output of the first Conv module is used as the input of the second Conv module, the output of the second Conv module is used as the input of the first C2f module, and the output of the first C2f module is used as the input of the third Conv module; the output of the third Conv module is used as the input of the second C2f module, one output of the second C2f module is used as the input of the third C2f module after downsampling processing, one output of the third C2f module is used as the input of the first C2fCIB module after downsampling processing, the output of the first C2fCIB module is used as the input of the SPPFCMSA module, and the output of the SPPFCMSA module is used as the input of the MSGE module;

[0015] One output of the MSGE module is spliced ​​with the other output of the third C2f module after upsampling, and then used as the input of the fourth C2f module; one output of the fourth C2f module is spliced ​​with the other output of the second C2f module after upsampling, and then used as the input of the first C2fSev2 module, and one output of the first C2fSev2 module is sent to the v10 detection head; another output of the first C2fSev2 module is used as the input of the fourth Conv module, the output of the fourth Conv module is spliced ​​with the other output of the fourth C2f module, and the spliced ​​feature map is used as the input of the second C2fSev2 module, and the output of the second C2fSev2 module is sent to the v10 detection head. At the same time, the spliced ​​feature map is spliced ​​with the other output of the MSGE module after downsampling, and then used as the input of the second C2fCIB module, and the output of the second C2fCIB module is sent to the v10 detection head.

[0016] Furthermore, the SPPFCMSA module first performs a convolution operation on the input features, and then performs multi-scale feature extraction through three serially connected maxpool2d layers. The output of each pooling layer is connected to the original features through a skip connection. All features are fused through the concat layer and then passed through the CMSA module and the final convolution layer to obtain the output features.

[0017] Furthermore, the CMSA module combines the attention mechanism and convolution operation, which is mainly divided into the upper path and the lower path;

[0018] In the upper path, the input X first passes through a 1×1 convolutional layer to generate three sets of feature maps, which are then passed through a 3×3 depthwise separable convolution to obtain three tensors Q, K, and V;

[0019] First, Q and K are matrix multiplied by the Softmax function to obtain the attention weight A. The calculation formula is as follows:

[0020]

[0021] Among them, d k is the dimension of K;

[0022] Next, the obtained attention weight A is applied to V and multiplied by V to obtain the final weighted feature output Z; the weighted feature output Z passes through a 1×1 convolution layer to adjust the number of channels to match the dimension of the final output; finally, the input X is residually connected to the output to achieve feature jump transfer; the residual connection is calculated as:

[0023] Z residual =X+Z(2)

[0024] Among them, Z residual Indicates the error offset of Z;

[0025] In the lower path, the input X undergoes average pooling and max pooling, which extract global features and local salient features respectively; the pooled results are fused to combine global and local information; then, the fused features pass through a 1×1 convolution to adjust the channel dimension and a GELU activation function to introduce nonlinearity; then, a second 1×1 convolution and a Sigmoid activation function are used to compress the features to between 0 and 1 for weighting the original input features;

[0026] Finally, the output features generated by the attention mechanism and pooling operation are added and fused to obtain new features, which are then added to the original input features to obtain the final output feature map.

[0027] Furthermore, the feature map output by the SPPFCMSA module enters the MSGE module and is combined with a feature X of size H×W×C i As input, H, W, and C represent height, bandwidth, and number of channels respectively; then, after global average pooling and maximum pooling operations, the information of each channel is extracted, and the two pooling operations are fused in a ratio of 6:4 to generate a feature vector z i , the formula is as follows:

[0028]

[0029] in, is the average value of channel i; X i (h, w) represents the value of the i-th feature map at position (h, w); Indicates the maximum value of channel i; Indicates that the maximum value is taken in the range of height from 1 to H and width from 1 to W;

[0030] These weights are then multiplied pixel by pixel with the original feature map through a dot product operation; then, the feature map is normalized; finally, the weight S is generated through a sigmoid activation function and a dot product operation. i And form a weighted feature map, the formula is as follows:

[0031] S i =σ(X i z i ) (6)

[0032] Among them, σ is the sigmoid function; the weighted feature map has the same shape as the input tensor.

[0033] Furthermore, the C2fSev2 module has a multi-branch structure. In the initial stage, 1×1Conv is used to reduce the computational complexity, and then the channels of the input feature map are divided into two paths through Split, one path is directly passed, and the other path enters a sequence composed of n SeNeck blocks; each SeNeck block contains two regular convolutional layers and an improved SELayerV2 module, supplemented by residual connections to optimize the gradient flow; these SeNeck blocks each process the input of half of the channels and output feature maps of the same dimension; finally, the two features are fused through the Concat splicing operation and output through the convolutional layer.

[0034] Furthermore, the improved SELayerV2 module is mainly composed of a SpatSpecSE module and a MultiScaleSE module.

[0035] Furthermore, in the SpatSpecSE module, the shape of the feature map x is first input as (h, w, c), which means that the height of the feature map is h, the width is w, and the number of channels is c; then the features are respectively input into two channels: spatial attention and channel attention;

[0036] For the spatial attention channel, the input feature map first passes through a 1×1 convolution layer, and the convolved features pass through a Sigmoid activation function to generate a spatial attention weight M, whose size is (h, w, 1). The Sigmoid function maps the value to the range of [0, 1];

[0037] For channel attention, the input first globalizes the information of each channel through global average pooling to obtain a new feature size of (1,1,c). Then, the pooled features are linearly transformed to reduce the number of channels to c / r, where r is the reduction ratio. Then, nonlinearity is introduced through the ReLU activation function to better learn the complex relationship between channels. Then, the number of channels is restored to c through a fully connected layer, and the channel attention weight S is generated through the Sigmoid activation function, whose size is (1,1,c).

[0038] Finally, the attention M and S obtained from the above two channels are multiplied to obtain the final output feature Y.

[0039] Furthermore, in the MultiScaleSE module, the input feature map is first subjected to multi-scale average pooling, namely AvgPool1, AvgPool2 and AvgPool3, to capture feature information at different resolutions; these features are then merged and input into the fully connected layer Fc1, after passing through the ReLU activation function, through the second fully connected layer Fc2, and finally through the sigmoid activation function, and the feature weights are adjusted through the scale operation to obtain the final output.

[0040] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0041] This paper combines the SPPF module in YOLOv10 with the attention mechanism CMSA to propose a new spatial pyramid module SPPFCMSA, which significantly improves the effect of remote sensing image detection while maintaining high efficiency. The fusion of this module fully utilizes the multi-scale feature extraction capability of spatial pyramid pooling and the attention mechanism of CMSA, and can better handle complex scenes in large-scale remote sensing images.

[0042] In terms of overall detection accuracy, the present invention introduces an improved SELayerv2 module in the C2f module of the Neck part to further enhance the feature extraction capability. This module can more accurately detect ships of different sizes through more effective feature fusion, while coping with complex background environments and adapting to various weather and lighting conditions, thereby improving the detection capability of small targets and dense scenes.

[0043] In terms of parameter quantity, the present invention replaces some of the self-attention modules (PSA) in the Backbone network with the maximum spatial grouping enhancement module (MSGE). As a relatively concise local attention mechanism, MSGE is different from the complex multi-scale feature processing required by PSA. This replacement effectively reduces the overall parameter quantity of the model. The reduction of parameters not only reduces storage requirements, improves detection efficiency, meets real-time detection requirements, but also reduces the risk of overfitting, making the model more robust in practical applications.

[0044] The present invention solves the problem of insufficient detection capability of complex scenes and small targets in large-scale remote sensing image detection through the above technical means, and achieves a significant improvement in detection effect. The principle of this improvement is that the combination of the SPPFCMSA module and the SELayerv2 module can capture target features more comprehensively, and the introduction of the MSGE module optimizes the structure of the model, ensuring that the complexity and parameter quantity are reduced while maintaining performance, thereby improving the applicability and stability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of a ship remote sensing image detection method based on a multi-dimensional attention mechanism provided by an embodiment of the present invention;

[0046] Figure 2 is a network structure diagram of a remote sensing image detection model in an embodiment of the present invention;

[0047] Figure 3 : is a network structure diagram of the SPPFCMSA module in an embodiment of the present invention;

[0048] Figure 4is a network structure diagram of a CMSA module in an embodiment of the present invention;

[0049] Figure 5 is a network structure diagram of the MSGE module in an embodiment of the present invention;

[0050] Figure 6 is a network structure diagram of the C2fSev2 module in an embodiment of the present invention;

[0051] Figure 7 is a network structure diagram of the improved SELayerv2 module in an embodiment of the present invention;

[0052] Figure 8 is a curve diagram of the remote sensing image detection model training process in an embodiment of the present invention;

[0053] Fig. 9 It is a diagram showing the effect of detection on a remote sensing ship data set in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings.

[0055] like Figure 1 As shown, the embodiment of the present invention provides a ship remote sensing image detection method based on a multi-dimensional attention mechanism, which specifically includes the following steps:

[0056] (1) Obtain a labeled remote sensing scene ship target detection dataset;

[0057] The public remote sensing ship dataset HRSC2016 is used. This dataset collects a total of 1,061 high-resolution satellite images from Google Earth. Each image is annotated with the specific location and type of the ship, covering a total of 26 different types of ships. In this dataset, 436 images are used for training, 181 images are used for verification, and 444 images are used for testing.

[0058] (2) Build a remote sensing image detection model based on YOLOv10, such as Figure 2 As shown, it includes Backbone network, Neck network and Head network.

[0059] The attention mechanism CMSA is introduced on the basis of the SPPF (Spatial Pyramid Pooling) module in the Backbone network to obtain global feature information and reduce the impact of different scales. Due to the high computational cost and memory consumption of the original network, the PSA (Partial Self Attention) module in the Backbone network is replaced by the MSGE (Map Spatial Group-wise Enhance) module to reduce storage requirements and reduce the risk of overfitting. The Neck network enhances the model's detection ability for targets of different scales by introducing an improved SELayerv2 module and integrates the features extracted from the backbone network. The Head network receives features of large, medium and small scales from the Neck network and performs target detection tasks.

[0060] Specifically, the Backbone network includes a first Conv module, a second Conv module, a third Conv module, a first C2f module, a second C2f module, a third C2f module, a first C2fCIB module, an SPPFCMSA module and an MSGE module; the Neck network includes a fourth Conv module, a fourth C2f module, a second C2fCIB module, a first C2fSev2 module and a second C2fSev2 module; the Head network includes three v10 detection heads.

[0061] The original image is input into the first Conv module, the output of the first Conv module is used as the input of the second Conv module, the output of the second Conv module is used as the input of the first C2f module, and the output of the first C2f module is used as the input of the third Conv module; the output of the third Conv module is used as the input of the second C2f module, one output of the second C2f module is used as the input of the third C2f module after downsampling (SCDown), one output of the third C2f module is used as the input of the first C2fCIB module after downsampling (SCDown), the output of the first C2fCIB module is used as the input of the SPPFCMSA module, and the output of the SPPFCMSA module is used as the input of the MSGE module.

[0062] One output of the MSGE module is concatenated (Concat) with another output of the third C2f module after upsampling (Upsample) processing, and then used as the input of the fourth C2f module; one output of the fourth C2f module is concatenated (Concat) with another output of the second C2f module after upsampling (Upsample) processing, and then used as the input of the first C2fSev2 module, and one output of the first C2fSev2 module is sent to the v10 detection head; another output of the first C2fSev2 module is used as the input of the fourth Conv module, the output of the fourth Conv module is concatenated (Concat) with another output of the fourth C2f module, and the concatenated feature map is used as the input of the second C2fSev2 module, and the output of the second C2fSev2 module is sent to the v10 detection head. At the same time, the concatenated feature map is concatenated (Concat) with another output of the MSGE module after downsampling (SCDown) processing, and then used as the input of the second C2fCIB module, and the output of the second C2fCIB module is sent to the v10 detection head.

[0063] The backbone network extracts ship features through each module. As the size of the input image decreases, the extracted feature information gradually increases. The features extracted by the second and third layers of the C2f and MSGE modules in the backbone network are output to the Neck network for further feature fusion. Through the fusion and redistribution method, the feature information is enhanced and becomes richer. Finally, the feature information of different sizes is input to the detection head, which determines whether there is an object to be detected.

[0064] like Figure 3 As shown in the figure, the SPPF module and the attention mechanism CMSA in the Backbone network are integrated into a new spatial pyramid module SPPFCMSA. In remote sensing ship detection, the combination of SPPF and attention mechanism modules can help the network focus on important features and capture long-term dependencies, thereby improving the detection accuracy of ships of different scales. SPPF captures multi-scale spatial features and enhances the model's adaptability to ships of different sizes; the attention mechanism focuses on important feature areas, filters noise, and further improves the detection capability of small targets; at the same time, the combination of the two effectively improves the detection effect, maintains high efficiency, and is suitable for processing complex scenes in large-scale remote sensing images. The specific workflow is:

[0065] The SPPFCMSA module first performs convolution operations on the input features, and then extracts multi-scale features through three series-connected maxpool2d layers. The output of each pooling layer is connected to the original features through a jump connection. All features are fused through the concat layer, and then the output features are obtained through the CMSA attention module and the final convolution layer. This structure can capture feature information of different receptive fields and enhance the feature expression ability of the model.

[0066] The CMSA module is a complex and powerful attention module that combines the CAFM (Convolution and Attention Fusion Module) module, the attention fusion module, and the improved SE (Squeeze-and-Excitation) module. The CAFM module contains local branches and global branches, which are used to fuse the global and local features captured by the convolution and attention mechanisms. In order for the network to better capture and utilize global context information, the SE module is introduced and improved in the CAFM module. At the same time, the local branches and complex local convolution processing in the CAFM module are removed, and a simpler depth-separable convolution (Dwconv) is adopted.

[0067] The CMSA module combines the attention mechanism and convolution operation, which is mainly divided into the upper path and the lower path, such as Figure 4 shown.

[0068] In the upper path, the input X first passes through a 1×1 convolutional layer to generate three sets of feature maps, which are then passed through a 3×3 depthwise separable convolution (Dwconv) to obtain three tensors Q (query), K (key), and V (value);

[0069] First, Q and K are matrix multiplied by the Softmax function to obtain the attention weight A. The calculation formula is as follows:

[0070]

[0071] Among them, d k is the dimension of K, used for scaling to ensure that the dot product value is not too large.

[0072] Next, the obtained attention weight A is applied to V and multiplied by V to obtain the final weighted feature output Z; the weighted feature output Z passes through a 1×1 convolution layer to adjust the number of channels to match the dimension of the final output; finally, the input X is residually connected to the output to achieve feature jump transmission in order to retain the input information and avoid the gradient vanishing problem; the residual connection is calculated as:

[0073] Z residual =X+Z(2)

[0074] Among them, Z residual Indicates the error offset of Z.

[0075] In the lower path, the input X undergoes average pooling and max pooling, which extract global features and local salient features respectively; the pooled results are fused to combine global and local information; then, the fused features pass through a 1×1 convolution to adjust the channel dimension and introduce nonlinear capabilities through the GELU activation function; then, they pass through a second 1×1 convolution and a Sigmoid activation function to compress the features to between 0 and 1 for weighting the original input features.

[0076] Finally, the output features generated by the attention mechanism and pooling operation are added and fused to obtain new features, which are then added to the original input features to obtain the final output feature map.

[0077] In order to better integrate the features generated by the SPFCMSA module and reduce the number of parameters of the model, the present invention adopts the MSGE module to replace the PSA module in the original network. Although the PSA module improves efficiency through parallel processing in the YOLOv10 model, enhances multi-scale feature capture and spatial information utilization, it also has the disadvantages of high computational cost and large memory consumption, which may make it difficult to deploy on resource-constrained devices. Therefore, the present invention introduces the MSGE module, which is improved based on the SGE (Spatial Group-wise Enhance, spatial grouping enhancement) module and introduces the maximum pooling operation. The MSGE module is a relatively concise local attention mechanism, which does not require complex multi-scale feature processing like the PSA module, so it can reduce the overall number of parameters of the model. Less parameters not only reduce storage requirements, but also reduce the risk of overfitting, making the model more robust.

[0078] The structure of the MSGE module is as follows Figure 5 As shown, the specific workflow is:

[0079] First, the feature map output by the SPPFCMSA module enters the MSGE module and is combined with a feature X of size H×W×C i As input, H, W, and C represent height, bandwidth, and number of channels respectively. Then, after global average pooling and maximum pooling operations, the information of each channel is extracted, and the two pooling operations are fused in a ratio of 6:4 to generate a feature vector z i , the formula is as follows:

[0080]

[0081] in, is the average value of channel i; X i(h, w) represents the value of the i-th feature map at position (h, w); Indicates the maximum value of channel i; Indicates that the maximum value is taken in the range from 1 to H for height and 1 to W for width.

[0082] The reason why the Map maximum pooling operation is added on the basis of the SGE attention mechanism is that the maximum pooling can reduce the dimension of the feature map and the amount of calculation by extracting the maximum value of the local area, while retaining the key features and improving the robustness of the model to changes in position and ship scale.

[0083] Subsequently, these weights are multiplied pixel by pixel with the original feature map through a dot product operation to dynamically adjust the feature response, thereby enhancing the features related to the ship. This method effectively highlights the ship and suppresses the interference of the background, making the model more accurate and robust, especially when dealing with complex backgrounds, which can significantly reduce the occurrence of missed detections and false detections. Next, the feature map is normalized to further improve the numerical stability and facilitate subsequent feature extraction. The core feature map module is responsible for learning and extracting important spatial and channel information in the image, and finally generates the weight S through the sigmoid activation function and dot product operation. i And form a weighted feature map, the formula is as follows:

[0084] S i =σ(X i z i ) (6)

[0085] Among them, σ is the sigmoid function;

[0086] This weighted feature map is consistent with the shape of the input tensor, allowing the model to automatically focus on important image areas through the attention mechanism, which helps to improve the overall performance of ship detection in remote sensing images.

[0087] The main task of the Neck network is to extract features of different scales from the feature map passed from the Backbone network, and gradually integrate the feature information through multiple upsampling and splicing operations to better perform target detection.

[0088] The following is a detailed description of the Neck network:

[0089] like Figure 2, the Neck network first receives the feature map output from the MSGE module, and upsamples the feature map with a size of 20×20 and a number of channels of 128 to double its original size, and finally obtains a feature map with a size of 40×40×128. The feature map is spliced ​​with the feature map output by the third C2f module in the channel dimension, and the output feature map size is 40×40 and the number of channels is 640; then, it is reduced to 512 through the fourth C2f module. Then, it is upsampled and spliced ​​again to make the number of channels reach 768 and the feature map size becomes 80×80. Subsequently, it is reduced to 256 through the first C2fSev2 module, and the size remains unchanged, forming the first output branch. Then, through the Conv convolution operation, the number of channels remains unchanged, and the size is halved to 40×40; then, the feature map obtained by convolution and the feature map obtained by the fourth C2f module are spliced ​​in the channel dimension, and the output feature map size is 40×40×768. It is then adjusted to 512 through the second C2fSev2 module to form the second output branch. Finally, after SCDown and concatenation operations, the number of channels increases to 640, with a size of 20 × 20. Finally, it is expanded to 1024 through the second C2fCIB module to form the third output branch.

[0090] This complex channel number adjustment process aims to effectively fuse multi-scale features while providing appropriate feature representation for object detection of different sizes. It ultimately outputs three feature maps of 256, 512, and 1024 channels, which are passed to the V10 detection head, corresponding to small, medium, and large-scale object detection, for object detection processing to improve the overall performance and detection accuracy of the model.

[0091] like Figure 6 As shown in the figure, the C2fSev2 module adopts an efficient multi-branch structure design. In the initial stage, the computational complexity is reduced by 1×1Conv, and then the channels of the input feature map are divided into two paths through the Split operation, one path is directly passed, and the other path enters a sequence composed of n SeNeck blocks; each SeNeck block contains two regular convolutional layers and an improved SELayerV2 module, supplemented by residual connections to optimize the gradient flow; these SeNeck blocks each process the input of half of the channels and output feature maps of the same dimension; the repeated SELayerV2 structure is conducive to capturing multi-scale feature information. Finally, the two-way features are fused through the Concat splicing operation and output through the convolution layer, providing rich feature representation for subsequent processing. This design cleverly balances computational efficiency and feature extraction capabilities, enhances the network's expressiveness, and maintains the flexibility and scalability of the structure, making it suitable for a variety of complex visual tasks.

[0092] Improved SELayerV2 modules, such as Figure 7As shown in the figure, it includes two submodules: SpatSpecSE (spectral spatial attention) module and MultiScaleSE (multi-scale attention) module. The traditional SElayer module simply extracts global features through global average pooling and learns the attention between channels through the fully connected layer, which is applied to the input features. However, in remote sensing images, ship detection faces the problem of complex environmental background and differences in the spectral and spatial characteristics of ship targets. SELayerV2 adds spectral and spatial attention (SpatSpecSE), as shown in the figure. Figure 7 As shown in the right figure, based on the traditional SE, it is further divided into spectral attention and spatial attention. Spectral attention adjusts the relationship between channels through adaptive average pooling and a fully connected layer, while spatial attention learns spatial feature enhancement through 1×1 convolution. This change enables the model to focus on both the channel information and spatial information of the feature, greatly enriching the expression ability of attention.

[0093] The specific workflow is as follows: first, the shape of the input feature map x is (h, w, c), which means that the height of the feature map is h, the width is w, and the number of channels is c; then the features are entered into two channels respectively: spatial attention and channel attention.

[0094] For the spatial attention channel, the input feature map first passes through a 1×1 convolution layer. The purpose of the convolution operation is to compress the spatial information and extract the importance information of the target area. The convolved features pass through a Sigmoid activation function to generate a spatial attention weight M, whose size is (h, w, 1). The Sigmoid function maps the value to the range of [0, 1] in order to assign an importance score to each spatial position. The spatial attention helps the model identify the important spatial position of the ship and reduce the interference of the ocean background.

[0095] For channel attention, the input first globalizes the information of each channel through global average pooling to obtain a new feature size of (1,1,c). Then, the pooled features are linearly transformed to reduce the number of channels to c / r, where r is the reduction ratio, usually 16, in order to reduce the amount of calculation and highlight important channel information. Then, nonlinearity is introduced through the ReLU activation function to better learn the complex relationship between channels. Then, the number of channels is restored to c through a fully connected layer, and the channel attention weight S is generated through the Sigmoid activation function, whose size is (1,1,c). Channel attention enhances the key channel information in the feature map, allowing the model to selectively strengthen channel features that are helpful for ship detection.

[0096] Finally, the attention M and S obtained from the above two channels are multiplied to obtain the final output feature Y. This mechanism that combines spatial and channel dual attention enables the model to more accurately identify and detect ships in complex ocean scenes, especially in the face of complex backgrounds and scale changes, greatly improving detection accuracy and robustness.

[0097] Since the problem of missed detection or false detection often occurs in remote sensing ship detection due to inconsistent ship sizes, SELayerV2 introduces multi-scale attention (MultiScaleSE), such as Figure 7 As shown in the left figure. It first performs multi-scale average pooling on the input feature map, namely AvgPool1, AvgPool2 and AvgPool3, to capture feature information at different resolutions, which is particularly important in ship detection because the size and shape of ships may vary greatly in remote sensing images; then these features are merged and input into the fully connected layer (Fc1), after the ReLU activation function, through the second fully connected layer (Fc2), and finally through the sigmoid activation function (σ), and the feature weights are adjusted through the scale operation to obtain the final output. This mechanism is similar to the attention mechanism, which enables the model to focus on features useful for ship detection and ignore the interference of complex backgrounds, thereby significantly improving the accuracy and efficiency of detection.

[0098] The improved SELayerv2 module is a channel-based attention mechanism that combines spectral (channel) and spatial attention and introduces multi-scale processing. This design enables the model to capture important features in the image more comprehensively. In the application of remote sensing ship detection, the improved SELayerV2 shows many advantages. It can effectively handle ships of different sizes, cope with complex background environments, and improve the detection capabilities of small targets and dense scenes. In addition, it also provides better model interpretability and has the potential to be applied to real-time processing systems.

[0099] (3) The remote sensing image detection model is trained using the HRSC2016 dataset, the model parameters with the highest accuracy on the validation set after the training process are saved, and the performance of the model is evaluated.

[0100] The experimental environment is configured with Ubuntu 20.04 operating system, NVIDIA GeForce RTX4060 graphics card, and PyTorch deep learning framework. The specific configuration is shown in Table 1.

[0101] Table 1 Experimental environment configuration table

[0102]

[0103] The training set image data in the remote sensing ship dataset is input into the improved network model, and GPU training is used to set the parameters of the improved network model. The number of training rounds is 300 rounds, the SGD optimizer is used, and the batch-size is 4. During the model training process, the model parameters with the best accuracy will be saved.

[0104] Figure 8 The figure is the change curve of various indicators during 300 rounds of model training. After the model training is completed, the accuracy of ships in the validation set is tested by the validation set, and the mAP reaches 75.1%, and the mAP reaches 76.7%. Compared with the original YOLOv10s, the algorithm of the present invention has improved the accuracy of ships by 7.9% and mAP by 2.4%; and the recall rate has also increased slightly from 0.689 to 0.709. Among them, the larger the accuracy index, the better the model recognition effect; mAP represents the quality of the model in all categories, and the larger the index, the better the model network performance. The recall rate is a measure of the integrity of the detection target. It indicates the proportion of samples that are actually positive and are correctly identified by the model. A high recall rate means that the model has fewer missed detections. The algorithm index comparison table is shown in Table 2, which proves the feasibility of the algorithm of the present invention.

[0105] Table 2 Comparison of experimental results of different models

[0106]

[0107] Fig. 9 This is a detection effect diagram of the algorithm of the present invention.

[0108] The present invention has good feature extraction capability, small model size, small calculation amount, strong real-time performance, and can meet the needs of deploying real-time remote sensing ship detection.

Claims

1. A ship remote sensing image detection method based on a multi-dimensional attention mechanism, characterized in that: include: (1) Obtain a labeled remote sensing scene ship target detection dataset; (2) Build a remote sensing image detection model based on YOLOv10, including: Replace the SPPF module in the Backbone network of the YOLOv10 model with the SPPFCMSA module. The SPPFCMSA module is formed by integrating the SPPF module with the attention mechanism CMSA. Replace the PSA module in the Backbone network of the YOLOv10 model with the MSGE module; Replace the last two C2f modules in the Neck network in the YOLOv10 model with C2fSev2 modules, which are formed by introducing an improved SELayerv2 module in the BottleNeck of the C2f module. (3) Use the dataset to train the remote sensing image detection model, use the trained remote sensing image detection model to perform remote sensing image ship detection, and evaluate the performance of the remote sensing image detection model.

2. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 1 is characterized in that: The dataset uses the remote sensing ship dataset HRSC2016.

3. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 1 is characterized in that: In the remote sensing image detection model, the Backbone network includes the first Conv module, the second Conv module, the third Conv module, the first C2f module, the second C2f module, the third C2f module, the first C2fCIB module, the SPPFCMSA module and the MSGE module; the Backbone network includes the fourth Conv module, the fourth C2f module, the second C2fCIB module, the first C2fSev2 module and the second C2fSev2 module; the Head network includes three detection heads; The original image is input into the first Conv module, the output of the first Conv module is used as the input of the second Conv module, the output of the second Conv module is used as the input of the first C2f module, and the output of the first C2f module is used as the input of the third Conv module; the output of the third Conv module is used as the input of the second C2f module, one output of the second C2f module is used as the input of the third C2f module after downsampling processing, one output of the third C2f module is used as the input of the first C2fCIB module after downsampling processing, the output of the first C2fCIB module is used as the input of the SPPFCMSA module, and the output of the SPPFCMSA module is used as the input of the MSGE module; One output of the MSGE module is spliced ​​with the other output of the third C2f module after upsampling, and then used as the input of the fourth C2f module; one output of the fourth C2f module is spliced ​​with the other output of the second C2f module after upsampling, and then used as the input of the first C2fSev2 module, and one output of the first C2fSev2 module is sent to the v10 detection head; another output of the first C2fSev2 module is used as the input of the fourth Conv module, the output of the fourth Conv module is spliced ​​with the other output of the fourth C2f module, and the spliced ​​feature map is used as the input of the second C2fSev2 module, and the output of the second C2fSev2 module is sent to the v10 detection head. At the same time, the spliced ​​feature map is spliced ​​with the other output of the MSGE module after downsampling, and then used as the input of the second C2fCIB module, and the output of the second C2fCIB module is sent to the v10 detection head.

4. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 3 is characterized in that: The SPPFCMSA module first performs a convolution operation on the input features, and then extracts multi-scale features through three serially connected maxpool2d layers. The output of each pooling layer is connected to the original features through a skip connection. All features are fused through the concat layer and then passed through the CMSA module and the final convolution layer to obtain the output features.

5. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 4 is characterized in that: The CMSA module combines the attention mechanism and convolution operation, which is mainly divided into the upper path and the lower path; In the upper path, the input X first passes through a 1×1 convolutional layer to generate three sets of feature maps, which are then passed through a 3×3 depthwise separable convolution to obtain three tensors Q, K, and V; First, Q and K are matrix multiplied by the Softmax function to obtain the attention weight A. The calculation formula is as follows: Among them, d k is the dimension of K; Next, the obtained attention weight A is applied to V and multiplied by V to obtain the final weighted feature output Z; the weighted feature output Z passes through a 1×1 convolution layer to adjust the number of channels to match the dimension of the final output; finally, the input X is residually connected to the output to achieve feature jump transfer; the residual connection is calculated as: Z residual =X+Z(2) Among them, Z residual Indicates the error offset of Z; In the lower path, the input X undergoes average pooling and max pooling, which extract global features and local salient features respectively; the pooled results are fused to combine global and local information; then, the fused features pass through a 1×1 convolution to adjust the channel dimension and a GELU activation function to introduce nonlinearity; then, a second 1×1 convolution and a Sigmoid activation function are used to compress the features to between 0 and 1 for weighting the original input features; Finally, the output features generated by the attention mechanism and pooling operation are added and fused to obtain new features, which are then added to the original input features to obtain the final output feature map.

6. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 5 is characterized in that: The feature map output by the SPPFCMSA module enters the MSGE module and is combined with a feature X of size H×W×C i As input, H, W, and C represent height, bandwidth, and number of channels respectively; then, after global average pooling and maximum pooling operations, the information of each channel is extracted, and the two pooling operations are fused in a ratio of 6:4 to generate a feature vector z i , the formula is as follows: in, is the average value of channel i; X i (h, w) represents the value of the i-th feature map at position (h, w); Indicates the maximum value of channel i; Indicates that the maximum value is taken in the range of height from 1 to H and width from 1 to W; These weights are then multiplied pixel by pixel with the original feature map through a dot product operation; then, the feature map is normalized; finally, the weight S is generated through a sigmoid activation function and a dot product operation. i And form a weighted feature map, the formula is as follows: S i =σ(X i z i ) (6) Among them, σ is the sigmoid function; the weighted feature map has the same shape as the input tensor.

7. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 3 is characterized in that: The C2fSev2 module has a multi-branch structure. In the initial stage, 1×1Conv is used to reduce the computational complexity. Then, the channels of the input feature map are divided into two paths through the Split operation. One path is passed directly, and the other enters a sequence composed of n SeNeck blocks. Each SeNeck block contains two regular convolutional layers and an improved SELayerV2 module, supplemented by residual connections to optimize the gradient flow. These SeNeck blocks each process the input of half of the channels and output feature maps of the same dimension. Finally, the two features are fused through the Concat splicing operation and output through the convolutional layer.

8. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 7 is characterized in that: The improved SELayerV2 module is mainly composed of the SpatSpecSE module and the MultiScaleSE module.

9. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 8 is characterized in that: In the SpatSpecSE module, the shape of the feature map x is first input as (h, w, c), which means that the height of the feature map is h, the width is w, and the number of channels is c; then the features are respectively input into two channels: spatial attention and channel attention; For the spatial attention channel, the input feature map first passes through a 1×1 convolution layer, and the convolved features pass through a Sigmoid activation function to generate a spatial attention weight M, whose size is (h, w, 1). The Sigmoid function maps the value to the range of [0, 1]; For channel attention, the input first globalizes the information of each channel through global average pooling to obtain a new feature size of (1,1,c). Then, the pooled features are linearly transformed to reduce the number of channels to c / r, where r is the reduction ratio. Then, nonlinearity is introduced through the ReLU activation function to better learn the complex relationship between channels. Then, the number of channels is restored to c through a fully connected layer, and the channel attention weight S is generated through the Sigmoid activation function, whose size is (1,1,c). Finally, the attention M and S obtained from the above two channels are multiplied to obtain the final output feature Y.

10. The ship remote sensing image detection method based on multi-dimensional attention mechanism according to claim 8 is characterized in that: In the MultiScaleSE module, the input feature map is first subjected to multi-scale average pooling, namely AvgPool1, AvgPool2 and AvgPool3, to capture feature information at different resolutions; then these features are merged and input into the fully connected layer Fc1, after passing through the ReLU activation function, and then through the second fully connected layer Fc2, and finally through the sigmoid activation function, and the feature weights are adjusted through the scale operation to obtain the final output.

Images