Small target ship detection method based on multistage attention mechanism and cross-layer fusion
By constructing a multi-level attention mechanism and a cross-layer fusion method for small target detection, the problem of missed detection and false detection of small targets and occluded targets in ship detection under complex scenarios is solved, and efficient and accurate detection of small targets on the sea surface is achieved.
Patent Information
- Application Number
- CN202511231625.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-31
- Publication Date
- 2025-12-16
AI Technical Summary
Existing ship detection algorithms struggle to effectively detect small and occluded targets in complex scenarios, resulting in missed or false detections, and are severely affected by background noise.
We employ a small target detection method based on multi-level attention mechanism and cross-layer fusion, which includes constructing a dataset focused on small targets, designing a multi-level attention fusion network model, and introducing a convolutional attention module, a cross-layer interactive feature reorganization pyramid network, and an efficient local attention module to enhance feature extraction and target localization capabilities.
It enables accurate detection of small targets on the sea surface of ships in complex environments, reduces missed detections and false detections, and improves detection accuracy and robustness.
Smart Images
Figure CN121147488A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to the technical field of intelligent shipping, and particularly relates to a small target ship detection method based on a multi-level attention mechanism and cross-layer fusion. BACKGROUND
[0002] Ship target detection is a key technology for realizing intelligent shipping. However, with the development of marine shipping, complex navigation environments and diversified ship types have brought great challenges to accurate ship detection. Ship detection methods are mainly divided into traditional picture processing technology-based methods and deep learning-based detection methods. The traditional method is based on manual feature extraction and is only suitable for a single scene and has poor generalization and consumes a large amount of manpower and material resources, and cannot meet the ship detection task in a complex scene. The other type, the deep learning-based detection method, relies on its powerful feature expression ability and is widely used in channel monitoring, intelligent shipping and other scenes. At present, although the existing target detection algorithm has achieved excellent detection results, there are still obvious problems in the face of ship detection tasks in complex scenes.
[0003] Marine ship targets have the characteristics of significant size difference and dense distribution, which makes most ships appear as small targets on the supervision image and are prone to mutual occlusion. This leads to many deficiencies of the existing algorithm in the face of ship detection tasks (1) susceptible to background noise interference, difficult to extract effective features; (2) missing detection and false detection of small targets and occluded targets, resulting in low detection accuracy. Therefore, it is urgent to propose a propagation detection method that can solve the problem of missing detection and false detection of small targets and occluded targets in the existing model in the ship detection in a complex scene. SUMMARY
[0004] The purpose of the present application is to solve the technical problems in the above background, and to propose a small target ship detection method based on a multi-level attention mechanism and cross-layer fusion, comprising the following steps:
[0005] S1, a small target ship detection data set is constructed, including data sets of different background environments, different weather conditions and common ship types;
[0006] S2, a small target detection network model based on a multi-level attention mechanism and cross-layer fusion is constructed;
[0007] S3, the small target detection network model constructed in step S2 is trained;
[0008] S4, the small target to be detected is detected.
[0009] In the preferred scheme, the small target detection network model in step S2 is designed based on a one-stage detection algorithm, including a backbone network, a neck network and a detection head three parts, and the specific construction process comprises the following steps:
[0010] S21, ResNet18 is used as a backbone network for extracting multi-scale high-resolution ship feature maps, the feature maps including deep features, middle features and shallow features;
[0011] S22, a convolutional attention fusion module is introduced at the output end of the backbone network;
[0012] S23, a cross-level interaction feature reorganization pyramid network is designed as a neck network;
[0013] S24, an efficient local attention module is embedded at the output end of the cross-level interaction feature reorganization pyramid network.
[0014] In a preferred scheme, the convolutional attention fusion module adopts a double-branch parallel structure, including a local branch and a global branch, wherein the local branch focuses on extracting local features and spatial adjacent relationships, and the global branch aims to capture long-distance dependencies and global context information, and the specific implementation process includes:
[0015] The local branch uses multi-level convolution operation combined with channel reorganization strategy to enhance cross-channel information interaction while preserving spatial locality;
[0016] The global branch uses an improved self-attention mechanism to realize effective long-range dependency modeling under the premise of reducing computational complexity;
[0017] Finally, the output features of the local branch and the global branch are integrated through an adaptive fusion strategy to form a final feature representation with both local accuracy and global consistency;
[0018] The output calculation formula of the CAFM module is:
[0019] F out =F conv +F att ;
[0020] Wherein, the output of F conv , F att is the output of the global branch.
[0021] In a preferred scheme, in the local branch, for the input feature X:
[0022] First, use a 1x1 convolution Conv 1×1 to adjust the channel dimension, then divide the input tensor into multiple groups along the channel dimension, and use a depth separable convolution in each group to perform channel shuffling operation, then reconnect the output tensor of each group along the channel dimension to obtain a new tensor, and finally output F conv after a 3x3 convolution Conv 3×3 , the expression is:
[0023] F conv =Conv 3×3 (CS(Conv 1×1 (X)));
[0024] wherein CS represents a channel shuffle operation.
[0025] In a preferred scheme, in the global branch, for the input feature X:
[0026] First, generate the query Q, the key K and the value V through 1x1 convolution and 3x3 deep convolution, to obtain three tensors with the shape of HxWxC; then, reshape the query Q to QER HW×C , and reshape the key K to KER C×HW , and then calculate the attention map AER C×C by the interaction of the reshaped Q and K, and the output Fatt of the global branch is defined as:
[0027] F att =Conv 1×1 Attention(Q,K,V)+X;
[0028] wherein,
[0029]
[0030] wherein, ε represents a learnable scaling parameter, which is used to control the amplitude of the product of the K and Q matrices before applying the softmax function.
[0031] In a preferred scheme, the cross-level interactive feature reorganization pyramid network additionally increases a direct feature fusion path from the deep layer to the shallow layer on the basis of the traditional FPN top-down path, efficiently injects high-level semantic information into the shallow layer, and thus suppresses the interference of background noise on small target detection.
[0032] In a preferred scheme, the dimer implementation process of the cross-level interactive feature reorganization pyramid network includes:
[0033] Take the deep layer feature at the end of the backbone network, perform upsampling and addition operation, and fuse with the shallow layer feature and the middle layer feature to obtain the intermediate feature; at the same time, retain the FPN top-down path to generate the original FPN features P3, P4; then, further fuse the fused feature and the original FPN features P3, P4 to obtain F3, F4;
[0034] After the double-path fusion, the feature maps F3, F4 retain the high-resolution spatial information of the shallow layer feature, and at the same time, fuse the strong semantic information of the deep layer feature, and balance and strengthen the hierarchical representation ability.
[0035] In a preferred scheme, the high-efficiency local attention module adopts a strip pooling strategy to replace the traditional spatial global pooling method.
[0036] In a preferred scheme, the specific implementation process of the high-efficiency local attention module includes the following steps:
[0037] The high-efficiency local attention module performs feature aggregation along the horizontal and vertical directions in the spatial dimension, effectively modeling the long-distance spatial dependency of the image, accurately focusing on the target relevant area through the directional feature extraction mechanism, and suppressing the interference of irrelevant background features, and the expression is:
[0038]
[0039] wherein, is a one-dimensional horizontal feature vector and is a one-dimensional vertical feature vector.
[0040] The high-efficiency local attention module further optimizes the calculation efficiency by using one-dimensional convolution for lightweight processing of directional feature vectors, wherein an adjustable kernel scaling parameter is introduced to control the range of local feature interaction; then, through the synergistic effect of group normalization and nonlinear activation function, the feature map is finely adjusted, and finally a high-precision position attention prediction map is generated, and the expression is:
[0041] y h =σ(G n (Q h (Z h )));
[0042] y w =σ(G n (Q w (Z w )));
[0043] wherein, Q h and Q w represent one-dimensional convolution, G n represents a group normalization layer, y h and y w represent the position attention weights in the horizontal and vertical dimensions, respectively.
[0044] Finally, the position attention prediction in the horizontal and vertical directions is fused with the original input feature map through a product operation to obtain an enhanced feature map Y.
[0045] Y=F×y h ×y w .
[0046] In a preferred solution, the background environment of the data set in step S1 includes near-shore, far-sea and port scenes, the weather conditions include foggy day, rainy day, sunny day and night, and the common ship types include cargo ship, passenger ship, fishing boat and yacht.
[0047] Compared with the prior art, the present application has the following beneficial effects:
[0048] (1) The present application realizes accurate detection of small ship targets on the sea surface in complex environments through efficient feature extraction and enhancement, improvement of precise positioning ability, and fusion of local and global information.
[0049] (2) The present application reconstructs the backbone network by introducing a convolutional attention fusion module, enhances the model's ability to extract target detail features, and focuses on the key areas of the ship, reducing the loss of features of small targets and occluded targets.
[0050] (3) The present application redesigns a brand new cross-level interactive feature pyramid network, improves the efficiency of semantic information transmission by establishing direct connections between different levels of features, and thus improves the model's detection ability for multi-scale ship targets.
[0051] (4) The present application embeds an efficient local attention module, while maintaining lightweight, accurately captures important features of ship targets, effectively suppresses the interference of complex sea surface backgrounds, and reduces the occurrence of missed detection and false detection.
[0052] (5) The algorithm framework of the present application can be more easily extended to other small target detection tasks in the industry. BRIEF DESCRIPTION OF DRAWINGS
[0053] Figure 1 Part of the images of the self-built small ship target data set.
[0054] Figure 2 The overall framework of the small target detection based on multi-level attention mechanism and cross-layer fusion proposed by the present application.
[0055] Figure 3 The principle diagram of the convolutional attention fusion module.
[0056] Figure 4 The structure diagram of the cross-level interactive feature reconstruction pyramid network.
[0057] Figure 5 The structure diagram of the efficient local attention module.
[0058] Figure 6 The detection visualization result comparison of part of the algorithm on the self-built data set. DETAILED DESCRIPTION
[0059] Two embodiments of the present application are given below. The specific embodiments are only used to further illustrate the present application, and do not limit the protection scope of the present application.
[0060] The present embodiment aims to propose a small target ship mark detection method based on multi-level attention mechanism and cross-layer fusion, to solve the problems of missing detection and false detection of small targets and occluded targets in ship detection in complex scenes by existing models.
[0061] In order to achieve the above purpose, the technical scheme adopted by the present application is:
[0062] The small target detection method based on multi-level attention mechanism and cross-layer fusion comprises:
[0063] Step 1: Construct a small target ship detection data set focusing on multiple background environments, different weather conditions and common ship types, such as Figure 1 as shown.
[0064] Step 2: Construct a small target detection network model based on multi-level attention mechanism and cross-layer fusion.
[0065] The present embodiment proposes a small target ship mark detection method based on multi-level attention mechanism and cross-layer fusion (Small Object Detection Based on Multi-level Attention Mechanism and Cross-layer Fusion, hereinafter referred to as MACF-SOD). The method is inspired by one-stage algorithm and mainly contains three parts: backbone network, neck network and detection head, as shown in Figure 2 Firstly, the algorithm uses ResNet18 as the backbone network to extract multi-scale high-resolution feature maps. Then, a convolution and attention fusion module (CAFM) is introduced at the output end of the backbone network, aiming to strengthen the detailed feature response of the ship target and focus on the key area. This module helps to reduce the feature loss of small targets and occluded targets, and retains more detailed information. Secondly, a cross-level interactive feature reorganization pyramid network is designed as the neck network. This network promotes the fusion of deep features in multi-scale space through cross-level feature interaction, shortens the feature transmission path to reduce information loss, and improves the semantic expression ability of shallow features, thereby enhancing the detection ability of the model for multi-scale ships. An efficient local attention (ELA) module is embedded at the output end of the feature pyramid, which accurately captures the edge features of the ship target, effectively suppresses the interference of the complex sea surface background, and reduces the occurrence of missing detection and false detection.
[0066] The convolutional attention fusion module (CAFM) is specifically:
[0067] Traditional convolutional neural networks mainly focus on local feature extraction, but are difficult to model long-range spatial dependencies due to their inherent receptive field. In contrast, attention mechanisms can effectively establish global feature associations, but often ignore fine local information and spatial neighboring relationships. Based on the complementarity of the two in feature representation capability, the convolutional attention fusion module fuses convolutional neural networks and attention mechanisms to simultaneously capture local fine features and global long-range dependencies through a cooperative working mechanism, achieving comprehensive modeling of complex features, as shown in Figure 3 The module adopts a dual-branch parallel structure, including a local branch and a global branch. The local branch focuses on extracting local features and spatial neighboring relationships, while the global branch aims to capture long-range dependencies and global context information. This dual-path architecture enables the network to simultaneously possess the ability to preserve local details and model global relationships through deep fusion at the feature level. The local branch uses multi-level convolution operations in conjunction with a channel reorganization strategy to enhance cross-channel information interaction while preserving spatial locality; the global branch uses an improved self-attention mechanism to effectively model long-range dependencies while reducing computational complexity. Finally, the output features of the two branches are integrated through an adaptive fusion strategy to form a final feature representation that combines local accuracy and global consistency.
[0068] Specifically, in the local branch, for input feature X, first use 1x1 convolution Conv 1×1 to adjust the channel dimension, then divide the input tensor into multiple groups along the channel dimension, and perform channel shuffle operation in each group using depthwise separable convolution, then re-connect the output tensor of each group along the channel dimension to obtain a new tensor, and finally pass through a 3x3 convolution Conv 3×3 to output F conv . Where CS represents the channel shuffle operation, which is expressed as:
[0069] F conv = Conv 3×3 (CS(Conv 1×1 (X)))
[0070] In the global branch, for input feature X, first generate query (Q), key (K) and value (V) through 1x1 convolution and 3x3 depthwise convolution, obtaining three tensors with shape HxWxC. Then, reshape the query (Q) to Q ∈ R HW×C , reshape the key (K) to K ∈ R C×HW , and calculate the attention map A ∈ R C×C by interacting the reshaped Q and K. The output Fatt of the global branch is defined as:
[0071] F att =Conv 1×1 Attention(Q,K,V)+X
[0072]
[0073] Here, ε represents a learnable scaling parameter, which is used to control the magnitude of the product of the K and Q matrices before applying the softmax function.
[0074] Finally, the output calculation formula of the CAFM module is:
[0075] F out =F conv +F att
[0076] The cross-level interaction feature reorganization pyramid network is specifically as follows:
[0077] This embodiment addresses the problem of high-level semantic information attenuation when fusing multi-scale features in traditional Feature Pyramid Networks (FPNs). It redesigns a Cross-level Interactive Feature Reorganization Pyramid Network (CL-FPN), as follows: Figure 4 As shown, traditional FPN enhances the semantic information of shallow features through a top-down path, but multiple upsampling and convolution operations lead to a gradual loss of semantic information. Shallow features are rich in spatial details but weak in semantics, while deep features are semantically strong but have low resolution. The reconstructed cross-level interactive feature reorganization pyramid network reorganizes features through a cross-level interaction mechanism, aiming to better balance local details and global semantics, improve the performance of small object detection, and alleviate the imbalance between semantics and details.
[0078] This network, building upon the traditional top-down path of FPN, adds an additional direct feature fusion path from deep to shallow layers. This efficiently injects high-level semantic information into the shallow layers, thereby suppressing the interference of background noise on small target detection. Specifically, the deep features at the end of the backbone network are upsampled and added, then fused with shallow and mid-level features to obtain intermediate features. Simultaneously, the top-down path of FPN is preserved, generating original FPN features P3 and P4. Finally, the fused features are further fused with the original FPN features P3 and P4 to obtain F3 and F4. The feature maps F3 and F4 obtained after dual-path fusion retain the high-resolution spatial information of shallow features while incorporating the strong semantic information of deep features, thus balancing and enhancing hierarchical representation capabilities. CL-FPN injects semantic information more effectively into shallow features across layers while preserving the spatial details of shallow features, enhancing the feature representation of mid- and low-level layers, thereby improving the detection capability of multi-scale targets in complex nearshore scenes.
[0079] The Efficient Local Attention Module (ELA) is as follows:
[0080] Attention mechanisms have gained widespread recognition in computer vision due to their ability to effectively improve the performance of deep neural networks. However, traditional attention mechanisms suffer from several problems: insufficient generalization ability of batch normalization, adverse effects of dimensionality reduction on channel attention, and a relatively complex attention generation process. In contrast, the ELA (Enhanced LocationAttention) module, through its innovative structural design, maintains channel dimensionality integrity and computational efficiency while fully leveraging spatial contextual information, significantly improving the accuracy of deep convolutional neural networks in locating target objects, such as... Figure 5 As shown. The core innovation of this module lies in replacing the traditional spatial global pooling method with a strip pooling strategy, a design inspired by the coordinate attention (CA) mechanism. Specifically, the ELA module performs feature aggregation along the horizontal and vertical directions in the spatial dimension, thereby constructing feature vectors with rich spatial information and effectively modeling the long-distance spatial dependencies of the image. This directional feature extraction mechanism can accurately focus on the target-related region while effectively suppressing the interference of irrelevant background features. For one-dimensional horizontal eigenvectors and Represented as a one-dimensional vertical eigenvector, the expression is:
[0081]
[0082] To further optimize computational efficiency, the module employs one-dimensional convolution to perform lightweight processing on the directional feature vectors, where an adjustable kernel scaling parameter is introduced to control the range of local feature interactions. Subsequently, the feature map is finely adjusted through the combined effect of group normalization and a non-linear activation function, ultimately generating a high-precision positional attention prediction map. The expression is:
[0083] y h +(G n (Q h (Z h )))
[0084] y w =σ(G n (Q w (Z w )))
[0085] Among them, Q h and Q w G represents one-dimensional convolution. n Represented as a group normalized layer, y h and y w These represent the positional attention weights in the horizontal and vertical dimensions, respectively.
[0086] Finally, the positional attention predictions in the horizontal and vertical directions are fused with the original input feature map through a product operation to obtain the enhanced feature map Y.
[0087] Y = F × y h ×y w
[0088] Step 3: Train a small target detection network model based on multi-level feature enhancement and cross-layer fusion.
[0089] Step 4: Inspect small targets on the vessel to be inspected.
[0090] The network is trained by inputting images containing ship targets into it, and outputs the detection results for each bounding rectangle containing the ship targets in the image. Visualization results of some algorithms on a self-built dataset are shown below. Figure 6 As shown.
[0091] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for detecting small target ships based on multi-level attention mechanism and cross-layer fusion, characterized by: Includes the following steps: S1. Construct a remote sensing image data set of ships, covering various background environments, different weather conditions, and common ship types; S2. Construct a remote sensing image ship detection network model based on multi-scale feature enhancement and rotation perception; S3. A remote sensing image ship detection network model based on multi-scale feature enhancement and rotation perception. Step 4: Detect ship targets in the remote sensing image to be detected.
2. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 1, characterized in that: The small target detection network model in step S2 is based on a one-stage detection algorithm and consists of three parts: a backbone network, a neck network, and a detection head. The specific construction process includes the following steps: S21. ResNet18 is used as the backbone network to extract multi-scale high-resolution ship feature maps, which include deep features, mid-level features and shallow features. S22. Introduce a convolutional attention fusion module at the output of the backbone network; S23. Design a cross-level interactive feature reorganization pyramid network as the neck network; S24. An efficient local attention module is embedded at the output of the cross-level interactive feature reorganization pyramid network.
3. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 2, characterized in that: convolution... The attention fusion module adopts a dual-branch parallel structure, including local and global branches. The local branch focuses on extracting local features and spatial adjacency relationships, while the global branch aims to capture long-range dependencies and global contextual information. The specific implementation process includes: Local branches employ multi-level convolution operations combined with channel recombination strategies to enhance cross-channel information interaction while preserving spatial locality. Global branches achieve effective long-range dependency modeling while reducing computational complexity through an improved self-attention mechanism; Finally, the output features of the local and global branches are integrated through an adaptive fusion strategy to form a final feature representation that combines local accuracy and global consistency. The output calculation formula of the CAFM module is: F out =F conv +F att ; Among them, F conv The output, F att This is the output of the global branch.
4. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 3, characterized in that: In the local branch, for the input feature X: First, use a 1×1 convolution Conv 1×1 The channel dimensions are adjusted, and the input tensor is then divided into multiple groups along the channel dimensions. Within each group, a depthwise separable convolution is used to perform channel shuffling. The output tensors of each group are then reconcatenated along the channel dimensions to obtain a new tensor. Finally, a 3×3 convolution is applied. 3×3 Output F conv The expression is: F conv =Conv 3×3 (CS(Conv 1×1 (X))); Here, CS represents the channel shuffling operation.
5. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 3, characterized in that: In the global branch, for the input feature X: First, query Q, key K, and value V are generated using 1×1 convolution and 3×3 depthwise convolution, resulting in three tensors of shape H×W×C; then, query Q is reshaped into Q∈R. HW×C Reshape the key K to K∈R C×HW Then, the attention map A∈R is calculated through the interaction of the reshaped Q and K. C×C The output Fatt of the global branch is defined as: F att =Conv 1×1 Attention(Q,K,V)+X; in, Here, ε represents a learnable scaling parameter, which is used to control the magnitude of the product of the K and Q matrices before applying the softmax function.
6. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 2, characterized in that: The cross-level interactive feature reorganization pyramid network adds a direct feature fusion path from deep to shallow layers to the traditional top-down path of FPN, which efficiently injects high-level semantic information into the shallow layers, thereby suppressing the interference of background noise on small target detection.
7. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 1, characterized in that: The dimerization process of the pyramid network, which reorganizes cross-level interactive features, includes: The deep features at the end of the backbone network are upsampled and added together with the shallow and mid-level features to obtain intermediate features. At the same time, the top-down path of FPN is preserved to generate the original FPN features P3 and P4. Then, the fused features are further fused with the original FPN features P3 and P4 to obtain F3 and F4. The feature maps F3 and F4 obtained after dual-path fusion retain the high-resolution spatial information of shallow features while incorporating the strong semantic information of deep features, thus balancing and enhancing the hierarchical representation capabilities.
8. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 1, characterized in that: The efficient local attention module uses a strip pooling strategy instead of the traditional spatial global pooling method.
9. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 7, characterized in that: The specific implementation process of the efficient local attention module includes the following steps: The efficient local attention module performs feature aggregation along the horizontal and vertical directions in the spatial dimension, effectively modeling the long-distance spatial dependencies of the image. Through a directional feature extraction mechanism, it precisely focuses on the target-related region and suppresses interference from irrelevant background features. The expression is: in, For one-dimensional horizontal eigenvectors and Represented as a one-dimensional vertical eigenvector; To further optimize computational efficiency, the efficient local attention module employs one-dimensional convolution to perform lightweight processing on the directional feature vectors. An adjustable kernel scaling parameter is introduced to control the range of local feature interactions. Then, the feature map is finely adjusted through the combined effect of group normalization and a non-linear activation function, ultimately generating a high-precision location attention prediction map, expressed as: y h =σ(G n (Q h (Z h ))); y w =σ(G n (Q w (Z w ))); Among them, Q h and Q w G represents one-dimensional convolution. n Represented as a group normalization layer, y h and y w These represent the positional attention weights in the horizontal and vertical dimensions, respectively. Finally, the positional attention predictions in the horizontal and vertical directions are fused with the original input feature map through a product operation to obtain the enhanced feature map Y. Y=F×y h ×y w 。 10. The small target ship detection method based on multi-level attention mechanism and cross-layer fusion according to claim 7, characterized in that: The background environment of the dataset in step S1 includes nearshore, offshore and port scenes, weather conditions include fog, rain, clear and night, and common ship types include cargo ships, passenger ships, fishing boats and yachts.