A SAR remote sensing image ship detection method based on a double backbone network

Abstract

The application discloses a SAR remote sensing image ship detection method based on a double-main-stem network, relates to the technical field of remote sensing image processing and target detection, and aims at the problem of low accuracy of existing synthetic aperture radar ship detection. The double-main-stem architecture design in parallel with CNN and the transformer is adopted, the CNN main stem is focused on local detail feature extraction of the ship to resist the blurring influence of coherent speckle noise on the edge, the transformer main stem captures global semantic information to accurately distinguish the ship from the background such as sea clutter and islands, the features are complementary to each other, and thus the accuracy of synthetic aperture radar ship detection is improved.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing and target detection technology, specifically a method for ship detection in SAR remote sensing images based on a dual-backbone network. Background Technology

[0002] Synthetic Aperture Radar (SAR), with its all-weather, day-and-night imaging capabilities, can penetrate adverse weather conditions such as clouds, fog, rain, and snow to acquire high-resolution information about the sea surface, making it a core technology for ship detection in the maritime field. SAR systems actively transmit microwave signals and receive the scattered echoes from targets, accurately reflecting the geometric structure and surface material characteristics of ships. However, its imaging mechanism also introduces inherent drawbacks: speckle noise severely blurs the edge texture of ships, leading to the loss of target details and consequently resulting in the low accuracy of existing synthetic aperture radar for ship detection. Summary of the Invention

[0003] The purpose of this invention is to address the problem of low accuracy in ship detection using existing synthetic aperture radar (SAR) systems by providing a ship detection method based on dual backbone networks from SAR remote sensing images.

[0004] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0005] A method for ship detection in SAR remote sensing images based on a dual-backbone network includes the following steps:

[0006] Acquire SAR remote sensing images containing SAR ship target images, and input the SAR remote sensing images into a multi-mechanism dynamic enhancement dual-backbone ship detection network to obtain the output ship detection results;

[0007] The multi-mechanism dynamic enhancement dual-backbone ship detection network includes a CNN backbone architecture, a Transformer backbone architecture, a neck network, and a prediction end. The CNN backbone architecture is used to extract multi-scale local feature maps, and the Transformer backbone architecture is used to extract multi-scale global feature maps.

[0008] The neck network is used to match and fuse multi-scale local feature maps and multi-scale global feature maps, and to perform multi-scale feature fusion and information transmission through a feature pyramid structure to obtain a multi-scale fused feature map for prediction.

[0009] The prediction end receives multi-scale fused feature maps from the neck network. For each scale of the fused feature map, the prediction end uses a decoupled detection head structure to process the data in parallel through two independent branches. One branch is used to predict the target classification probability, and the other branch is used to regress the coordinates and size of the bounding box. Finally, the outputs of all scales and branches are integrated, and the final ship detection result is obtained through confidence threshold filtering and non-maximum suppression.

[0010] Furthermore, the CNN backbone architecture specifically performs the following steps:

[0011] After receiving the input image, the CNN backbone architecture first performs feature enhancement processing on the input image to obtain the enhanced feature map. Then, the enhanced feature map is input into the CSP structure to obtain multi-scale local feature maps F1, F2, and F3.

[0012] The Transformer backbone architecture specifically executes the following steps:

[0013] After receiving the input image, the Transformer backbone architecture first divides the input image into feature blocks through image block partitioning, and then projects the feature blocks to a high-dimensional feature space through linear embedding to obtain initial embedded features. Subsequently, the initial embedded features are processed by the Swing Transformer to obtain multi-scale global feature maps P1, P2, and P3 that match the scale of the CNN backbone architecture.

[0014] Furthermore, the matching and fusion processing in the neck network is performed through a cross-backbone feature fusion module, which specifically executes the following steps:

[0015] Step 1: Match the multi-scale feature maps F1, F2, F3 with the multi-scale global feature maps P1, P2, P3 to obtain three sets of feature maps:

[0016] Step 2: For each set of feature maps, analyze the local feature maps. and global feature map First, the channel dimensions are unified through 1×1 convolution, and then through a BN layer, the features to be fused for channel matching are obtained. and Features to be fused and Represented as:

[0017] ,

[0018] ,

[0019] in, For 1×1 convolution, For BatchNorm operations;

[0020] Step 3: Based on the features to be fused and The difference characteristics are obtained through bidirectional difference calculation. and Then, the difference features are superimposed with the features to be fused to obtain the enhanced features. and Enhanced features and Represented as:

[0021] ,

[0022] ,

[0023] ,

[0024] ;

[0025] Step 4: Targeting Spatial features are extracted using 3×3 depthwise separable convolutions, and spatial attention weights are obtained by combining them with Sigmoid activation. Finally, the spatial attention weights are used to... By weighting, spatial optimization features are obtained. Spatial optimization features Represented as:

[0026] ,

[0027] ,

[0028] in, It is a 3×3 depthwise separable convolution. Spatial attention weights, Use the Sigmoid activation function;

[0029] against By performing nonlinear transformation of semantic features through convolutional MLP, preliminary semantically optimized features are obtained. and based on Semantic optimization features are obtained by utilizing SE channel attention. Semantic optimization features Represented as:

[0030] ,

[0031] ,

[0032] ,

[0033] ,

[0034] in, and As an intermediate variable, For channel attention weights, It is a fully connected layer. For global average pooling;

[0035] Step 5: Optimize spatial features With semantic optimization features Dynamic fusion is performed to obtain preliminary fusion characteristics. Then, a 3×3 depthwise separable convolution is used to process the initial fusion features to obtain a cross-backbone fusion feature map. Cross-backbone fusion feature map Represented as:

[0036] ,

[0037] ,

[0038] in, A 3×3 depthwise separable convolution;

[0039] Step 6: Repeat steps 2 to 5 to obtain multi-scale cross-backbone fusion feature maps D1, D2, and D3.

[0040] Furthermore, the feature enhancement processing in the CNN backbone architecture is performed through a feature enhancement module that integrates geometric priors. This feature enhancement module includes a multi-scale convolutional branch, a direction-aware gated convolutional branch, a geometric prior masking branch, and a spatial attention mechanism.

[0041] The multi-scale convolutional branch takes the input image received by the CNN backbone architecture and performs 3×3 and 5×5 depth-separable convolutions respectively to obtain multi-scale initial features. and ;

[0042] The orientation-aware gated convolutional branch extracts horizontal, vertical, and diagonal ship edge and structural features from the input image received by the CNN backbone through 1×3, 3×1, and 3×3 parallel directional convolutions, respectively. Combined with a gating mechanism, the horizontal, vertical, and diagonal ship edge and structural features are fused to obtain orientation-enhanced features. ;

[0043] The geometric prior mask branch takes the input image received by the CNN backbone architecture, performs dimensionality reduction and pooling to generate a mask, and then upsamples the mask and multiplies it with the orientation enhancement features to obtain geometric enhancement features. It also incorporates multi-scale initial features. , Directional enhancement features The geometric enhancement features are fused together to obtain a fused feature map;

[0044] The spatial attention mechanism is used to enhance key regions of the fused feature map, resulting in an enhanced feature map.

[0045] Furthermore, the geometric prior mask branch specifically performs the following steps:

[0046] The geometric prior mask branch first processes the input image received by the CNN backbone through 1×1 convolution and grid average pooling to divide the features into... Each grid cell is used to calculate the depth distance between grid cells based on cosine similarity and the spatial distance based on Manhattan distance. Then, the depth and spatial distances are fused using learnable weights to obtain a distance matrix. This distance matrix is ​​then convolved and activated with a sigmoid function to obtain a geometric prior mask. This geometric prior mask is then bilinearly interpolated and multiplied element-wise with the orientation enhancement features to obtain the geometric enhancement features. Then, multi-scale initial features , Directional enhancement features and geometric enhancement features By concatenating along the channel dimension, multi-branch concatenated features are obtained. Then, 1×1 convolution is used to integrate the multi-branch concatenated features through the channels to obtain a fused feature map.

[0047] Furthermore, the depth distance is expressed as:

[0048] ,

[0049] in, For grid cells and The depth distance between them is used to measure the difference in similarity between their feature vectors. , They are grid cells , eigenvectors, For dot product operation, The L2 norm is used to avoid the influence of differences in vector magnitude on similarity calculation.

[0050] Furthermore, the spatial distance is expressed as:

[0051] ,

[0052] in, For grid cells and The spatial distance between them is used to measure the difference in their positions on the feature map. , They are grid cells , Coordinates on the feature map.

[0053] Furthermore, the distance matrix is ​​represented as:

[0054] ,

[0055] in, The fused distance matrix, and These are the learnable weights for depth distance and spatial distance.

[0056] Furthermore, the geometric enhancement features Represented as:

[0057] ,

[0058] ,

[0059] ,

[0060] in, This is the intermediate feature map of the geometric prior mask. and These are convolution operations of 1×1 and 3×3, respectively. This is the geometric prior mask representation after upsampling. Upsample is a bilinear interpolation upsampling operation. , The original feature map resolution is given by ReLU, and ReLU is the activation function.

[0061] The direction enhancement feature Represented as:

[0062] ,

[0063] ,

[0064] ,

[0065] ,

[0066] ,

[0067] ,

[0068] in, , , These are horizontal, vertical, and diagonal edge features, respectively. For feature splicing operations, For 1×1 convolution, It is the Sigmoid activation function. For element-wise multiplication, For gating weights, For gating weights After element-wise weighted processing, the height dimension of the horizontal edge feature map is... For gating weights The width dimension of the vertical edge feature map after element-wise weighted processing. For gating weights The channel dimension of the oblique edge feature map after element-wise weighted processing.

[0069] Furthermore, the spatial attention mechanism specifically performs the following steps:

[0070] First, spatial dependencies of the fused feature map are extracted using 3×3 convolution. Padding is used during the convolution process to keep the feature map size unchanged. Then, after ReLU activation, the number of channels of the feature map is compressed to 1 dimension by 1×1 convolution. Finally, the feature values ​​are mapped to the [0,1] interval by Sigmoid activation to generate a spatial attention weight map. The spatial attention weight map is then multiplied element-wise with the fused feature map to obtain the feature map after feature enhancement.

[0071] The beneficial effects of this invention are:

[0072] This application employs a dual-backbone architecture design that combines CNN and Transformer in parallel. The CNN backbone focuses on extracting local details of the ship to resist the blurring effect of speckle noise on the edges, while the Transformer backbone captures global semantic information to accurately distinguish the ship from background elements such as sea clutter and islands. The two complement each other in terms of features, thereby improving the accuracy of synthetic aperture radar ship detection. Attached Figure Description

[0073] Figure 1 This is a flowchart of the application;

[0074] Figure 2 This is a network structure diagram of this application;

[0075] Figure 3 This is a structural diagram of the feature enhancement module for geometric priors in this application;

[0076] Figure 4 This is a structural diagram of the cross-backbone feature fusion module in this application;

[0077] Figure 5This is a diagram showing the detection results of this application within the entire Gaofen-3 SAR remote sensing image. Detailed Implementation

[0078] It should be noted that, where there is no conflict, the various embodiments disclosed in this application can be combined with each other.

[0079] Specific Implementation Method 1: The SAR remote sensing image ship detection method based on a dual-backbone network described in this implementation method includes the following steps:

[0080] Acquire SAR remote sensing images containing SAR ship target images, and input the SAR remote sensing images into a multi-mechanism dynamic enhancement dual-backbone ship detection network to obtain the output ship detection results;

[0081] The multi-mechanism dynamic enhancement dual-backbone ship detection network includes a CNN backbone architecture, a Transformer backbone architecture, a neck network, and a prediction end. The CNN backbone architecture is used to extract multi-scale local feature maps, and the Transformer backbone architecture is used to extract multi-scale global feature maps.

[0082] The neck network is used to match and fuse multi-scale local feature maps and multi-scale global feature maps, and to perform multi-scale feature fusion and information transmission through a feature pyramid structure to obtain a multi-scale fused feature map for prediction.

[0083] The prediction end receives multi-scale fused feature maps from the neck network. For each scale of the fused feature map, the prediction end uses a decoupled detection head structure to process the data in parallel through two independent branches. One branch is used to predict the target classification probability, and the other branch is used to regress the coordinates and size of the bounding box. Finally, the predicted boxes on all scale feature maps are summarized, and the final ship detection result is obtained through confidence threshold filtering and non-maximum suppression.

[0084] The specific steps described above are as follows:

[0085] Acquire SAR ship dataset and high-resolution SAR remote sensing images. The dataset contains SAR ship target images and corresponding annotation files. Unify the image format to a preset format and set the annotation file format to adapt to the subsequent network training. Divide the dataset into training set, test set and validation set according to a preset ratio.

[0086] The following steps outline how to build a geometric prior feature enhancement module as part of the CNN branch:

[0087] Construct an overall structure for a geometric prior feature enhancement module that includes multi-scale convolutional branches, orientation-aware gated convolutional branches, geometric prior mask branches, and channel attention branches;

[0088] The local feature maps output by the CNN backbone are first fed into multi-scale convolutional branches. Local details and large-scale structural features are extracted through 3×3 and 5×5 depth separable convolutions to obtain multi-scale initial features.

[0089] Multi-scale initial features are fed into the orientation-aware gated convolutional branch, and edge features are extracted through 1×3, 3×1, and 3×3 parallel directional convolutions. Combined with the gating mechanism, orientation-enhanced features are generated.

[0090] The directional enhancement features are fed into the geometric prior mask branch, and a mask is generated through dimensionality reduction, pooling, and distance calculation. After upsampling, the mask is multiplied with the directional enhancement features to highlight the ship target area.

[0091] Features enhanced by geometric priors are fed into the channel attention branch to generate channel weight-adjusted features. Finally, the residuals are fused with the adjusted features and the multi-scale initial features to obtain a preliminary enhanced feature map.

[0092] A cross-backbone feature fusion module is built as part of the neck network construction; the specific steps are as follows:

[0093] After the CNN and Transformer branch features are input into the cross-main feature fusion module, the channel dimension is unified through 1×1 convolution, and normalized features are obtained by combining BatchNorm normalization.

[0094] Perform bidirectional differencing on the normalized features to generate two sets of differential features, which are then added to the corresponding branch normalized features to obtain CNN-enhanced features and Transformer-enhanced features.

[0095] The CNN-enhanced features are processed by 3×3 depthwise separable convolutions and spatial attention weighting, combined with residual connections to obtain spatially optimized features;

[0096] Transformer-enhanced features are transformed by convolutional MLP and weighted by channel attention, then combined with residual connections to obtain semantically optimized features;

[0097] The two types of optimized features are initially fused by element-wise addition, and then refined by depthwise separable convolution to form a cross-scale fused feature map.

[0098] A multi-mechanism dynamic enhancement dual-backbone ship detection network is constructed. This network includes a dual-backbone architecture with parallel CNN and Transformer, a neck network, and a prediction end. The CNN branch integrates a feature enhancement module that fuses geometric priors, and the neck network integrates a cross-backbone feature fusion module.

[0099] The SAR remote sensing images in the training set are input into the dual-backbone ship detection network, and feature extraction is performed through the dual-backbone architecture: the CNN backbone combines the input images with the feature enhancement module based on geometric priors to perform convolution operations, capture local details and edge features of ships, and output multi-scale local feature maps (F1, F2, F3); the Transformer backbone models the global context and long-range dependencies of the images through a self-attention mechanism, and outputs multi-scale global feature maps (P1, P2, P3).

[0100] The local feature maps (F1, F2, F3) and global feature maps (P1, P2, P3) output from the dual-backbone architecture are input into the neck network for sequential feature processing. The channel-optimized feature map is input into the cross-backbone feature fusion module. Through multi-scale convolutional kernels and feature alignment algorithms, the matching and fusion of features at different scales are achieved. The cross-scale fused feature maps (C1, C2, C3) are output to the feature pyramid structure for feature enhancement. The multi-scale fused feature maps (D1, D2, D3) are output to the prediction end respectively.

[0101] The constructed dual-backbone network is trained using the training set, and the network parameters are adjusted using the test set and validation set until the network converges, resulting in a trained network.

[0102] A sliding window detection strategy is used to segment high-resolution SAR remote sensing images. The sliding window side length and overlapping area parameters are set, and the segmented image blocks are sequentially input into the trained network to obtain the ship detection results of each image block. Then, the ship detection results of the complete image are obtained through the result stitching algorithm.

[0103] The geometric prior feature enhancement module of the CNN branch enhances the multi-directional contours and geometric features of ships through multi-scale convolution, orientation-aware gating, and geometric prior masks. The cross-trunk feature fusion module of the neck achieves efficient fusion of two types of heterogeneous features through bidirectional differential modeling and attention weighting, which significantly reduces the false alarm rate of small-scale ships and the background false alarm rate. Ultimately, it achieves high-precision detection of ships of different scales and headings in complex SAR scenarios, providing efficient and reliable technical support for practical applications such as marine monitoring and maritime traffic management.

[0104] Example:

[0105] The RSDD-SAR rotating SAR ship detection dataset was obtained from the internet. This dataset contains 7000 slices, involving 10263 ship instances from multiple sources, with multiple observation modes, multiple polarization modes, and multiple resolutions. The images in the dataset are in JPG format, and the annotation files are in XML format. Relevant images and annotations were selected and divided into training, testing, and validation sets in a 7:2:1 ratio. The divided images and corresponding annotations were then used as the input to the network.

[0106] Reference Figure 2 A feature enhancement module integrating geometric priors is constructed as a core component to enhance local features output from the CNN backbone; the specific steps are as follows:

[0107] We construct a feature enhancement module structure that integrates multi-scale convolution branches, orientation-aware gated convolution branches, geometric prior mask branches, and channel attention branches. Each branch acts in parallel on the local feature map output by the CNN backbone.

[0108] Multi-scale convolution branch: Two depthwise separable convolutions with different receptive fields, 3×3 and 5×5, are used to extract local detail features and large-scale structural features from the input local feature map respectively, to obtain multi-scale initial features;

[0109] Orientation-aware gated convolution branch: Three parallel orientation-aware depthwise convolutions (1×3, 3×1, and 3×3) are used to extract horizontal, vertical, and diagonal ship edge and structural features from the feature map. A gating mechanism is then introduced, and through the fusion of these three directional features and adaptive weight adjustment, orientation-enhanced features are generated to strengthen the representation of the ship's multi-directional contours. The formula is as follows:

[0110] ,

[0111] ,

[0112] ,

[0113] ,

[0114] ,

[0115] ,

[0116] in, , , These are horizontal, vertical, and diagonal edge features, respectively. This indicates a feature concatenation operation. For 1×1 convolution, It is the Sigmoid activation function. This is element-wise multiplication. Gating weights. Based on the intensity of the feature response, the weight of the effective edge of the ship can be adaptively increased, the invalid directional features of noise interference can be suppressed, and directional enhancement features can be generated.

[0117] Geometric prior mask branch: First, perform 1×1 convolutional dimensionality reduction and grid average pooling on the feature map to divide the features into... In this embodiment, there are grid cells. That is, each grid cell corresponds to an 8×8 pixel area of ​​the original image. Feature regions are divided by grid average pooling, and the formula is:

[0118] ,

[0119] ,

[0120] Next, the depth distance between grid cells is calculated based on cosine similarity, and the spatial distance is calculated based on Manhattan distance. The two distances are fused using learnable weights to generate a distance matrix. After convolution and sigmoid activation, a geometric prior mask is generated. The mask is upsampled to the original feature resolution through bilinear interpolation and then multiplied element-wise with the orientation enhancement features to highlight the target area that conforms to the geometric distribution of the ship. The formula and steps are as follows:

[0121] ,

[0122] in, For grid cells and The depth distance between them is used to measure the difference in similarity between their feature vectors. , They are grid cells , eigenvectors, For dot product operation, The L2 norm is used to avoid the influence of differences in vector magnitude on similarity calculation.

[0123] ,

[0124] in, For grid cells and The spatial distance between them is used to measure the difference in their positions on the feature map. , They are grid cells , In the coordinates on the feature map, the denominator is set to avoid calculation errors where the denominator is 0. When two grid cells overlap, the denominator is set to 1.

[0125] Learnable weights , The depth distance and spatial distance are fused, and a mask is generated through 3×3 convolution, ReLU activation, 1×1 convolution, and Sigmoid activation. Then, it is upsampled to the original feature map resolution using bilinear interpolation and multiplied element-wise with the orientation enhancement features to highlight the ship target region. The formula is as follows:

[0126] ,

[0127] ,

[0128] ,

[0129] ,

[0130] in, The fused distance matrix is ​​obtained by fusing depth distance and spatial distance according to weights, and is used to comprehensively measure the characteristics and positional relationships between grid cells. and Learnable weights for depth distance and spatial distance. The intermediate feature map of the geometric prior mask is obtained after convolution and activation function processing, and is used to generate the final mask. and The convolution operations are 1×1 and 3×3, respectively, used to perform feature transformation and channel adjustment on the fused distance matrix. This is the upsampled geometric prior mask representation, upsampled to the original feature map resolution using bilinear interpolation, and used for element-wise operations with the original feature map. `Upsample` is the bilinear interpolation upsampling operation. , The original feature map resolution is used, and ReLU is the activation function used to introduce non-linearity. The enhanced feature map is composed of the original feature map and the upsampled mask. Element-wise multiplication yields feature enhancement based on geometric priors;

[0131] Next, the 3×3 depthwise separable convolutional features output by the multi-scale convolutional branches are processed. ), 5×5 depthwise separable convolutional features ( ), directional enhancement features output by direction-aware gated convolution branches ( ), the upsampled geometric prior mask of the geometric prior mask branch output ( ) Execution of collaborative integration:

[0132] First of all , , , The features are concatenated along the channel dimension to obtain multi-branch concatenated features. Then, a 1×1 convolutional layer is used to integrate the channels of the initial fused features, adjusting the number of channels to the dimension that is compatible with the subsequent CSP structure of the CNN backbone architecture. The 1×1 convolution is used to eliminate the channel redundancy of the multi-branch features, generating a fused feature map with unified dimensions.

[0133] Based on this, a spatial attention mechanism is introduced to enhance key regions of the fused feature map. First, the spatial dependencies of the fused feature map are extracted through a 3×3 convolutional layer. During the convolution process, padding is used to keep the feature map size unchanged. After introducing a nonlinear transformation through the ReLU activation function, the number of channels of the feature map is compressed to 1 dimension through a 1×1 convolutional layer. Finally, the feature values ​​are mapped to the [0,1] interval through the Sigmoid activation function to generate a spatial attention weight map. The spatial attention weight map is then multiplied element-wise with the fused feature map to further highlight the feature response of the spatial region where the ship target is located, while suppressing interference from background regions such as sea clutter and land.

[0134] Finally, residual connections are constructed to ensure the integrity of feature information. The feature map processed by the spatial attention mechanism is added element-wise to the original input feature (denoted as X) of the feature enhancement module that incorporates geometric priors. This residual connection not only preserves the ship's basic structural information in the original input feature X, but also alleviates the feature degradation problem that may occur during multi-branch fusion and spatial attention processing through the residual path. The final output is an enhanced feature map that combines multi-scale details, multi-directional edges, geometric prior constraints and spatial focusing characteristics, providing highly discriminative input for the CSP structure of the subsequent CNN backbone for multi-scale local feature extraction.

[0135] Reference Figure 2 The following are the specific steps to construct a multi-mechanism, dynamically enhanced dual-backbone ship detection network:

[0136] After receiving the input image, the backbone network first performs feature enhancement processing through a geometric prior feature enhancement module. Subsequently, the feature map enhanced by the geometric prior feature enhancement module is fed into a series of CSP structures consisting of the C3K2 module, C2PSA module, BottleneckBlock, and SPPF structures to complete basic feature extraction and encoding. The C3K2 module balances shallow detail preservation and deep feature aggregation through multi-path branching and 3×3 small kernel convolution. The C2PSA module combines the CSP structure with multi-scale spatial attention to enhance the response of key regions. With progressive downsampling, the final output is multi-scale feature maps F1, F2, and F3 after 8x, 16x, and 32x downsampling, with 128, 256, and 512 channels, respectively, providing a foundation for subsequent local feature correlation processing.

[0137] After receiving the input image, the second backbone network first transforms the image into a sequence of feature blocks through the PatchPartition operation. Then, it projects the feature blocks into a high-dimensional feature space through Linear Embedding to obtain the initial embedded features. Subsequently, the initial embedded features pass through four Stage structures, each containing a Swing Transformer Block and Patch Merging (except for the last Stage). The Swing Transformer Block models the global context and long-distance dependencies through dynamic window self-attention and enhances spatial awareness by combining relative position bias. The Patch Merging module achieves feature downsampling and channel dimension enhancement by splicing adjacent feature blocks. Finally, it obtains 8x, 16x, and 32x downsampled feature maps P1, P2, and P3, which match the scale of the first backbone network, providing support for subsequent global feature optimization and cross-backbone fusion.

[0138] Reference Figure 5 A cross-branch feature fusion module is built as the core component for cross-branch fusion of dual-branch output features. The specific steps are as follows:

[0139] Feature preprocessing: Receive feature maps F1 / F2 / F3 output from backbone network one and feature maps P1 / P2 / P3 output from backbone network two. First, unify the channel dimensions of the two types of features through a 1×1 convolutional layer. Then, combine the BatchNorm operation to achieve feature normalization, obtaining the channel-matched features to be fused. The formula is:

[0140] ,

[0141] ,

[0142] in, It is a 1×1 convolution used for channel dimension adjustment. The BatchNorm operation ensures that the two types of features are fused within the same distribution space.

[0143] To address the heterogeneity between CNN local features (strong spatial details, weak global correlations) and Transformer global features (strong global correlations, weak spatial details), bidirectional difference calculation is used to capture the complementary information of the two types of features. Then, the difference features are superimposed with the original features to generate enhanced features, as shown in the formula:

[0144] ,

[0145] ,

[0146] ,

[0147] ,

[0148] in, Extracting local details from CNN features that are superior to those from Transformer Extracting global semantics from Transformer features that are superior to those of CNN, and then adding them to the original normalized features, makes... (CNN-enhanced features) Preserve local details while incorporating global relationships. (Transformer-enhanced features) preserve global semantics while supplementing spatial details.

[0149] Based on the core advantages of the two types of enhanced features, spatial attention and channel attention are used for targeted optimization to strengthen ship target features and suppress background interference. Leveraging its spatial detail advantage, the feature map's spatial structure is refined through 3×3 depthwise separable convolutions. This is combined with Sigmoid activation to generate spatial attention weights, which are then adaptively weighted based on the spatial location of the feature map. The formula is as follows:

[0150] ,

[0151] ,

[0152] in, This uses a 3×3 depthwise separable convolution, which enhances spatial features while reducing computational cost. This represents the spatial attention weight; a higher weight indicates a greater probability that the corresponding spatial location is a ship target. The spatial optimization feature can accurately locate the area where the ship is located and suppress background spatial noise.

[0153] against Leveraging its global semantic advantages, a convolutional MLP is used to achieve non-linear transformation of semantic features, followed by the introduction of a channel attention module to adjust channel importance. The formula is as follows:

[0154] Semantic transformation of convolutional MLP:

[0155] ,

[0156] Channel attention weight generation:

[0157] ,

[0158] ,

[0159] Attention-weighted and residual fusion:

[0160] ,

[0161] in, This is global average pooling (compressing channel features into a single value to capture global information). It is a fully connected layer. For channel attention weights, priority is given to strengthening channels related to ship semantics (such as channels corresponding to hull structure and heading features). This is a semantically optimized feature that effectively weakens semantic interference from background elements such as sea clutter and islands.

[0162] Space optimization features With semantic optimization features To achieve dynamic fusion and compress redundant information while improving feature discriminative power, the advantages of the two types of features are first integrated by adding them element by element, thus achieving complementarity between "local space + global semantics". The formula is as follows:

[0163] ,

[0164] Then, a 3×3 depthwise separable convolution is used to refine the initial fused features, filtering out duplicate background and noise features and compressing feature redundancy. The formula is as follows:

[0165] ,

[0166] in, The refined cross-scale fusion feature map retains both the local ship details of the CNN branch and the global semantic associations of the Transformer branch.

[0167] Residual connection enhancement: The initial features to be fused are added to the refined fused features through residual edges, preserving the original feature information, avoiding the loss of key features during the fusion process, and enhancing gradient flow. The formula is as follows:

[0168] ,

[0169] Final output feature map The feature scales of the input cross-core feature fusion module correspond one-to-one. Subsequently, the input feature pyramid structure will be used to generate feature maps (D1, D2, D3) that are more suitable for multi-scale ship detection through top-down feature transfer and bottom-up feature enhancement, providing highly robust feature inputs for the prediction end.

[0170] A deep learning network consisting of a dual backbone network, a feature pyramid constructed by a neck cross-backbone feature fusion module, and a prediction end was trained using the RSDD-SAR ship dataset. During training, the scale of the input image was set to 640×640, the epoch was 150, the optimizer used stochastic gradient descent (SGD), the initial learning rate was set to 0.01, and a cosine annealing learning strategy was used to dynamically adjust the learning rate so that the training set loss and the validation set loss tended to be balanced, thus obtaining the trained network model.

[0171] The trained network was used to detect L1A-level full-frame Gaofen-3 (GF-3) SAR remote sensing images with a collection area in the Strait of Malacca region and a polarization mode of HH. During the detection process, an adaptive sliding window strategy was first adopted:

[0172] The sliding window is set to a side length of 1200 pixels and an overlap area of ​​300 pixels. The slices are traversed along the image's longitudinal and transverse directions to prevent the ship target from being segmented by the window; background filtering is also introduced.

[0173] A grayscale mask is generated using the Kittler binarization thresholding algorithm to filter out water background areas with grayscale values ​​below the threshold, retaining only slices with a foreground proportion greater than 5% as input to the network. Finally, after stitching the detection results, quadratic nonmaximum suppression is used to eliminate duplicate detection boxes in overlapping areas and remove false alarms, thus obtaining accurate ship target detection results.

[0174] It should be noted that the specific embodiments are merely explanations and illustrations of the technical solution of the present invention and should not be used to limit the scope of protection. Any modifications made in accordance with the claims and specification of the present invention that are only partial should still fall within the protection scope of the present invention.

Claims

1. A method for ship detection in SAR remote sensing images based on a dual-backbone network, characterized in that... Includes the following steps: Acquire SAR remote sensing images containing SAR ship target images, and input the SAR remote sensing images into a multi-mechanism dynamic enhancement dual-backbone ship detection network to obtain the output ship detection results; The multi-mechanism dynamic enhancement dual-backbone ship detection network includes a CNN backbone architecture, a Transformer backbone architecture, a neck network, and a prediction end. The CNN backbone architecture is used to extract multi-scale local feature maps, and the Transformer backbone architecture is used to extract multi-scale global feature maps. The neck network is used to match and fuse multi-scale local feature maps and multi-scale global feature maps, and to perform multi-scale feature fusion and information transmission through a feature pyramid structure to obtain a multi-scale fused feature map for prediction. The prediction end receives the multi-scale fused feature map of the neck network. For each scale of the fused feature map, the prediction end adopts a decoupled detection head structure to process it in parallel through two independent branches. One branch is used to predict the target classification probability, and the other branch is used to regress the coordinates and size of the bounding box. Finally, the outputs of all scales and branches are integrated, and the final ship detection result is obtained through confidence threshold filtering and non-maximum suppression. The CNN backbone architecture specifically executes the following steps: After receiving the input image, the CNN backbone architecture first performs feature enhancement processing on the input image to obtain the enhanced feature map. Then, the enhanced feature map is input into the CSP structure to obtain multi-scale local feature maps F1, F2, and F3. The Transformer backbone architecture specifically executes the following steps: After receiving the input image, the Transformer backbone architecture first divides the input image into feature blocks through image block partitioning, and then projects the feature blocks to a high-dimensional feature space through linear embedding to obtain initial embedded features. Subsequently, the initial embedded features are processed by the Swing Transformer to obtain multi-scale global feature maps P1, P2, and P3 that match the scale of the CNN backbone architecture. The matching and fusion processing in the neck network is performed through a cross-backbone feature fusion module, which specifically executes the following steps: Step 1: Match the multi-scale feature maps F1, F2, F3 with the multi-scale global feature maps P1, P2, P3 to obtain three sets of feature maps: Step 2: For each set of feature maps, analyze the local feature maps. and global feature map First, the channel dimensions are unified through 1×1 convolution, and then through a BN layer, the features to be fused for channel matching are obtained. and Features to be fused and Represented as: ， ， in, For 1×1 convolution, For BatchNorm operations; Step 3: Based on the features to be fused and The difference characteristics are obtained through bidirectional difference calculation. and Then, the difference features are superimposed with the features to be fused to obtain the enhanced features. and Enhanced features and Represented as: ， ， ， ； Step 4: Targeting Spatial features are extracted using 3×3 depthwise separable convolutions, and spatial attention weights are obtained by combining them with Sigmoid activation. Finally, the spatial attention weights are used to... By weighting, spatial optimization features are obtained. Spatial optimization features Represented as: ， ， in, It is a 3×3 depthwise separable convolution. Spatial attention weights, Use the Sigmoid activation function; against By performing nonlinear transformation of semantic features through convolutional MLP, preliminary semantically optimized features are obtained. and based on Using channel attention, semantically optimized features are obtained. Semantic optimization features Represented as: ， ， ， ， in, and As an intermediate variable, For channel attention weights, It is a fully connected layer. For global average pooling; Step 5: Optimize spatial features With semantic optimization features Dynamic fusion is performed to obtain preliminary fusion characteristics. Then, a 3×3 depthwise separable convolution is used to process the initial fusion features to obtain a cross-backbone fusion feature map. Cross-backbone fusion feature map Represented as: ， ， in, A 3×3 depthwise separable convolution; Step 6: Repeat steps 2 to 5 to obtain multi-scale cross-backbone fusion feature maps D1, D2, and D3.

2. The method for ship detection in SAR remote sensing images based on a dual-backbone network according to claim 1, characterized in that... The feature enhancement processing in the CNN backbone architecture is performed through a feature enhancement module that integrates geometric priors. This feature enhancement module includes a multi-scale convolutional branch, a direction-aware gated convolutional branch, a geometric prior masking branch, and a spatial attention mechanism. The multi-scale convolutional branch takes the input image received by the CNN backbone architecture and performs 3×3 and 5×5 depth-separable convolutions respectively to obtain multi-scale initial features. and ; The orientation-aware gated convolutional branch extracts horizontal, vertical, and diagonal ship edge and structural features from the input image received by the CNN backbone through 1×3, 3×1, and 3×3 parallel directional convolutions, respectively. Combined with a gating mechanism, the horizontal, vertical, and diagonal ship edge and structural features are fused to obtain orientation-enhanced features. ; The geometric prior mask branch takes the input image received by the CNN backbone architecture, performs dimensionality reduction and pooling to generate a mask, and then upsamples the mask and multiplies it with the orientation enhancement features to obtain geometric enhancement features. It also incorporates multi-scale initial features. , Directional enhancement features The geometric enhancement features are fused together to obtain a fused feature map; The spatial attention mechanism is used to enhance key regions of the fused feature map, resulting in an enhanced feature map.

3. The method for ship detection in SAR remote sensing images based on a dual-backbone network according to claim 2, characterized in that... The geometric prior mask branch specifically executes the following steps: The geometric prior mask branch first processes the input image received by the CNN backbone through 1×1 convolution and grid average pooling to divide the features into... Each grid cell is used to calculate the depth distance between grid cells based on cosine similarity and the spatial distance based on Manhattan distance. Then, the depth and spatial distances are fused using learnable weights to obtain a distance matrix. This distance matrix is ​​then convolved and activated with a sigmoid function to obtain a geometric prior mask. This geometric prior mask is then bilinearly interpolated and multiplied element-wise with the orientation enhancement features to obtain the geometric enhancement features. Then, multi-scale initial features , Directional enhancement features and geometric enhancement features By concatenating along the channel dimension, multi-branch concatenated features are obtained. Then, 1×1 convolution is used to integrate the multi-branch concatenated features through the channels to obtain a fused feature map.

4. The method for ship detection in SAR remote sensing images based on a dual-backbone network according to claim 3, characterized in that... The depth distance is represented as: ， in, For grid cells and The depth distance between them is used to measure the difference in similarity between their feature vectors. , They are grid cells , eigenvectors, For dot product operation, The L2 norm is used to avoid the influence of differences in vector magnitude on similarity calculation.

5. The method for ship detection in SAR remote sensing images based on a dual-backbone network according to claim 4, characterized in that... The spatial distance is expressed as: ， in, For grid cells and The spatial distance between them is used to measure the difference in their positions on the feature map. , They are grid cells , Coordinates on the feature map.

6. The method for ship detection in SAR remote sensing images based on a dual-backbone network according to claim 5, characterized in that... The distance matrix is ​​represented as follows: ， in, The fused distance matrix, and These are the learnable weights for depth distance and spatial distance.

7. The method for ship detection in SAR remote sensing images based on a dual-backbone network according to claim 6, characterized in that... The geometric enhancement features Represented as: ， ， ， in, This is the intermediate feature map of the geometric prior mask. and These are convolution operations of 1×1 and 3×3, respectively. This is the geometric prior mask representation after upsampling. Upsample is a bilinear interpolation upsampling operation. , The original feature map resolution is given by ReLU, and ReLU is the activation function. The direction enhancement feature Represented as: ， ， ， ， ， ， in, , , These are horizontal, vertical, and diagonal edge features, respectively. For feature splicing operations, For 1×1 convolution, It is the Sigmoid activation function. For element-wise multiplication, For gating weights, For gating weights After element-wise weighted processing, the height dimension of the horizontal edge feature map is... For gating weights The width dimension of the vertical edge feature map after element-wise weighted processing. For gating weights The channel dimension of the oblique edge feature map after element-wise weighted processing.

8. The method for ship detection in SAR remote sensing images based on a dual-backbone network according to claim 7, characterized in that... The spatial attention mechanism specifically performs the following steps: First, spatial dependencies of the fused feature map are extracted using 3×3 convolution. Padding is used during the convolution process to keep the feature map size unchanged. Then, after ReLU activation, the number of channels of the feature map is compressed to 1 dimension by 1×1 convolution. Finally, the feature values ​​are mapped to the [0,1] interval by Sigmoid activation to generate a spatial attention weight map. The spatial attention weight map is then multiplied element-wise with the fused feature map to obtain the feature map after feature enhancement.

Images