A sar image overlay identification method with attention mechanism

Abstract

The application discloses a SAR image shadow identification method with an attention mechanism, comprising the following steps: acquiring a SAR image and performing pretreatment; then inputting a parallel CNN encoder and a ViT encoder respectively, and extracting multi-scale features; using a bidirectional feature fusion module to perform deep fusion on the multi-scale features; then inputting a decoder, recovering the feature resolution through step-by-step upsampling, using an attention gate module and an attention residual module to optimize the features in the decoding process, and introducing a convolution block attention module in the fusion stage of the decoder path to enhance spatial context information; performing deep supervision on lateral outputs at different scales, using a weighted loss function to train, and generating shadow region identification results. The application can adaptively learn geometric context information from SAR image data, eliminate the dependence on high-precision DEMs, and effectively solve the problems of boundary blur, under-segmentation and noise sensitivity.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing and computer vision technology, specifically a SAR image overlay recognition method with an attention mechanism. Background Technology

[0002] Synthetic Aperture Radar (SAR) can acquire surface information under all-weather, day-and-night conditions through active microwave side-looking imaging, thus possessing significant application value in target identification, topographic mapping, and disaster monitoring. Unlike the central projection imaging mechanism of optical remote sensing, SAR is based on the side-looking radar echo ranging principle, mapping three-dimensional targets on the surface onto the slant range plane. This range- and Doppler-based imaging geometry makes the geometric features of SAR images strongly correlated with the radar incident angle and local terrain slope, inevitably causing geometric distortions such as perspective contraction, overlay, and shadows in undulating terrain.

[0003] Overlay can lead to phenomena such as compression of spatial distance and location, disruption of terrain texture continuity, and spatial overlap, thereby reducing the interpretability of SAR images and the accuracy of target detection, especially in areas with large terrain undulations such as mountains and high-rise buildings.

[0004] From a radiometric perspective, overlay causes backscattered signals from multiple ground objects to be compressed in the range direction and superimposed into the same resolution cell. This results in a significant increase in the equivalent scattering contribution within the cell, manifesting as an increase in intensity values ​​and the formation of bright bands or patches. Since the overlapping signals in the overlay area are physically mixed and irreversible, the most effective strategy to mitigate geometric distortion is to accurately identify the overlay area. Accurate identification is a fundamental prerequisite for preventing data distortion from contaminating downstream tasks (such as land cover mapping and multi-track data fusion).

[0005] To address the problem of overlap detection, researchers both domestically and internationally have proposed various traditional methods, including the RD model method, threshold segmentation method, local frequency estimation method, eigenvalue decomposition method, and multi-feature joint detection method. However, these methods are highly dependent on the quality of external digital elevation models (DEMs) and accurate orbital parameters. When high-resolution DEMs are outdated or unavailable, traditional geometric simulations will fail. Furthermore, traditional methods are sensitive to random speckle noise, making it difficult to achieve high-precision automatic detection and fine segmentation.

[0006] With the development of deep learning technology, Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs) have made significant progress in remote sensing image segmentation. CNNs can extract local spatial features, while ViTs utilize self-attention mechanisms to capture global contextual information. The combination of the two provides a new solution for SAR overlay recognition. However, when existing deep learning models are directly applied to SAR image overlay recognition, there are still obvious shortcomings: First, general models often fail to capture the long-distance structural continuity of ridges, resulting in undersegmentation of spatially broad overlay areas; second, due to the lack of sufficient multi-scale feature integration to consider SAR physical characteristics, the boundaries between overlay areas and other strong scatterers are often severely blurred; third, the model has a weak ability to represent low-reflection areas in complex terrain, making it susceptible to speckle noise interference and leading to unstable recognition results. Summary of the Invention

[0007] In view of the problems existing in the prior art, the purpose of this invention is to provide a SAR image overlay recognition method with an attention mechanism, which can adaptively learn geometric context information from SAR image data, eliminate dependence on high-precision DEMs, and effectively solve the problems of boundary ambiguity, undersegmentation and noise sensitivity in SAR overlay recognition.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A SAR image overlay recognition method with an attention mechanism includes the following steps: Step S1: Acquire the SAR image to be recognized and preprocess it; Step S2: Input the preprocessed SAR image into a parallel CNN encoder and a ViT encoder respectively to extract multi-scale features; Step S3: Use a bidirectional feature fusion module to perform deep fusion of the heterogeneous multi-scale features extracted by the CNN encoder and the ViT encoder to obtain a comprehensive feature map; wherein, the bidirectional feature fusion module performs different attention enhancement processing on the features from the ViT encoder and the features from the CNN encoder respectively, and then fuses the processed features; Step S4: Input the comprehensive feature map into the decoder, restore the feature resolution by upsampling step by step, and use an attention gating module and an attention residual module to optimize the features during the decoding process. At the same time, a convolutional block attention module is introduced in the fusion stage of the decoder path to enhance the spatial context information, resulting in an optimized feature map; Step S5: Perform deep supervision based on the lateral output of the decoder at different scales, and use a weighted loss function for training to generate the final overlay region recognition result.

[0010] As a further improvement to the above technical solution:

[0011] In step S2, the CNN encoder uses the EfficientNet-B3 network, the ViT encoder uses the SwinTransformer network, and the multi-scale features include feature maps of four scales with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image resolution, respectively.

[0012] The bidirectional feature fusion module in step S3 includes a parallel Transformer branch head and a CNN branch head. The Transformer branch head is used to perform channel attention recalibration on the feature map output by the ViT encoder, and the CNN branch head is used to perform spatial geometric adaptation enhancement on the feature map output by the CNN encoder. The features processed by the Transformer branch head and the features processed by the CNN branch head are concatenated and enhanced by the attention residual module to obtain the comprehensive feature map.

[0013] The Transformer branch head performs channel attention recalibration on the feature map output by the ViT encoder. Specifically, this includes: performing average pooling and max pooling operations on the feature map output by the ViT encoder, inputting the pooling results into depthwise separable convolutional layers with different dilation rates to generate attention weights, performing ReLU activation, depthwise separable convolution, and Sigmoid activation on the attention weights in sequence, and multiplying the activation results with the original input feature map to obtain the recalibrated features.

[0014] The CNN branch head performs spatial geometric adaptation enhancement on the feature map output by the CNN encoder. Specifically, it performs 3×3 deformable convolution and spatial and channel compression activation operations on the feature map output by the CNN encoder in sequence, followed by depthwise separable convolution and Sigmoid activation. The activation result is multiplied with the original input feature map to obtain the enhanced features.

[0015] The attention gating module in step S4 takes deep decoding features and shallow coding features as input and performs the following operations: inputting the deep decoding features and shallow coding features into depthwise separable convolutional layers with different dilation rates, and performing batch normalization processing; adding the two processed features; performing ReLU activation; and then sequentially passing them through at least one CB layer composed of a 1×1 convolutional layer and a batch normalization layer and a Sigmoid activation function to generate a spatial attention map; and multiplying the spatial attention map element-wise with the shallow coding features to obtain the gated enhanced features.

[0016] In step S4, the attention residual module takes the input features as the processing object and performs the following operations: performs a first convolution processing on the input features and a second convolution processing containing at least two convolutional layers, batch normalization, and ReLU activation. After adding the results of the two convolution processing, the input space and channel squeezing excitation module is used to obtain the enhanced output features.

[0017] In step S4, the convolutional block attention module performs the following operations on the input features: performs average pooling and max pooling on the input features respectively; concatenates the average pooling results and max pooling results along the channel dimension; inputs the concatenated result into the convolutional layer and activates it with the Sigmoid function to generate a spatial attention weight map; and multiplies the spatial attention weight map element-wise with the input features to obtain the spatially enhanced features.

[0018] The weighted loss function mentioned in step S5 is the sum of the weighted smoothed cross-entropy loss and the Dice loss. The weighted smoothed cross-entropy loss introduces a label smoothing strategy, which smooths the real labels through a smoothing factor. The Dice loss is used to measure the regional overlap between the predicted result and the real label.

[0019] Step S5, which involves performing deep supervision based on lateral output, includes setting different loss weights for the lateral output layers of the decoder at different scales, wherein the weight of the deep output is greater than the weight of the shallow output, and then weighting and fusing the losses of each layer to obtain the total loss for training.

[0020] The beneficial effects of this invention are:

[0021] (1) This invention achieves deep complementarity between local texture features and global contextual information by constructing a CNN-ViT dual-branch architecture. It can achieve high-precision automatic identification of SAR overlay regions without relying on external digital elevation models (DEMs), eliminating the dependence of traditional geometric methods on high-precision DEMs and orbital parameters, and providing a robust technical means to mitigate the impact of geometric distortion in complex terrain. The CNN branch extracts fine-grained backscattered texture, while the ViT branch effectively simulates the long-range geometric dependencies of terrain features.

[0022] (2) Through the bidirectional feature fusion module (BiFFM), the multi-scale features of the heterogeneous encoder are effectively integrated. Deformable convolution is used to enhance the model’s adaptability to the geometric features of SAR images, solve the semantic alignment problem between CNN and ViT features, and significantly improve the effectiveness of feature fusion.

[0023] (3) By introducing the Attention Gated Module (AGM) and the Attention Residual Module (ARM), the cross-layer semantic association and intra-channel detail recognition capabilities are enhanced, effectively improving the model's ability to capture overlay boundaries, alleviating the confusion between shadows and overlay regions, and improving segmentation accuracy and boundary consistency. The collaborative design of the AGM and ARM modules gives the invention stronger adaptability in the feature fusion stage: the AGM is responsible for enhancing cross-layer semantic association, while the ARM focuses on strengthening the intra-channel detail recognition capability. The combination of the two effectively improves the overlay recognition performance and enhances the accuracy and boundary consistency of SAR image segmentation.

[0024] (4) By embedding a Convolutional Block Attention (CBAM) module in the decoder path, the learning of spatial context information during multi-scale feature fusion is further enhanced, expanding the effective recognition range of the model and alleviating the problem of undersegmentation or incomplete recognition caused by insufficient learning of the edges and adjacent features of the overlapping region. On the one hand, it enables the model to focus on the significant edge features of the overlapping region, alleviating the problems of boundary blurring and undersegmentation; on the other hand, through the spatial attention weight map generated by convolution and the Sigmoid function, the model can capture contextual relationship information within a larger receptive field, thereby expanding the recognition range and improving the spatial perception ability of the model, making the present invention exhibit better robustness and accuracy in target recognition and boundary extraction tasks under complex terrain.

[0025] (5) The comprehensive experimental results show that the F1 score of the present invention on the Gaofen-3 (GF-3) satellite dataset is 0.8962, and the average intersection-union ratio (MIoU) is significantly improved compared with the existing mainstream methods. This verifies the superiority of the CNN and Transformer fusion paradigm in high-precision identification of SAR image overlay, and lays a solid technical foundation for subsequent high-precision target interpretation and reliable information extraction in mountainous areas. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the process of the present invention.

[0027] Figure 2 This is a structural diagram of the BiFFM module of the present invention.

[0028] Figure 3 This is a structural diagram of the AGM module of the present invention.

[0029] Figure 4 This is a structural diagram of the ARM module of the present invention.

[0030] Figure 5This is a schematic diagram of thermal features according to an embodiment of the present invention. In the figure, (a1), (a2), and (a3) ​​are SAR images, (b1), (b2), and (b3) are feature maps before input to AGM, (c1), (c2), and (c3) are feature maps after AGM, and (d1), (d2), and (d3) are feature maps after ARM.

[0031] Figure 6 This is a structural diagram of the CBAM module of the present invention.

[0032] Figure 7 This is a schematic diagram of the experimental visualization results of an embodiment of the present invention. In the figure, (a1), (a2), (a3), and (a4) are SAR images, (b1), (b2), (b3), and (b4) are the corresponding labels of the SAR images, and (c1), (c2), (c3), and (c4) are the test results of the present invention.

[0033] Figure 8 This is a comparison chart of the accuracy of different methods.

[0034] Figure 9 This is a comparison chart of IoU for different methods.

[0035] Figure 10 A visualization comparison of different methods for identifying overlays is provided. In the figure, (a1), (a2), (a3), (a4), (a5), and (a6) are SAR images; (b1), (b2), (b3), (b4), (b5), and (b6) are ground truth labels; (c1), (c2), (c3), (c4), (c5), and (c6) are PSPNet prediction results; (d1), (d2), (d3), (d4), (d5), and (d6) are Deeplabv3+ prediction results; (e1) (e1), (e2), (e3), (e4), (e5), and (e6) are the prediction results of CLCFormer; (f1), (f2), (f3), (f4), (f5), and (f6) are the prediction results of this invention; (g1), (g2), (g3), (g4), (g5), and (g6) are the distribution maps comparing the prediction results of this invention with the true values; and (h1), (h2), (h3), (h4), (h5), and (h6) are visualization maps of overlaying the prediction results of this invention onto the corresponding SAR images.

[0036] Figure 11 This is a comparison chart of ablation tests. Detailed Implementation

[0037] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0038] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0039] A SAR image overlay recognition method with an attention mechanism, the overall process of which is as follows: Figure 1 As shown, this method addresses the problems of noise interference, blurred boundaries, and undersegmentation in overlapping region identification. It systematically integrates the advantages of convolutional neural networks (CNN) and visual transformers (ViT), adopts a hybrid dual encoder-decoder structure, and is supplemented by multi-level attention mechanisms and feature fusion strategies to effectively achieve accurate extraction of overlapping regions in SAR images.

[0040] The identification method includes the following steps:

[0041] Step S1: Acquire the SAR image to be identified and perform preprocessing.

[0042] In this embodiment, the raw data comes from the Gaofen-3 C-band Synthetic Aperture Radar satellite, with an imaging mode of Fine Strip II, a spatial resolution of 10 meters, and a single polarization HH. In this embodiment, the data source area is selected in the western mountainous region of Hubei Province, China, an area with dramatic surface undulations and numerous typical overlay phenomena caused by topographic distortion.

[0043] The acquired wide-swath SAR imagery is cropped to obtain several non-overlapping image patches of 512×512 pixels as images to be identified. To broaden the training sample distribution and prevent model overfitting, an online data augmentation strategy is employed during the training phase, including random flipping and rotation of the image patches, which significantly expands the final equivalent sample distribution used for training.

[0044] In this embodiment, the labeling employs a meticulous manual annotation strategy, strictly referencing optical imagery and digital elevation models (DEMs) to eliminate interpretation ambiguities. Based on terrain geometry and scattering characteristics, pixel semantics are categorized into three types: Background areas are flat regions with relatively stable echo intensity; Mountains are steep slopes with exposed or vegetated terrain where signals do not overlap; and Layover areas are bright, distorted regions where signals from the front and back slopes overlap due to the radar incident angle being smaller than the local slope angle. Finally, the dataset is randomly divided into training and validation sets in a 3:1 ratio to ensure the model can fully learn the feature distribution from a massive amount of samples.

[0045] Step S2: Input the preprocessed SAR image into a dual-channel parallel encoder to extract multi-scale and multi-granular features.

[0046] This step employs a dual encoder-decoder architecture, using parallel CNN encoder branches and ViT encoder branches to comprehensively capture the rich features in SAR images.

[0047] Specifically, the first encoder branch is a CNN encoder based on EfficientNet-B3, which excels at extracting local details and is used to extract local spatial detail features of SAR images. The second encoder branch is a ViT encoder based on Swin Transformer, used to capture global contextual information of SAR images. This complementary combination can provide multi-scale, multi-granularity feature representations. In this invention, the dual-path parallel encoder provides feature maps at four scales, corresponding to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image resolution.

[0048] The ViT encoder uses a pre-trained, popular SwinV2 backbone. It first partitions the image into non-overlapping patches (i.e., markers), then uses a linear embedding layer to project the features of each patch into a preset dimension C, which is 96 in this embodiment. Next, SwinV2 blocks based on windowed multi-head self-attention and shifted W-MSA are used to extract global contextual features. As the network depth increases, the patch merging layer gradually reduces the patch resolution. For the input image, we obtain feature maps at four levels.

[0049] The CNN encoder uses a pre-trained robust and lightweight EfficientNet-B3 backbone. It first extracts shallow features using 3x3 convolutions (Conv), followed by batch normalization (BN) and a sigmoid-weighted linear unit (SiLU) function. Subsequently, a series of moving inverted bottleneck convolutions (MBConv) blocks are used to capture multi-level spatial details. In this way, the CNN-based encoder reduces the number of parameters while preserving fine-grained spatial details. Similarly, we obtain four feature maps at different levels.

[0050] Step S3: Use the Bidirectional Feature Fusion Module (BiFFM) to perform deep fusion of the heterogeneous multi-scale features extracted by the dual encoder.

[0051] Due to the limitations of convolutional operations, CNN-based encoders can only extract local features, while ViT encoders can effectively extract global contextual information, but they are insufficient in capturing detailed spatial features. To effectively extract global contextual information and further capture detailed spatial features, this invention designs a BiFFM module to deeply cross-fuse complementary features obtained from dual encoders. By combining local details and global contextual information, spatial resolution is restored and the segmentation results are refined. This addresses the complex geometric characteristics of overlapping regions in SAR images, reduces noise interference during overlapping recognition, and improves the ability to identify overlapping regions.

[0052] During the encoder-decoder integration phase, a bidirectional feature fusion module (BiFFM) is used to deeply fuse multi-scale features from the two heterogeneous encoders. The specific structure of this module can be found in [link to documentation]. Figure 2 Among them, 3×3 deformable convolution is used to enhance geometric adaptability, allowing the sampling position of the convolution kernel to dynamically learn the offset according to the content of the input feature map, and the feature discriminative power is enhanced through the Spatial and Channel Squeeze Excitation (scSE) module.

[0053] Specifically, assuming and These are the feature maps obtained from the nth layer ViT encoder and the CNN encoder, respectively. Figure 1 and 2 As shown, BiFFM contains two parallel branches: TBH (Transformer Branch Head) is used to process ViT encoder features. CBH (CNN Branch Head) is used to process features from the CNN encoder. .

[0054] For the feature map output by the ViT encoder The channel features are obtained using average pooling and max pooling operations, and the results are fed into depthwise separable convolutional layers (DWConv) with different dilation rates to generate attention weights. The process can be described by formula (1):

[0055]

[0056] in, This indicates a depthwise separable convolution operation. This indicates the average pooling operation. This indicates a max pooling operation.

[0057] Then, the convolutional pooling results are activated through a series of nonlinear operations, and the activation results are then combined with... Multiply to further refine the characteristics of each channel, and the result is denoted as... The process is shown in formula (2):

[0058]

[0059] ReLU and Sigmoid are both activation functions used to add nonlinear capabilities to neural networks.

[0060] For the features acquired by the CNN encoder, a 3×3 deformable convolution (DFConv) is used to adaptively capture the geometric features of the image. Then, the target features are enhanced using an scSE channel spatial attention module and further processed by depthwise separable convolution. Finally, the features are combined with... After multiplication and processing, the result is recorded as... The process is shown in formula (3):

[0061]

[0062] in, This is an excitation for the compression of space and channels.

[0063] Finally, and The feature results are concatenated and integrated, and the feature representation is enhanced using an attention residual module (ARM). The decision to use a dropout layer is based on the descent rate to avoid overfitting. The result is denoted as... This process allows for the acquisition of composite feature maps at different scales in the early stages of the decoder operation. .

[0064] It should be noted that the above bidirectional feature fusion process employs an asymmetric design: for ViT features, which excel at global modeling, multi-scale context recalibration is performed using depthwise separable convolutions with different dilation rates; while for CNN features, which capture local geometric details, deformable convolutions and scSE modules are introduced to enhance adaptability to spatial geometric deformations. This design can specifically compensate for the shortcomings of each of the two encoder branches, rather than a simple symmetrical addition, thereby more effectively achieving deep fusion of heterogeneous features.

[0065] Step S4: Input the comprehensive feature map obtained in step S3 into the decoder, restore the feature resolution through step-by-step upsampling and multi-level attention mechanism, and optimize the boundary and structural information.

[0066] Traditional encoder-decoder networks use jump links to aggregate low-level and high-level semantics to balance spatial details and global semantics in segmentation tasks. However, this approach fails to fully utilize the complementary information of features at different levels when dealing with complex terrain or multi-scale targets, leading to problems such as blurred boundaries and inaccurate segmentation. To address this, this invention employs an attention-gated module (AGM) and an attention residual module (ARM) to achieve efficient fusion of multi-scale features and detailed modeling of boundary regions. Specifically, the steps include:

[0067] S41. Upsample the current scale's decoded features to match their spatial resolution with the previous level's encoded features. In other words, the decoder uses a step-by-step upsampling strategy to restore the feature resolution.

[0068] S42. Using an attention-gating module (AGM), deep-decoded features are used as gating signals to spatially weight and filter shallow-coded features. The module structure is as follows: Figure 3 As shown.

[0069] This invention introduces and improves the Attention Gating (AGM) module. By employing multi-layer convolution to expand the receptive field and enhance the attention map generation process, it ensures that shallow detail features are effectively filtered and strengthened when fused with deep contextual information, thereby reducing noise and increasing attention to key regions. In other words, the AGM module aims to improve the contextual interaction between shallow and deep features, thereby improving the recognition accuracy of overlapping and under-segmented regions. This module uses deep feature decoding... and shallow coding features The input is processed by depthwise separable convolutions with different dilation rates and batch normalization (BN) to compress redundant information. The result is denoted as... The process is shown in formula (4):

[0070]

[0071] Finally, convolution and The processed features are input sequentially Layers and Sigmoid generate attention maps (the results are denoted as...). The process is shown in formula (5):

[0072]

[0073] in, Indicates a 1×1 convolutional layer and Layer combination.

[0074] This attention mechanism effectively enhances the correlation between features through spatial weighting, enabling the model to highlight important details and suppress noise and irrelevant responses during the feature fusion stage. Unlike traditional skip connections, the AGM module not only enhances the interaction of cross-layer information but also guides shallow detail features in the spatial dimension. This makes the model more accurate and provides clearer boundaries in the structural recognition of complex terrains and overlapping regions.

[0075] This invention employs attention residual modules (ARMs) at multiple locations to enhance the expressive power of deep features. For ease of description, a consistent convention is used: the deep features input to the ARM module are denoted as... The output characteristics after ARM processing are denoted as .

[0076] S43. The Attention Residual Module (ARM) is used to jointly enhance the channel and spatial features output by the AGM module. This module introduces the scSE mechanism, which simultaneously squeezes spatial and channel information to enhance the saliency of key target regions, fill feature gaps, improve boundary clarity, and further enhance the feature representation capability of this invention in complex scenes. See the specific structure below. Figure 4 .

[0077] The ARM module introduces spatial channel features on top of the channel attention mechanism, using scSE with enhanced channel-spatial channel features. The ARM module enhances deep feature extraction by introducing a strong channel interaction mechanism. To preserve more boundary and structural information during the fusion process, the result is denoted as... This reduces the boundary blurring caused by shadows and overlapping areas. The calculation process is shown in formula (6):

[0078]

[0079] in, This represents a combination of BN and ReLU, and Conv is a 3×3 convolutional layer.

[0080] By extracting features simultaneously at both the channel and spatial levels, the ARM module enhances the saliency of key target regions, allowing for more refined calibration and enhancement of features during transmission and fusion. This suppresses low-reflectivity noise regions, achieving a balance between preserving structural details and boundary clarity, thereby further improving the discriminative power of the features. In other words, ARM further processes the fused features from convolution and BiFFM upsampling, applying scSE adaptive enhancement contextual information to the residual connections to ensure feature quality.

[0081] Clearly, for the ARM module that follows the AGM module, its input... This is the output of the AGM module. .

[0082] To visually verify the effectiveness of the AGM and ARM modules in feature optimization, this embodiment visualizes the heatmap evolution of features before and after processing by these two modules during the decoding process, such as... Figure 5 As shown, although the features before inputting to AGM already contain semantic information, there is still diffuse noise response in the background region, and the target boundary is relatively blurred. From the features after AGM, it is evident that AGM plays a gating and filtering role: background noise is suppressed, and the model's attention is more focused on overlapping regions with significant geometric features. This verifies AGM's ability to highlight key details and suppress irrelevant responses. Compared to the features after AGM, the features output by ARM are no longer limited to discrete skeletons but present a more continuous and fuller form. ARM fills in feature gaps inside the target by enhancing the interaction between channels and space, while making the boundary between the target and the background sharper and smoother. This transformation from sparse features to dense semantics strongly demonstrates the key contribution of the ARM module in feature refinement and structural integrity restoration.

[0083] S44. Convolutional Block Attention (CBAM) modules are introduced at each fusion stage of the decoder path to further address the undersegmentation problem of overlapping regions. The module structure and flow can be found in [link to documentation]. Figure 6 .

[0084] In the process of overlay recognition, the recognized area is often smaller than the actual labeled area. This indicates that the model has not learned enough features at the edges of the overlay area and its neighbors, resulting in undersegmentation or incomplete recognition. To address this, a Convolutional Block Attention (CBAM) module is introduced.

[0085] First, the input features Average pooling and max pooling operations are performed separately to extract global statistical information and salient response features at different scales. Then, the two are concatenated along the channel dimension and fed into a convolutional layer to aggregate contextual information through local receptive fields, generating a spatial attention map. Finally, the attention map is normalized using the sigmoid function and multiplied element-wise with the input features to obtain the optimized feature map. This achieves spatial adaptive feature enhancement, as shown in formula (7):

[0086]

[0087] in, This means concatenating the results of average pooling and max pooling along the channel dimension.

[0088] This mechanism effectively enhances the model's sensitivity to target edges and details by capturing the contextual dependencies between pixels at different locations. This allows the invention to focus on more discriminative target regions and suppress redundant background responses, thereby enhancing the model's ability to extract salient features in the spatial dimension.

[0089] CBAM performs spatial adaptive weighting on the fused features, helping the model better capture pixel features in and around the overlapping area, thus expanding the recognition range. The CBAM module is embedded in the decoder path after BiFFM and upsampling, further enhancing the learning of spatial context information during multi-scale feature fusion.

[0090] S45. Repeat steps S41 to S44 until the feature map is restored to the same resolution as the input image through 3x3 convolution (Conv).

[0091] Step S5: Perform depth supervision based on multi-scale lateral output to generate the final overlapping region identification result.

[0092] Finally, by utilizing a multi-level attention mechanism and a deep supervision strategy involving multi-scale lateral outputs, hierarchical optimization and global feature guidance of the model were achieved. In this step, deep supervision was implemented through multi-scale lateral outputs, and bilinear interpolation was used to restore all predicted maps to their original resolution to obtain better recognition results.

[0093] To alleviate the imbalance between overlapping regions and background samples in SAR images during training and to improve the model's classification accuracy for boundary pixels, this invention employs a combination of weighted smoothed cross-entropy loss and Dice loss as the total loss function. :

[0094]

[0095] in, For the total loss, To weighted smooth cross-entropy loss, This is a loss for Dice.

[0096]

[0097] in, Indicates the total number of pixels. Represents the number of categories (in this article) ). For the model to predict the first The pixel belongs to the first The probability of a class. The formula for calculating the true label after smoothing is as follows:

[0098]

[0099] in, For the original one-hot label (0 or 1). Set as a smoothing factor (0.05).

[0100]

[0101] In the formula, To prevent smoothing terms with zero denominators, the model can simultaneously achieve pixel-level classification accuracy and region-level structural integrity by jointly optimizing the aforementioned loss function.

[0102] The cross-entropy loss incorporates a label smoothing strategy to suppress overfitting and improve generalization ability; the Dice loss addresses the class imbalance between foreground and background. This structural loss is calculated at each prediction layer. Focusing on pixel-level classification accuracy, smoothing the category labels effectively alleviates the problem of model overconfidence and improves training stability; This method measures the overlap between predicted and ground truth masks to improve the model's robustness in imbalanced scenarios. To emphasize high-level semantic information, weights at different levels are progressively increased, and the final total loss is weighted and fused across the prediction outputs of each layer. This allows the model to simultaneously optimize pixel-level classification accuracy and global region consistency at different feature levels. This effectively alleviates the imbalanced class problem and enhances the model's ability to segment details and represent boundaries.

[0103] To comprehensively verify the segmentation performance and generalization ability of the proposed method in complex mountainous scenarios, this embodiment designs the following experiments based on the constructed GF-3 dataset:

[0104] Comparative Experiment: PSPNet, DeepLabv3+, and CLCFormer, three mainstream semantic segmentation networks, were selected as benchmarks. Through horizontal comparison on the dataset of western mountainous areas in Hubei Province, the performance advantages of the proposed method in identifying mountain overlap were evaluated.

[0105] Ablation experiments: To quantify the contributions of each innovative component, the experiment first constructed a baseline network retaining only the dual-stream encoder (CNN encoder branch and ViT encoder branch) and the standard decoder. Based on this, a progressively cumulative experiment was conducted using a dataset from the western mountainous region of Hubei Province, sequentially embedding four core modules: BiFFM, AGM, ARM, and CBAM, to verify their specific roles in heterogeneous feature fusion, cross-layer contextualization, boundary detail preservation, and spatial noise reduction.

[0106] To comprehensively evaluate the model's generalization ability across different terrain categories, this paper not only tests the segmentation results for each single category, including precision (P), recall (R), F1 score (F1), intersection-over-union (IoU), and Dice coefficient, but also calculates the average performance metrics for all categories, using average precision (P) as the benchmark. ), average recall rate ( ), average F1 score ( ), average crossover ratio ( ) and average Dice coefficient ( The above average indicators are used as the primary evaluation criteria. The calculation formulas for these average indicators are uniformly defined as follows:

[0107]

[0108]

[0109]

[0110]

[0111]

[0112] in, Indicates the total number of categories (in this article) (corresponding to overlapping areas, mountains, and background respectively). , and Representing the first The number of true positive, false positive and false negative pixels in a class. and The first Independent precision and recall for each class.

[0113] The experiment was conducted on an AMD Ryzen 7 7735H CPU and Radeon 680M GPU platform, using Python 3.10 + PyTorch 2.1. The optimizer used was AdamW, and the initial learning rate was set to... The batch size was set to 2. The training iterations for this invention and CLCFormer were 25, while the training iterations for DeepLabv3+ and PSPNet were 150. To ensure the fairness of the experimental results, the comparison models DeepLabv3+, PSPNet, and CLCFormer all used the same input size, data augmentation methods, and hyperparameter configurations.

[0114] Finally, the experimental results were analyzed from two dimensions: quantitative evaluation indicators and qualitative visualization effects, in order to comprehensively address the model's ability to identify overlapping areas in mountainous regions.

[0115] This embodiment demonstrates the experimental visualization results of the method of the present invention on the validation set, as shown below. Figure 7 As shown in Table 1, the segmentation performance is demonstrated. The comprehensive evaluation results confirm the analytical advantages of the method of this invention in complex mountainous scenes.

[0116] Table 1. Experimental results of this embodiment

[0117]

[0118] Table 1 Quantitative Assessment and Figure 7 From the overall performance perspective, the visualization results show that the model's evaluation indicators for both background and mountain features are close to ideal, indicating that the model has almost no false detections or missed detections in the background area and has extremely high discrimination reliability. This lays a solid foundation for background suppression in the fine extraction of overlapping areas.

[0119] The model also performed excellently in overlay category recognition. The accuracy (0.8951) and recall (0.8973) for overlay categories were highly similar and remained at a high level, indicating that the model effectively overcame the common contradiction between high false positives and high false negatives in SAR target detection, achieving a dynamic balance between the two. Furthermore, although overlay regions are typically long and fragmented strips, the model still achieved an IoU of 0.8119 and a Dice coefficient of 0.8962. Combined with... Figure 7 The visualization results show that the predicted mask not only accurately covers the overlaid subject but also exhibits extremely high fit at the edges, demonstrating the model's edge segmentation capability in pixel-level classification. Finally, through observation... Figure 7In areas with dramatic terrain undulations, the model was not misled by speckle noise; the prediction results were coherent and smooth, with very few fragmented noise misjudgments. This visual performance corresponds to the overall average accuracy of 0.9593 in Table 1, fully verifying that the present invention can effectively remove environmental noise and capture significant terrain structural features through the fusion of deep semantic features, thus verifying the robustness of the model in complex terrain.

[0120] The comparative experimental analysis is as follows: as shown in Table 2 and Figure 8 , 9 As shown, the overall performance of this invention on the same dataset significantly outperforms existing open-source methods. Firstly, from... Figure 8 The accuracy curve comparison shows that the present invention (solid red line) consistently ranks above the comparison method (dashed line) across all category intervals, demonstrating a clear performance envelope advantage. Particularly in the overlapping categories, the improvement of this model is the steepest, indicating that the present invention effectively suppresses false alarms without sacrificing recall capability for complex terrains, maintaining excellent class balance. Secondly, Figure 9 The differences in the Cross-Union-Placement (CUP) ratio among the various methods are visually demonstrated. This invention shows the most significant improvement in CUP for the overlay category, indicating that the model possesses stronger region segmentation capabilities under complex terrain structures and SAR imaging artifacts, exhibiting more accurate region segmentation and context capture capabilities. In the relatively flat mountain category, the red line also shows a small but stable lead, validating the model's effectiveness in boundary consistency extraction.

[0121] Table 2 shows the performance comparison data of different methods. Although the accuracy of the background category fluctuated slightly due to the emphasis on difficult-to-distinguish samples, it did not affect the overall advantage. Ultimately, the mP and mIoU of this invention were improved by approximately 0.33 and 2.90 percentage points, respectively, compared with the suboptimal method, fully verifying its effectiveness in interpreting complex SAR scenes.

[0122] Table 2. Performance comparison results of different methods on the western Hubei dataset.

[0123]

[0124] Figure 10 The visualization comparison results of different methods on the overlapping region segmentation task are presented in the distribution map, where the yellow part represents the overlapping area between the ground truth and the predicted image, the red part is the overlapping region identified by this invention and the region that is not overlapping in the ground truth, and the green part is the overlapping region that exists in the ground truth but does not exist in the predicted image.

[0125] from Figure 10It can be observed that PSPNet and DeepLabv3+ generally suffer from boundary ambiguity in regions with slender overlapping structures and complex geometries, resulting in significant missed detections, indicating their limited ability to model detailed features under SAR imaging distortion. CLCFormer shows some improvement in overall recognition, but still exhibits some ambiguity in identifying boundaries in strongly varying regions. Furthermore, it is prone to false activations within large, continuous regions, leading to non-overlapping areas being incorrectly identified as overlapping, reflecting its insufficient local feature constraint capabilities. In contrast, the prediction results of this invention demonstrate the best performance in terms of spatial consistency and structural integrity. In the overlay visualization results, the number of false and missed detection areas is significantly reduced, the predicted region is highly consistent with the actual annotation in shape and position, and the boundaries are clear and continuous, especially showing stronger robustness in complex terrain transition regions. These advantages are mainly attributed to the proposed CNN and ViT cross-fusion framework. Among them, BiFFM effectively integrates multi-scale heterogeneous features, AGM and ARM enhance the feature representation of key regions, and the CBAM mechanism further enhances the spatial adaptability, enabling the model to achieve stable and accurate overlay segmentation under complex SAR noise and geometric distortion conditions.

[0126] To systematically evaluate the contribution of each functional module to the model performance, this embodiment gradually introduces a bidirectional feature fusion module (BiFFM, denoted as B), an attention residual module (ARM, denoted as R), an attention gating module (AGM, denoted as G), and a convolutional block attention module (CBAM, denoted as C) on the basis of the baseline model, and conducts ablation experiments on the western Hubei dataset. The corresponding quantitative results are shown in Table 3, and the performance change trend is intuitively displayed in the form of a line graph. Figure 11 The overall results show that with the gradual introduction of each module, the model exhibits a stable and continuous improvement trend in evaluation metrics such as precision (P), recall (R), F1 score, and intersection-over-union (IoU), indicating that the proposed modules have good functional complementarity. The improvement in quantitative metrics is consistent with the improvement in segmentation quality in the visualization results, verifying the rationality of the model design.

[0127] Specifically, the baseline model's segmentation results in complex overlapping regions are relatively coarse and have blurred boundaries. Furthermore, it is prone to missegmentation at the boundaries of complex overlapping regions, reflecting its limited coverage and spatial modeling capabilities for the target region.

[0128] After introducing the BiFFM module, the model fully integrates feature information of different scales and types, making the overall outline of the overlapping region more complete and significantly improving the segmentation effect. Although the accuracy decreased slightly, the recall, F1 score, and intersection-over-union ratio were all significantly improved, indicating that the model's ability to cover the overlapping region was significantly enhanced.

[0129] Subsequently, the integration of the ARM and AGM modules further optimized feature representation. The ARM module enhances salient features through a residual attention mechanism to mitigate boundary blurring, while the AGM module promotes cross-scale feature interaction to suppress irrelevant noise. As a result, segmentation of structurally complex regions is more stable, false detections are reduced, and boundary continuity is enhanced.

[0130] Finally, the addition of the CBAM module optimizes the overall model performance, allowing it to focus more intently on key target regions. Specifically, the IoU is improved by approximately 4.59 percentage points compared to the baseline model. This performance improvement is achieved through clearer boundaries, higher region consistency, and a significant reduction in false positives and false negatives.

[0131] In summary, the quantitative results and visualization analysis of the ablation experiments are highly consistent in terms of trends and conclusions, fully verifying the synergistic effect of the BiFFM, ARM, AGM, and CBAM modules in SAR overlay region segmentation tasks.

[0132] Table 3. Ablation Experiment Results

[0133]

[0134] Finally, it is necessary to state that the above embodiments are only used to further illustrate the technical solution of the present invention in detail, and should not be construed as limiting the scope of protection of the present invention. Any non-essential improvements and adjustments made by those skilled in the art based on the above content of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A SAR image overlay recognition method with an attention mechanism, characterized in that, Includes the following steps: Step S1: Acquire the SAR image to be identified and perform preprocessing; Step S2: Input the preprocessed SAR images into the parallel CNN encoder and ViT encoder respectively to extract multi-scale features; Step S3: Use the bidirectional feature fusion module to perform deep fusion on the heterogeneous multi-scale features extracted by the CNN encoder and the ViT encoder to obtain a comprehensive feature map; The bidirectional feature fusion module includes parallel Transformer branch heads and CNN branch heads; The Transformer branch head performs channel attention recalibration on the feature map output by the ViT encoder, specifically including: performing average pooling and max pooling operations on the feature map output by the ViT encoder respectively, inputting the pooling results into depthwise separable convolutional layers with different dilation rates to generate attention weights, performing ReLU activation, depthwise separable convolution and Sigmoid activation on the attention weights in sequence, and multiplying the activation results with the original input feature map to obtain the recalibrated features; The CNN branch head performs spatial geometric adaptation enhancement on the feature map output by the CNN encoder, specifically including: sequentially performing 3×3 deformable convolution and spatial and channel compression activation operations on the feature map output by the CNN encoder, then performing depthwise separable convolution and Sigmoid activation, and multiplying the activation result with the original input feature map to obtain the enhanced features; The features processed by the Transformer branch head and the features processed by the CNN branch head are concatenated and enhanced by the attention residual module to obtain the comprehensive feature map. Step S4: Input the comprehensive feature map into the decoder, restore the feature resolution through stepwise upsampling, and optimize the features using the attention gating module and the attention residual module during the decoding process; wherein, the attention gating module takes deep decoding features and shallow coding features as input, inputs the deep decoding features and shallow coding features into depthwise separable convolutional layers with different dilation rates respectively, and performs batch normalization processing, adds the two processed features, activates them through ReLU, and then sequentially passes them through at least one CB layer composed of a 1×1 convolutional layer and a batch normalization layer and a Sigmoid activation function to generate a spatial attention map, and multiplies the spatial attention map element-wise with the shallow coding features to obtain the gated enhanced features; The attention residual module takes the input features as the processing object, performs a first convolution processing on the input features and a second convolution processing containing at least two convolutional layers, batch normalization and ReLU activation. After adding the results of the two convolution processing, the input space and channel squeezing excitation module is used to obtain the enhanced output features. Simultaneously, a convolutional block attention module is introduced in the fusion stage of the decoder path to enhance spatial context information. The convolutional block attention module performs the following operations on the input features: average pooling and max pooling are performed on the input features respectively; the average pooling results and max pooling results are concatenated in the channel dimension; the concatenated result is input into the convolutional layer and activated by the Sigmoid function to generate a spatial attention weight map; the spatial attention weight map is multiplied element-wise with the input features to obtain the spatially enhanced features. Step S5: Perform depth supervision based on the lateral output of the decoder at different scales, and train using a weighted loss function to generate the final overlapping region recognition result.

2. The identification method according to claim 1, characterized in that: In step S2, the CNN encoder uses the EfficientNet-B3 network, the ViT encoder uses the Swin Transformer network, and the multi-scale features include feature maps of four scales with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image resolution, respectively.

3. The identification method according to claim 1, characterized in that: The weighted loss function mentioned in step S5 is the sum of the weighted smoothed cross-entropy loss and the Dice loss. The weighted smoothed cross-entropy loss introduces a label smoothing strategy, which smooths the real labels through a smoothing factor. The Dice loss is used to measure the regional overlap between the predicted result and the real label.

4. The identification method according to claim 3, characterized in that: Step S5, which involves performing deep supervision based on lateral output, includes setting different loss weights for the lateral output layers of the decoder at different scales, wherein the weight of the deep output is greater than the weight of the shallow output, and then weighting and fusing the losses of each layer to obtain the total loss for training.

Images