A remote sensing image segmentation method and system based on a refined sieve double-domain attention mechanism

Abstract

This invention discloses a remote sensing image segmentation method and system based on a fine-screening dual-domain attention mechanism, belonging to the field of remote sensing image processing technology. This invention constructs a multi-level feature extraction network with a fine-screening dual-domain attention unit as the core, and integrates pyramid-like cross-layer residual connections to significantly enhance the semantic expression and spatial distribution perception capabilities in remote sensing images. Furthermore, a fine-screening dual-domain attention unit is introduced into each layer of the encoding module, combined with a multi-level feature fusion mechanism, to achieve full modeling of the texture, structure, and contextual information of remote sensing images at different scales, and to complete remote sensing image segmentation on this basis.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and more specifically to a remote sensing image segmentation method based on a fine-screening dual-domain attention mechanism. Background Technology

[0002] With the continuous improvement of remote sensing imaging technology and satellite observation capabilities, remote sensing images are playing an increasingly important role in fields such as urban planning, geographic information extraction, environmental monitoring, and land use surveys. Among these, semantic segmentation of remote sensing images, as a fundamental processing task, aims to perform pixel-by-pixel accurate classification of various land features (such as roads, buildings, water bodies, and farmland) in high-resolution remote sensing images, and is one of the key modules of automated remote sensing understanding systems.

[0003] Traditional remote sensing image segmentation methods are mostly based on convolutional neural network (CNN) structures, such as U-Net, SegNet, and DeepLab. While these methods can extract spatial texture and local semantic features in general scenes, they suffer from limited receptive fields and insufficient context-dependent capabilities. Especially when dealing with targets of vastly different scales in remote sensing images, they are prone to problems such as "missed detection of small targets" and "blurred edges." Furthermore, remote sensing images often have complex background interference and large-scale scene structures, making traditional methods inadequate in feature modeling and target differentiation.

[0004] In recent years, global attention mechanisms based on Transformers have been gradually introduced into remote sensing image processing tasks, such as the Swing Transformer and SegFormer. Theoretically, these methods possess strong long-range dependency modeling capabilities and can capture the global contextual semantics in remote sensing images. However, due to the large size and diverse types of targets in remote sensing images, traditional fully connected attention computation incurs significant computational overhead and memory burden, making it difficult to deploy in practical engineering. Furthermore, these methods still have limitations in characterizing small targets or fine-grained edges, and are easily overwhelmed by global semantics.

[0005] Therefore, how to maintain high model accuracy while taking into account the ability to perceive small targets, preserve edge details, and model global context in remote sensing images has become the focus and challenge of current research on remote sensing semantic segmentation.

[0006] To address the aforementioned issues, existing research has attempted to improve segmentation performance through methods such as pyramid structures, multi-scale branching, and lightweight attention mechanisms. However, the following shortcomings remain: First, it is difficult to balance global information modeling capability with computational efficiency, especially when dealing with small-sized remote sensing images; second, target details and edge information are often lost due to feature downsampling and coarse aggregation, affecting the detection accuracy of small targets; and third, in the feature fusion stage, cross-layer semantic alignment and contextual association lack fine-grained modeling, resulting in blurred boundaries and category confusion in the semantic segmentation results.

[0007] Therefore, how to improve the segmentation accuracy of multi-category targets in remote sensing images is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] In view of this, the present invention provides a remote sensing image segmentation method and system based on a fine screening dual-domain attention mechanism, which aims to achieve high-precision and robust segmentation of multi-class targets (especially small targets and boundary targets) in remote sensing images by fusing region and pixel-level attention mechanisms with multi-scale feature modeling structures.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] This invention first discloses a remote sensing image segmentation method based on a fine-screening dual-domain attention mechanism, comprising the following steps:

[0011] Convolutional layers are used to extract features from remote sensing images, and the extracted features are downsampled to obtain the initial features of the remote sensing images.

[0012] Depth-separable convolutional layers are used to further extract features from the initial features, thereby obtaining local spatial information of the remote sensing image;

[0013] The local spatial information is residually connected with the initial features, and the residual connection result is input into the fine-screen dual-domain attention unit for feature enhancement to obtain the initial attention-enhanced features;

[0014] The initial features are downsampled layer by layer to obtain multiple downsampled features. Each downsampled feature is then enhanced by a dual-domain attention unit to obtain downsampled enhanced features at different levels.

[0015] The obtained downsampled features are upsampled according to the hierarchy and fused with the downsampled features of the previous layer to obtain the downsampled fused features;

[0016] The initial attention-enhanced features are fused with the downsampled fusion features to obtain highly semantically information-rich fusion features;

[0017] The high semantic information fusion features are mapped to the category space through convolution to obtain the remote sensing image segmentation result.

[0018] Furthermore, before using convolutional layers to extract features from remote sensing images, the following steps are also included:

[0019] Standard preprocessing is performed on remote sensing images, including normalization, resizing, and noise suppression.

[0020] Furthermore, the step of obtaining the initial attention-enhanced features also includes:

[0021] The initial attention-enhanced features are then normalized and processed for feature perception in sequence.

[0022] The result of the feature perception processing is then residually connected with the initial attention enhancement feature to obtain the final attention enhancement feature.

[0023] Furthermore, the step of downsampling the initial features layer by layer specifically includes:

[0024] Use convolutional or pooling layers with a stride of 2 to downsample the initial features of the remote sensing image layer by layer.

[0025] Furthermore, feature enhancement is performed by refining the dual-domain attention units, specifically including the following steps:

[0026] Step 2.1: Region-Pixel Dual-Domain Linear Projection:

[0027] First, the feature vector F input to the refined dual-domain attention unit is processed by a 1×1 convolution. In The linear projection consists of a query matrix Q, a key matrix K, and a value matrix V.

[0028] Next, region-level average pooling is performed on the query matrix Q and the key matrix K:

[0029] The query matrix Q and the key matrix K are each divided into several regions using an S×S non-overlapping grid. Average pooling is then performed on the elements within each region to obtain the region-level query matrix Q. r and regional bond matrix K r ;

[0030] Step 2.2: Calculation of the regional correlation matrix:

[0031] For the regional query matrix Q r and regional bond matrix K r Perform regional-level correlation matrix calculation to obtain regional-level attention weights A. r For the regional attention weight A r After performing Top-k1 selection, the sparse mask index F of the region is obtained. r ;

[0032] Step 2.3: Region-Pixel Index Sampling: Based on the filtered region sparse mask index F r Perform the gather operation on the original key matrix K and value matrix V respectively to obtain the regionally filtered key K. g With value V g ;

[0033] Step 2.4: Key K after region filtering g The pixel-level attention weights A are obtained by performing pixel-level relevance matrix calculation on the query matrix Q. p For pixel-level attention weights A p Top-k2 selection yields the pixel sparse mask index F. p Based on the pixel sparse mask index F obtained through filtering p Value V after regional filtering g Perform the gather operation to obtain the pixel-level filtered value V. gg Pixel-level attention weights A p Compared with the pixel-level filtered value V gg The final sparse attention feature F is obtained by weighted summation. att ;

[0034] Step 2.5: Input the feature vector F of the refined dual-domain attention unit In Local texture features F are obtained through depthwise separable convolution. loc and sparse attention feature F att The enhanced features of the final output are obtained by splicing and fusion.

[0035] Furthermore, the highly semantically information-fused features are mapped to the category space through convolution to obtain the remote sensing image segmentation result, specifically including:

[0036] The highly semantically information-fused features are linearly projected through a 1×1 convolutional layer, mapping the channel dimension C to the number of semantic categories N, to obtain the prediction score:

[0037] Perform a pixel-by-pixel softmax operation on the output predicted scores to obtain the predicted probabilities for each category:

[0038] The predicted probabilities of each category are input into the argmax function to obtain the final predicted category for remote sensing image segmentation.

[0039] Another aspect of the present invention discloses a remote sensing image segmentation system based on a fine-screening dual-domain attention mechanism, comprising a remote sensing image feature extraction network and a remote sensing image segmentation module;

[0040] The remote sensing image feature extraction network is used to extract high semantic information fusion features from the input remote sensing image; specifically, it includes the following steps: using convolutional layers to extract features from the remote sensing image, and downsampling the extracted features to obtain the initial features of the remote sensing image;

[0041] Depth-separable convolutional layers are used to further extract features from the initial features, thereby obtaining local spatial information of the remote sensing image;

[0042] The local spatial information is residually connected with the initial features, and the residual connection result is input into the fine-screen dual-domain attention unit for feature enhancement to obtain the initial attention-enhanced features;

[0043] The initial features are downsampled layer by layer to obtain multiple downsampled features. Each downsampled feature is then enhanced by a dual-domain attention unit to obtain downsampled enhanced features at different levels.

[0044] The obtained downsampled features are upsampled according to the hierarchy and fused with the downsampled features of the previous layer to obtain the downsampled fused features;

[0045] The initial attention-enhanced features are fused with the downsampled fusion features to obtain highly semantically information-rich fusion features;

[0046] The remote sensing image segmentation module is used to map the high semantic information fusion features to the category space through convolution to obtain the remote sensing image segmentation result.

[0047] Preferably, before inputting the remote sensing image into the remote sensing image feature extraction network, the method further includes:

[0048] Standard preprocessing is performed on remote sensing images, including normalization, resizing, and noise suppression.

[0049] Preferably, the remote sensing image feature extraction network further includes the following steps in obtaining the initial attention-enhanced features:

[0050] The initial attention-enhanced features are then normalized and processed for feature perception in sequence.

[0051] The result of the feature perception processing is then residually connected with the initial attention enhancement feature to obtain the final attention enhancement feature.

[0052] Preferably, the feature enhancement performed by the refined dual-domain attention unit includes the following steps:

[0053] Step 2.1: Region-Pixel Dual-Domain Linear Projection:

[0054] First, the feature vector F input to the refined dual-domain attention unit is processed by a 1×1 convolution. InThe linear projection consists of a query matrix Q, a key matrix K, and a value matrix V.

[0055] Next, region-level average pooling is performed on the query matrix Q and the key matrix K:

[0056] The query matrix Q and the key matrix K are each divided into several regions using an S×S non-overlapping grid. Average pooling is then performed on the elements within each region to obtain the region-level query matrix Q. r and regional bond matrix K r ;

[0057] Step 2.2: Calculation of the regional correlation matrix:

[0058] For the regional query matrix Q r and regional bond matrix K r Perform regional-level correlation matrix calculation to obtain regional-level attention weights A. r For the regional attention weight A r After performing Top-k1 selection, the sparse mask index F of the region is obtained. r ;

[0059] Step 2.3: Region-Pixel Index Sampling: Based on the filtered region sparse mask index F r Perform the gather operation on the original key matrix K and value matrix V respectively to obtain the regionally filtered key K. g With value V g ;

[0060] Step 2.4: Key K after region filtering g The pixel-level attention weights A are obtained by performing pixel-level relevance matrix calculation on the query matrix Q. p For pixel-level attention weights A p Top-k2 selection yields the pixel sparse mask index F. p Based on the pixel sparse mask index F obtained through filtering p Value V after regional filtering g Perform the gather operation to obtain the pixel-level filtered value V. gg Pixel-level attention weights A p Compared with the pixel-level filtered value V gg The final sparse attention feature F is obtained by weighted summation. att ;

[0061] Step 2.5: Input the feature vector F of the refined dual-domain attention unit In Local texture features F are obtained through depthwise separable convolution. loc and sparse attention feature F att The enhanced features of the final output are obtained by splicing and fusion.

[0062] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a remote sensing image segmentation method and system based on a fine-screening dual-domain attention mechanism, which has the following beneficial effects:

[0063] 1) This invention innovatively proposes a "refined dual-domain attention mechanism" (FBDA) to achieve two-stage sparse modeling at the region and pixel levels, balancing global semantic dependencies with edge target details. While traditional global attention mechanisms can capture long-distance dependencies, they incur huge computational costs in large remote sensing images; local convolutional models, although efficient, have limited receptive fields, making it difficult to capture relationships between distant regions. This invention's refined dual-domain attention mechanism first performs initial screening of key regions at the region level, and then performs pixel-level re-screening within each region, significantly reducing computational complexity while effectively highlighting target regions and edge features. This mechanism improves the consistency of segmentation for small targets and fine edges while maintaining information discriminativeness.

[0064] 2) A multi-level feature extraction network with FBDA units as the core was constructed and pyramid-like residual connections were fused to significantly enhance the semantic expression and spatial distribution perception capabilities in remote sensing images.

[0065] 3) Compared to traditional U-Net or DeepLab series networks, this invention introduces FBDA attention units in each encoding layer, combined with a multi-layer feature fusion mechanism, to achieve full modeling of texture, structure, and contextual information of remote sensing images at different scales. This structure is particularly suitable for complex scenes in remote sensing images where large-scale areas (such as water bodies and farmland) and fine-grained targets (such as roads and ships) coexist. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0067] Figure 1 This is a schematic diagram of the overall process architecture of the remote sensing image segmentation system provided by the present invention.

[0068] Figure 2 This is a schematic diagram of the dual-domain attention unit structure for fine screening provided by the present invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] This invention discloses a remote sensing image segmentation method based on a fine-screening dual-domain attention mechanism, which effectively extracts multi-scale, complex texture, and spatial structure information from remote sensing images. The method first constructs a multi-level feature extraction network. This network uses a fine-screening dual-domain attention unit as its core structure, combined with a residual connection mechanism, to achieve feature modeling from the bottom layer to the top layer, providing rich semantic representation for subsequent semantic segmentation.

[0071] The specific method includes the following steps:

[0072] Convolutional layers are used to extract features from remote sensing images, and the extracted features are downsampled to obtain the initial features of the remote sensing images.

[0073] Depth-separable convolutional layers are used to further extract features from the initial features, thereby obtaining local spatial information of the remote sensing image;

[0074] The local spatial information is residually connected with the initial features, and the residual connection result is input into the fine-screen dual-domain attention unit for feature enhancement to obtain the initial attention-enhanced features;

[0075] The initial features are downsampled layer by layer to obtain multiple downsampled features. Each downsampled feature is then enhanced by a dual-domain attention unit to obtain downsampled enhanced features at different levels.

[0076] The obtained downsampled features are upsampled according to the hierarchy and fused with the downsampled features of the previous layer to obtain the downsampled fused features;

[0077] The initial attention-enhanced features are fused with the downsampled fusion features to obtain highly semantically information-rich fusion features;

[0078] The high semantic information fusion features are mapped to the category space through convolution to obtain the remote sensing image segmentation result.

[0079] The remote sensing image feature extraction network constructed in this invention, such as Figure 1 As shown, this method aims to progressively extract spatial-frequency composite features from remote sensing images by combining multi-level convolutions with attention mechanisms. While maintaining the integrity of key information, it constructs a highly expressive feature representation, providing reliable input for subsequent semantic decoding and fine-grained segmentation prediction. This remote sensing image feature extraction network performs the following steps:

[0080] First, remote sensing images Before entering the feature extraction network, standard preprocessing is performed, including normalization, resizing, and noise suppression. Then, the initial convolutional layer is used to obtain the features. . Following the downsampling stage, the core objective is to reduce the image resolution (i.e., size) while preserving as much important information as possible and avoiding aliasing distortion. This is achieved through pooling layers or convolutional layers with a stride greater than 1, yielding the initial features. .

[0081] Then, a refined dual-domain attention unit is constructed to filter the initial features. Processing is then performed. At this stage, to enhance the network's ability to perceive cross-scale spatial structure and local frequency features in remote sensing images, a fine-tuned dual-domain attention model (FBDA) is introduced for modeling.

[0082] First, initial features Local spatial information is extracted using depthwise separable convolution (DWConv) layers, and residual connections are made with the original input features.

[0083]

[0084] then, Entering the dual-domain attention mechanism unit for fine screening, attention enhancement features are obtained. To further enhance nonlinear modeling capabilities, it is fed into a structure containing a normalization layer (LayerNorm) and a multilayer perceptron (MLP):

[0085]

[0086] This structure uses two residual connections to form Transformer-style modules, ensuring stable information propagation and enhancing feature representation capabilities.

[0087] Simultaneously, multi-level feature extraction and downsampling are performed on the initial features. Specifically, for Perform layer-by-layer downsampling (using convolution or pooling operations with a stride of 2) to extract deeper-level features sequentially, resulting in... , . , Each layer is processed using a finely screened dual-domain attention unit. , .

[0088] Then to Upsampling aims to increase the resolution (i.e., size) of an image, typically achieved using interpolation or deconvolution, yielding f3. At this moment, f3 Again with Perform addition and fusion, and then upsampling to obtain feature f2. f2 In Perform addition and fusion, and then upsampling to obtain the feature f. This feature f This will be used as input to the subsequent semantic decoding module for fine-grained pixel-level segmentation prediction.

[0089] This invention constructs a fine-screen dual-domain attention unit for remote sensing scenarios, such as... Figure 2 As shown, remote sensing images typically cover a wide range of ground features, including both large homogeneous areas (such as farmland and water bodies) and small-scale detailed targets (such as roads, buildings, and ships). Traditional global self-attention mechanisms are computationally too expensive, while the local receptive field of CNNs is insufficient to capture long-range dependencies. Therefore, this refined dual-domain attention model, tailored to the characteristics of remote sensing images, performs two sparse selections within both "regional" and "pixel" domains, calculating attention only on the most discriminative tokens, thus considering both global context and fine-grained edge information. The refined dual-domain attention unit constructed in this invention, designed for remote sensing scenes, solves the above problems. Specifically, the refined dual-domain attention unit performs the following steps:

[0090] Step 2.1: Region-Pixel Dual-Domain Linear Projection. Assume the input feature tensor of the refined dual-domain attention unit... This is a remote sensing feature map at a certain stage (containing multispectral or hyperspectral information). First, it is linearly projected into Query Q, Key K, and Value V through a 1×1 convolution.

[0091] Next, to reduce redundant computation, "region-level average pooling" is performed on Q and K: the feature map is divided into several regions using an S×S non-overlapping grid (e.g., S=8 corresponds to 64 region tokens); average pooling is performed within each region to obtain the region-level query. and Key .

[0092] This step, while preserving the macroscopic semantics, drastically reduces the number of tokens from H×W to S^2, significantly alleviating the memory pressure caused by large remote sensing images.

[0093] Step 2.2: Calculation of the regional correlation matrix. and The regional correlation matrix was calculated to obtain After performing a Top-k1 selection, we get... .

[0094] Step 2.2: Regional significance screening. Calculate the regional correlation matrix:

[0095]

[0096] in, This indicates a transpose operation. Its elements characterize the semantic relationships between different geographical regions (e.g., water bodies and wetlands, cities and suburbs). A Top-k1 strategy (k1 = 16 in the experiment) is used to retain only the most relevant regional indices for each region. This creates a sparse mask. This step can suppress interference from irrelevant background areas such as clouds and mountains on subsequent attention, highlighting potential target areas.

[0097] Step 2.3: Region-Pixel Index Sampling. Based on the index... Perform a gather operation on the original K and V to obtain the filtered Key and Value:

[0098]

[0099] at this time It only includes all pixel-level tokens within the Top-k1 region, achieving the first pruning "from coarse to fine".

[0100] Step 2.4: Pixel-level correlation matrix calculation is performed with Q to obtain , Features were obtained through Top-k2 selection. . In Obtained by index sampling . and Multiply to get .

[0101] Step 2.4 Pixel-level fine-grained re-filtering. The retained pixel-level keys... Calculate the pixel-level relevance matrix with all Query Q:

[0102]

[0103] Top-k2 is used again (specifically, λ=1 / 8 can be adaptively selected) to select the most relevant pixel tokens, resulting in sparser attention weights. and its index .right Perform a second index sampling:

[0104]

[0105] Finally, perform a weighted summation:

[0106] .

[0107] Output It captures long-range contextual dependencies while focusing on key remote sensing targets, such as building boundaries and road centerlines.

[0108] Step 2.5 Local Texture Enhancement and Residual Fusion. Remote sensing images have rich textures, and relying solely on sparse attention may result in the loss of local details. Therefore, the original features F of the finely screened dual-domain attention units are enhanced. In Local texture features are obtained through 5×5 depthwise separable convolution. The final output of the refined dual-domain attention unit is represented as:

[0109] .

[0110] The precision screening dual-domain attention unit designed in this invention can achieve global-local information complementarity. This design enables precision screening dual-domain attention for remote sensing scenes to significantly improve the segmentation accuracy and edge consistency of scenes with coexisting "small targets + large scenes" in remote sensing images while maintaining high computational efficiency.

[0111] After completing cross-level feature fusion, the fused feature map The image is already at the original resolution, possessing rich spatial semantic representation and multi-scale contextual information. This step constructs a lightweight semantic segmentation head to achieve pixel-by-pixel prediction of multi-category ground objects in remote sensing images. Simultaneously, multiple loss functions are designed to enhance small object detection and boundary preservation capabilities, improving the overall accuracy and robustness of remote sensing image segmentation.

[0112] Step 3.1: Convolutional mapping to the class space. This fuses the features. A linear projection is performed using a 1 × 1 convolutional layer to map the channel dimension C to the number of semantic categories N, resulting in a predicted score map:

[0113] ;

[0114] Where N represents the number of preset land cover categories (e.g., roads, buildings, water bodies, woodlands, etc.), and each pixel (i,j) corresponds to a category probability vector of length N.

[0115] Step 3.2: Semantic Prediction and Normalization. Perform a pixel-wise softmax operation on the output \hat{Y} to obtain the predicted probabilities for each category:

[0116]

[0117] The final prediction category is: .

[0118] in, At position (i,j), the predicted score (logit) for the nth category is the input to the softmax operation. The probability value of belonging to class n at pixel position (i,j). This is the softmax output, which is normalized to the interval [0,1]. The exponential function is used to convert logits to non-negative numbers, ensuring the probability is computable. `i` and `j` are the two-dimensional coordinates of pixels in the image. `n` is the class index, indicating which class it belongs to. `k` is the class index used for summation, normalizing the denominator in the softmax function. `H` represents the image height (number of pixels). `W` represents the image width (number of pixels). `N` represents the total number of classes. The predicted probabilities of each class are input into the `argmax` function to obtain the final predicted class for remote sensing image segmentation.

[0119] Step 3.3: Design of Joint Supervision Loss Function. To optimize the segmentation effect, a three-objective function was designed, comprising backbone cross-entropy loss, Dice loss, and boundary enhancement loss, as detailed below:

[0120] 1) Main classification loss: Cross-entropy loss

[0121]

[0122] in The one-hot encoding representing the true label of pixel (i,j).

[0123] 2) Class Imbalance Handling: Dice Loss

[0124] To enhance the segmentation ability of small target classes, the Dice loss function is introduced:

[0125]

[0126] Where P and Y are the predicted and true probability plots (expanded by category), respectively, and εsilon is a small constant to prevent the denominator from being zero.

[0127] 3) Edge-guided loss (optional)

[0128] To improve boundary details, an auxiliary edge detection branch is constructed. Using BCE loss and edge truth graph (Can be obtained from Y via Sobel or Laplacian calculations) for training:

[0129]

[0130] Step 3.4: Total Loss Function and Optimization Objective. The final training optimization objective is the multi-task joint loss function:

[0131]

[0132] in These are the weights for the loss term. During the training phase, end-to-end joint optimization is performed on all modules through backpropagation.

[0133] After this step is completed, the model can output an accurate semantic segmentation map of the same size as the input image, realizing pixel-level classification and precise boundary positioning of multiple types of land features in remote sensing images, providing reliable support for applications such as geographic information extraction, urban planning, and environmental monitoring.

[0134] On the other hand, the present invention also discloses a remote sensing image segmentation system based on a fine screening dual-domain attention mechanism, including a remote sensing image feature extraction network and a remote sensing image segmentation module.

[0135] The specific data execution steps of the remote sensing image feature extraction network and the remote sensing image segmentation module are described in the relevant method and will not be repeated here.

[0136] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0137] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image segmentation method based on fine-screen dual-domain attention mechanism, characterized in that, Includes the following steps: Convolutional layers are used to extract features from remote sensing images, and the extracted features are downsampled to obtain the initial features of the remote sensing images. Depth-separable convolutional layers are used to further extract features from the initial features, thereby obtaining local spatial information of the remote sensing image; The local spatial information is residually connected with the initial features, and the residual connection result is input into the fine-screen dual-domain attention unit for feature enhancement to obtain the initial attention-enhanced features; The initial features are downsampled layer by layer to obtain multiple downsampled features. Each downsampled feature is then enhanced by a dual-domain attention unit to obtain downsampled enhanced features at different levels. The obtained downsampled features are upsampled according to the hierarchy and fused with the downsampled features of the previous layer to obtain the downsampled fused features; The initial attention-enhanced features are fused with the downsampled fusion features to obtain highly semantically information-rich fusion features; The highly semantically rich fusion features are mapped to the category space via convolution to obtain the remote sensing image segmentation result; The feature enhancement process, which involves refining dual-domain attention units, specifically includes the following steps: Step 2.1: Region-Pixel Dual-Domain Linear Projection: First, the feature vector F inputting the fine-screen dual-domain attention unit is passed through a 1x1 convolution In Linear projection is a query matrix Q, a key matrix K, and a value matrix V. Next, region-level average pooling is performed on the query matrix Q and the key matrix K: The query matrix Q and the key matrix K are respectively divided into a plurality of regions according to S*S non-overlapping grids, average pooling is performed on elements in each region, and a regional query matrix Q r and a regional key matrix K r are obtained. Step 2.2: Calculation of the regional correlation matrix: Q r and region-level key matrix K r region-level correlation matrix calculation, get region-level attention weight A r , region-level attention weight A r Top-k1 selection after getting region sparse mask index F r ; Step 2.3: Region-pixel index sampling: according to the region sparse mask index F obtained by screening r The gather operation is performed on the original key matrix K and value matrix V respectively, to obtain the region-level screened key K g and value V g ; Step 2.4: Key K after region filtering g The pixel-level attention weights A are obtained by performing pixel-level relevance matrix calculation on the query matrix Q. p For pixel-level attention weights A p Top-k2 selection is performed to obtain the pixel sparse mask index F. p Based on the pixel sparse mask index F obtained through filtering p Value V after regional filtering g Perform the gather operation to obtain the pixel-level filtered value V. gg Pixel-level attention weights A p Compared with the pixel-level filtered value V gg The final sparse attention feature F is obtained by weighted summation. att ; Step 2.5: Concatenate the feature vector F In Local texture features F are obtained by deep separable convolution loc And sparse attention features F att The final output enhanced features are obtained by splicing and fusion.

2. The remote sensing image segmentation method based on the fine-screen dual-domain attention mechanism according to claim 1, characterized in that, Before using convolutional layers to extract features from remote sensing images, the following steps are also included: Standard preprocessing is performed on remote sensing images, including normalization, resizing, and noise suppression.

3. The remote sensing image segmentation method based on the fine-screen dual-domain attention mechanism according to claim 1, characterized in that, The step of obtaining the initial attention-enhanced features also includes: The initial attention-enhanced features are then normalized and processed for feature perception in sequence. The result of the feature perception processing is then residually connected with the initial attention enhancement feature to obtain the final attention enhancement feature.

4. The remote sensing image segmentation method based on the fine-screen dual-domain attention mechanism according to claim 1, characterized in that, The step of downsampling the initial features layer by layer specifically includes: Use convolutional or pooling layers with a stride of 2 to downsample the initial features of the remote sensing image layer by layer.

5. The remote sensing image segmentation method based on the fine-screen dual-domain attention mechanism according to claim 1, characterized in that, The highly semantically rich fusion features are mapped to the category space via convolution to obtain the remote sensing image segmentation result, specifically including: The highly semantically information-fused features are linearly projected through a 1×1 convolutional layer, mapping the channel dimension C to the number of semantic categories N, to obtain the prediction score: Perform a pixel-by-pixel softmax operation on the output predicted scores to obtain the predicted probabilities for each category: The predicted probabilities of each category are input into the argmax function to obtain the final predicted category for remote sensing image segmentation.

6. A remote sensing image segmentation system based on fine-screen dual-domain attention mechanism, characterized in that, This includes a remote sensing image feature extraction network and a remote sensing image segmentation module; The remote sensing image feature extraction network is used to extract high semantic information fusion features from the input remote sensing image; specifically, it includes the following steps: using convolutional layers to extract features from the remote sensing image, and downsampling the extracted features to obtain the initial features of the remote sensing image; Depth-separable convolutional layers are used to further extract features from the initial features, thereby obtaining local spatial information of the remote sensing image; The local spatial information is residually connected with the initial features, and the residual connection result is input into the fine-screen dual-domain attention unit for feature enhancement to obtain the initial attention-enhanced features; The initial features are downsampled layer by layer to obtain multiple downsampled features. Each downsampled feature is then enhanced by a dual-domain attention unit to obtain downsampled enhanced features at different levels. The obtained downsampled features are upsampled according to the hierarchy and fused with the downsampled features of the previous layer to obtain the downsampled fused features; The initial attention-enhanced features are fused with the downsampled fusion features to obtain highly semantically information-rich fusion features; The remote sensing image segmentation module is used to map the high semantic information fusion features to the category space through convolution to obtain the remote sensing image segmentation result; The refined dual-domain attention unit performs feature enhancement, specifically including the following steps: Step 2.1: Region-Pixel Dual-Domain Linear Projection: First, the feature vector F inputting the fine-screen dual-domain attention unit is passed through a 1x1 convolution In Linear projection is a query matrix Q, a key matrix K, and a value matrix V. Next, region-level average pooling is performed on the query matrix Q and the key matrix K: The query matrix Q and the key matrix K are respectively divided into a plurality of regions according to S*S non-overlapping grids, average pooling is performed on elements in each region, and a regional query matrix Q r and a regional key matrix K r are obtained. Step 2.2: Calculation of the regional correlation matrix: Q r and region-level key matrix K r region-level correlation matrix calculation, get region-level attention weight A r , region-level attention weight A r Top-k1 selection after getting region sparse mask index F r ; Step 2.3: Region-pixel index sampling: according to the region sparse mask index F obtained by screening r The gather operation is performed on the original key matrix K and value matrix V respectively, to obtain the region-level screened key K g and value V g ; Step 2.4: Key K after region screening g and query matrix Q to obtain pixel-level attention weight A p , pixel-level attention weight A p is Top-k2 selected to obtain pixel sparse mask index F p ; according to the pixel sparse mask index F p screened, the value V g after region screening is obtained gg , pixel-level attention weight A p and the value V gg after pixel-level screening are weighted and summed to obtain the final sparse attention feature F att ; Step 2.5: Concatenate the feature vector F In Local texture features F are obtained by deep separable convolution loc And sparse attention features F att The final output enhanced features are obtained by splicing and fusion.

7. The remote sensing image segmentation system based on the refined sieve dual-domain attention mechanism according to claim 6, characterized in that, Before inputting the remote sensing image into the remote sensing image feature extraction network, the following steps are also included: Standard preprocessing is performed on remote sensing images, including normalization, resizing, and noise suppression.

8. The remote sensing image segmentation system based on the refined sieve dual-domain attention mechanism according to claim 6, characterized in that, The remote sensing image feature extraction network further includes the following steps in obtaining the initial attention-enhanced features: The initial attention-enhanced features are then normalized and processed for feature perception in sequence. The result of the feature perception processing is then residually connected with the initial attention enhancement feature to obtain the final attention enhancement feature.

Images