Remote sensing image segmentation method based on global dependency and local texture fusion attention

Abstract

This invention discloses a remote sensing image segmentation method based on global dependency and local texture fusion attention, belonging to the field of image processing technology. The method includes the following steps: inputting the remote sensing image to be segmented into a trained multi-scale global-local fusion remote sensing segmentation network to obtain a segmented image of the remote sensing image to be segmented; wherein the multi-scale global-local fusion remote sensing segmentation network adopts a symmetrical encoder-decoder structure, and introduces global dependency and local texture fusion attention modules at each scale layer. This invention can effectively fuse local texture details and global spatial dependencies, achieving high-precision segmentation and high-efficiency inference simultaneously in complex remote sensing scenes, and significantly improving the recognition ability of small targets, fine boundaries, and large-scale ground structures.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a remote sensing image segmentation method based on global dependency and local texture fusion attention. Background Technology

[0002] Currently, with the rapid development of high-resolution satellite and UAV imagery, remote sensing image segmentation has become a crucial step in tasks such as geographic information extraction, urban planning, agricultural monitoring, environmental analysis, and disaster assessment. Existing methods are mainly divided into two categories: one is based on convolutional neural networks (CNNs), such as U-Net and the DeepLab series, which achieve pixel-level semantic segmentation through encoder-decoder structures and multi-scale feature extraction. However, their local receptive fields are difficult to model large-scale spatial dependencies such as road networks, river extensions, and urban and rural area layouts in remote sensing images, easily leading to category confusion and boundary blurring. The other category is based on Transformers, such as SegFormer and Swin Transformer. Although they can capture global context and improve scene consistency through self-attention mechanisms, their computational complexity increases quadratically with image resolution, making it difficult to directly process high-resolution remote sensing images, and their ability to represent small targets and detailed boundaries remains insufficient. In recent years, some studies have attempted to combine the advantages of local modeling of CNNs and global modeling of Transformers, and introduce multi-scale feature fusion and lightweight attention mechanisms. However, these methods still have obvious limitations: First, they do not specifically model the unique characteristics of remote sensing images, such as the coexistence of "multi-scale small targets and large-scale global structures"; second, they are difficult to maintain high segmentation accuracy while ensuring computational efficiency, which restricts their practical application in high-resolution, large-field-of-view remote sensing scenarios.

[0003] Therefore, how to provide a remote sensing image segmentation method that can effectively integrate local texture details and global spatial dependencies, achieve high-precision segmentation and high-efficiency inference in complex remote sensing scenes, and significantly improve the ability to identify small targets, fine boundaries and large-scale ground structures is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a remote sensing image segmentation method based on global dependency and local texture fusion attention to overcome or at least partially solve the above problems.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A remote sensing image segmentation method based on global dependency and local texture fusion attention includes the following steps:

[0007] The remote sensing image to be segmented is input into a trained multi-scale global-local fusion remote sensing segmentation network to obtain a segmented image of the remote sensing image to be segmented; wherein, the multi-scale global-local fusion remote sensing segmentation network adopts a symmetrical encoder-decoder structure and introduces a global dependency and local texture fusion attention module in each scale layer.

[0008] Preferably, the remote sensing image to be segmented is input into a trained multi-scale global-local fusion remote sensing segmentation network to obtain a segmented image of the remote sensing image to be segmented, specifically including the following steps:

[0009] The remote sensing image to be segmented is input into the initial convolutional layer to obtain the first shallow feature layer. ;

[0010] For the first shallow layer features Perform downsampling to obtain the second shallow layer features. ;

[0011] The first shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the first enhanced feature. ;

[0012] For the second shallow layer features Perform downsampling to obtain the third shallow layer features. ;

[0013] The second shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the second enhanced feature. ;

[0014] For the third shallow layer features Perform downsampling to obtain the fourth layer of shallow features. ;

[0015] The third shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the third enhanced feature. ;

[0016] For the fourth shallow layer features Perform upsampling to obtain the first upsampled feature. ;

[0017] The first upsampled feature and the third enhancement feature By performing element-wise addition, the first fusion feature is obtained. ;

[0018] For the first fusion feature Perform upsampling to obtain the second upsampled feature. ;

[0019] The second upsampled feature and the second enhancement feature By performing element-wise addition, the second fusion feature is obtained. ;

[0020] For the second fusion feature Perform upsampling to obtain the third upsampled feature. ;

[0021] The third upsampling feature With the first enhanced feature By performing element-wise addition, the third fusion feature is obtained. ;

[0022] Regarding the third fusion feature After upsampling, the data is input into the segmentation head to obtain a segmented image of the remote sensing image to be segmented.

[0023] Preferably, the first shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the first enhanced feature. Specifically, it includes the following steps:

[0024] The first shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ;

[0025] The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2.

[0026] The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ;

[0027] The segmentation features Normalized features are obtained after processing by the normalization layer. ;

[0028] The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ;

[0029] The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ;

[0030] The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ;

[0031] The rearrangement feature and the first shallow layer features By summing the residuals, intermediate features can be obtained. ;

[0032] The intermediate features Normalized features are obtained after processing by the normalization layer. ;

[0033] The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0034] The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ;

[0035] The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0036] The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ;

[0037] The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ;

[0038] The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ;

[0039] The intermediate features and the intermediate features The first enhanced feature is obtained by summing the residuals. .

[0040] Preferably, the second shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the second enhanced feature. Specifically, it includes the following steps:

[0041] The second shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ;

[0042] The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2.

[0043] The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ;

[0044] The segmentation features Normalized features are obtained after processing by the normalization layer. ;

[0045] The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ;

[0046] The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ;

[0047] The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ;

[0048] The rearrangement feature and the second shallow layer features By summing the residuals, intermediate features can be obtained. ;

[0049] The intermediate features Normalized features are obtained after processing by the normalization layer. ;

[0050] The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0051] The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ;

[0052] The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0053] The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ;

[0054] The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ;

[0055] The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ;

[0056] The intermediate features and the intermediate features The residuals are summed to obtain the second enhanced feature. .

[0057] Preferably, the third shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the third enhanced feature. Specifically, it includes the following steps:

[0058] The third shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ;

[0059] The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2.

[0060] The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ;

[0061] The segmentation features Normalized features are obtained after processing by the normalization layer. ;

[0062] The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ;

[0063] The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ;

[0064] The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ;

[0065] The rearrangement feature and the third shallow layer features By summing the residuals, intermediate features can be obtained. ;

[0066] The intermediate features Normalized features are obtained after processing by the normalization layer. ;

[0067] The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0068] The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ;

[0069] The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0070] The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ;

[0071] The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ;

[0072] The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ;

[0073] The intermediate features and the intermediate features The third enhancement feature is obtained by summing the residuals. .

[0074] Preferably, the N dilated convolutional layers of different scales include 3*3 dilated convolutional layers, 5*5 dilated convolutional layers, and 7*7 dilated convolutional layers.

[0075] Preferably, the initial convolutional layer includes a convolutional layer, a normalization layer, and a SiLU activation function layer connected sequentially from input to output.

[0076] Preferably, the segmentation head includes a convolutional layer and a softmax function layer connected sequentially from input to output.

[0077] Preferably, the trained multi-scale global-local fusion remote sensing segmentation network is obtained based on the following total loss function:

[0078] ;

[0079] In the formula, Represents the total loss function; Represents pixel-level multi-class cross-entropy; Indicates Dice loss; The weight coefficients represent the pixel-level multi-class cross-entropy. The weighting coefficients represent the Dice loss.

[0080] Preferably, the pixel-level multi-class cross-entropy and the Dice loss are obtained based on the following formulas:

[0081] ;

[0082] ;

[0083] In the formula, N represents the number of pixels in the remote sensing image; K represents the number of land cover categories; Let represent the true label of the i-th pixel, where if the i-th pixel belongs to the k-th land cover category, then... It equals 1, otherwise it equals 0; This represents the probability value of predicting that the i-th pixel belongs to the k-th land cover category.

[0084] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a remote sensing image segmentation method based on global dependency and local texture fusion attention, which can effectively fuse local texture details and global spatial dependency, achieve high-precision segmentation and high-efficiency inference in complex remote sensing scenes, and significantly improve the ability to identify small targets, fine boundaries and large-scale ground structures. Attached Figure Description

[0085] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0086] Figure 1 This is a schematic diagram of the structure of the multi-scale global-local fusion remote sensing segmentation network provided in this embodiment of the invention;

[0087] Figure 2 This is a schematic diagram of the structure of the global dependency and local texture fusion attention module provided in an embodiment of the present invention. Detailed Implementation

[0088] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0089] This invention discloses a remote sensing image segmentation method based on global dependency and local texture fusion attention, comprising the following steps:

[0090] The remote sensing image to be segmented is input into a trained multi-scale global-local fusion remote sensing segmentation network to obtain a segmented image of the remote sensing image to be segmented; wherein, the multi-scale global-local fusion remote sensing segmentation network adopts a symmetrical encoder-decoder structure and introduces a global dependency and local texture fusion attention module in each scale layer.

[0091] In one or more embodiments, such as Figure 1 As shown, the remote sensing image to be segmented is input into a trained multi-scale global-local fusion remote sensing segmentation network to obtain a segmented image of the remote sensing image to be segmented. Specifically, this includes the following steps:

[0092] The remote sensing image to be segmented is input into the initial convolutional layer to obtain the first shallow feature layer. ;

[0093] In one or more embodiments, the initial convolutional layer includes a convolutional layer, a normalization layer, and a SiLU activation function layer connected sequentially from input to output.

[0094] Specifically: Assume the remote sensing image to be segmented is Where H and W are the height and width of the remote sensing image to be segmented, respectively, and 3 represents the number of channels (specifically, the three RGB channels).

[0095] Then the first shallow layer features ;in, This represents the initial convolutional layer.

[0096] For the first shallow layer features Perform downsampling to obtain the second shallow layer features. ;

[0097] Specifically: ; where Down(.) represents a convolution or pooling operation with a stride of 2, which is used for spatial downsampling.

[0098] The first shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the first enhanced feature. ;

[0099] It should be noted that: the first enhancement feature It retains fine-grained edges such as roads and rooftops while incorporating scene-level contextual dependencies.

[0100] For the second shallow layer features Perform downsampling to obtain the third shallow layer features. ;

[0101] Specifically: ; where Down(.) represents a convolution or pooling operation with a stride of 2, which is used for spatial downsampling.

[0102] The second shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the second enhanced feature. ;

[0103] For the third shallow layer features Perform downsampling to obtain the fourth layer of shallow features. ;

[0104] Specifically: ; where Down(.) represents a convolution or pooling operation with a stride of 2, which is used for spatial downsampling.

[0105] The downsampling hierarchical structure provided by this invention simulates the semantic gradual abstraction process of "fine-grained – mesoscale – large-scale" in remote sensing images: the lower layer focuses on the texture of ground object boundaries, while the higher layer focuses more on the spatial organization of the overall scene.

[0106] The third shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the third enhanced feature. ;

[0107] For the fourth shallow layer features Perform upsampling to obtain the first upsampled feature. ;

[0108] Specifically: Where Up represents bilinear interpolation or transposed convolution upsampling;

[0109] The first upsampled feature and the third enhancement feature By performing element-wise addition, the first fusion feature is obtained. ;

[0110] Specifically: ; where ⊕ represents element-wise addition.

[0111] For the first fusion feature Perform upsampling to obtain the second upsampled feature. ;

[0112] Specifically: Where Up represents bilinear interpolation or transposed convolution upsampling;

[0113] The second upsampled feature and the second enhancement feature By performing element-wise addition, the second fusion feature is obtained. ;

[0114] Specifically: ; where ⊕ represents element-wise addition.

[0115] For the second fusion feature Perform upsampling to obtain the third upsampled feature. ;

[0116] Specifically: Where Up represents bilinear interpolation or transposed convolution upsampling;

[0117] The third upsampling feature With the first enhanced feature By performing element-wise addition, the third fusion feature is obtained. ;

[0118] Specifically: ; where ⊕ represents element-wise addition.

[0119] Regarding the third fusion feature After upsampling, the data is input into the segmentation head to obtain a segmented image of the remote sensing image to be segmented.

[0120] Specifically: First, consider the third fusion feature. Upsampling is performed to obtain output features that fuse multi-level contextual and detail information. Then output features The input is fed into the segmentation head to obtain a segmented image of the remote sensing image to be segmented; where Up represents bilinear interpolation or transposed convolution upsampling.

[0121] In one or more embodiments, the segmentation head includes a 1x1 convolutional layer and a softmax function layer connected sequentially from input to output.

[0122] Specifically: the segmentation head is used to predict land cover categories pixel by pixel, and its output is... ;in, B represents the batch size; K represents the number of land feature categories (such as "buildings, roads, water bodies, vegetation", etc.).

[0123] This invention's multi-scale global-local fusion remote sensing segmentation network extracts multi-scale features from remote sensing images layer by layer during the encoding process. During decoding, it employs a layer-by-layer fusion and attention enhancement mechanism, ensuring both the clarity of target boundaries such as buildings and roads and accurately modeling large-scale dependencies across regions. Compared to the traditional U-Net, this multi-scale global-local fusion remote sensing segmentation network offers advantages in maintaining boundary accuracy and global consistency, making it particularly suitable for segmentation tasks of high-resolution remote sensing images.

[0124] In one or more embodiments, such as Figure 2 As shown, the first shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the first enhanced feature. Specifically, it includes the following steps:

[0125] The first shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ;

[0126] The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2.

[0127] In one or more embodiments, the N dilated convolutional layers of different scales include 3*3 dilated convolutional layers, 5*5 dilated convolutional layers, and 7*7 dilated convolutional layers.

[0128] It is understandable that 3x3, 5x5, and 7x7 dilated convolutional layers can be configured with increasing dilation rates; by using three sets of dilated convolutional layers in parallel, different receptive fields can be covered, ranging from small targets such as rooftops / vehicles to linear features (roads / rivers). Multi-scale dilated convolution can expand the receptive field without significantly increasing computation, taking into account both detail and structure, which is particularly crucial for remote sensing semantic segmentation (such as separating building boundaries and shadows).

[0129] The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ;

[0130] The segmentation features Normalized features are obtained after processing by the normalization layer. ;

[0131] Specifically: Where LN represents the normalization layer;

[0132] The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ;

[0133] Specifically: ;

[0134] in, DWConv represents a normalized layer; DWConv represents a separable convolutional layer; SiLU represents a SiLU activation function layer; SS2D represents an SS2D layer.

[0135] SS2D stands for Selective Scan 2D: Based on Mamba / Vmamba, it extends 1D selective scanning to four-way cross-scanning of an image, capturing long-range dependencies across the entire image with linear complexity, making it ideal for large-format remote sensing feature modeling.

[0136] The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ;

[0137] Specifically: ;in, represents matrix multiplication; SiLU represents the SiLU activation function layer; this branch can emphasize macroscopic spatial organization such as urban-suburban-farmland and long-distance relationships across land features.

[0138] The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ;

[0139] It should be noted that channel shuffle can facilitate cross-branch feature interaction;

[0140] The rearrangement feature and the first shallow layer features By summing the residuals, intermediate features can be obtained. ;

[0141] It should be noted that: intermediate features It preserves the original semantics while integrating multi-scale and global context, which helps to improve the stability of ground feature boundaries.

[0142] The intermediate features Normalized features are obtained after processing by the normalization layer. ;

[0143] The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0144] The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ;

[0145] The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0146] The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ;

[0147] The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ;

[0148] Specifically: ; Where SiLU represents the SiLU activation function layer; Softmax represents the Softmax activation function layer;

[0149] The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ;

[0150] The intermediate features and the intermediate features The first enhanced feature is obtained by summing the residuals. .

[0151] In one or more embodiments, the second shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the second enhanced feature. Specifically, it includes the following steps:

[0152] The second shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ;

[0153] The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2.

[0154] In one or more embodiments, the N dilated convolutional layers of different scales include 3*3 dilated convolutional layers, 5*5 dilated convolutional layers, and 7*7 dilated convolutional layers.

[0155] It is understandable that 3x3, 5x5, and 7x7 dilated convolutional layers can be configured with increasing dilation rates; by using three sets of dilated convolutional layers in parallel, different receptive fields can be covered, ranging from small targets such as rooftops / vehicles to linear features (roads / rivers). Multi-scale dilated convolution can expand the receptive field without significantly increasing computation, taking into account both detail and structure, which is particularly crucial for remote sensing semantic segmentation (such as separating building boundaries and shadows).

[0156] The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ;

[0157] The segmentation features Normalized features are obtained after processing by the normalization layer. ;

[0158] Specifically: Where LN represents the normalization layer;

[0159] The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ;

[0160] Specifically: ;

[0161] in, DWConv represents a normalized layer; DWConv represents a separable convolutional layer; SiLU represents a SiLU activation function layer; SS2D represents an SS2D layer.

[0162] SS2D stands for Selective Scan 2D: Based on Mamba / Vmamba, it extends 1D selective scanning to four-way cross-scanning of an image, capturing long-range dependencies across the entire image with linear complexity, making it ideal for large-format remote sensing feature modeling.

[0163] The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ;

[0164] Specifically: ;in, represents matrix multiplication; SiLU represents the SiLU activation function layer; this branch can emphasize macroscopic spatial organization such as urban-suburban-farmland and long-distance relationships across land features.

[0165] The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ;

[0166] It should be noted that channel shuffle can facilitate cross-branch feature interaction;

[0167] The rearrangement feature and the second shallow layer features By summing the residuals, intermediate features can be obtained. ;

[0168] It should be noted that: intermediate features It preserves the original semantics while integrating multi-scale and global context, which helps to improve the stability of ground feature boundaries.

[0169] The intermediate features Normalized features are obtained after processing by the normalization layer. ;

[0170] The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0171] The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ;

[0172] The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0173] The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ;

[0174] The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ;

[0175] Specifically: ; Where SiLU represents the SiLU activation function layer; Softmax represents the Softmax activation function layer;

[0176] The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ;

[0177] The intermediate features and the intermediate features The residuals are summed to obtain the second enhanced feature. .

[0178] In one or more embodiments, the third shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the third enhanced feature. Specifically, it includes the following steps:

[0179] The third shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ;

[0180] The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2.

[0181] In one or more embodiments, the N dilated convolutional layers of different scales include 3*3 dilated convolutional layers, 5*5 dilated convolutional layers, and 7*7 dilated convolutional layers.

[0182] It is understandable that 3x3, 5x5, and 7x7 dilated convolutional layers can be configured with increasing dilation rates; by using three sets of dilated convolutional layers in parallel, different receptive fields can be covered, ranging from small targets such as rooftops / vehicles to linear features (roads / rivers). Multi-scale dilated convolution can expand the receptive field without significantly increasing computation, taking into account both detail and structure, which is particularly crucial for remote sensing semantic segmentation (such as separating building boundaries and shadows).

[0183] The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ;

[0184] The segmentation features Normalized features are obtained after processing by the normalization layer. ;

[0185] Specifically: Where LN represents the normalization layer;

[0186] The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ;

[0187] Specifically: ;

[0188] in, DWConv represents a normalized layer; DWConv represents a separable convolutional layer; SiLU represents a SiLU activation function layer; SS2D represents an SS2D layer.

[0189] SS2D stands for Selective Scan 2D: Based on Mamba / Vmamba, it extends 1D selective scanning to four-way cross-scanning of an image, capturing long-range dependencies across the entire image with linear complexity, making it ideal for large-format remote sensing feature modeling.

[0190] The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ;

[0191] Specifically: ;in, represents matrix multiplication; SiLU represents the SiLU activation function layer; this branch can emphasize macroscopic spatial organization such as urban-suburban-farmland and long-distance relationships across land features.

[0192] The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ;

[0193] It should be noted that channel shuffle can facilitate cross-branch feature interaction;

[0194] The rearrangement feature and the third shallow layer features By summing the residuals, intermediate features can be obtained. ;

[0195] It should be noted that: intermediate features It preserves the original semantics while integrating multi-scale and global context, which helps to improve the stability of ground feature boundaries.

[0196] The intermediate features Normalized features are obtained after processing by the normalization layer. ;

[0197] The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0198] The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ;

[0199] The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ;

[0200] The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ;

[0201] The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ;

[0202] The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ;

[0203] The intermediate features and the intermediate features The third enhancement feature is obtained by summing the residuals. .

[0204] It should be noted that the multi-scale global-local fusion remote sensing segmentation network is trained using remote sensing images labeled with the land cover category for each pixel.

[0205] In one or more embodiments, a trained multi-scale global-local fusion remote sensing segmentation network is obtained based on the following total loss function:

[0206] ;

[0207] In the formula, Represents the total loss function; Represents pixel-level multi-class cross-entropy; Indicates Dice loss; The weight coefficients represent the pixel-level multi-class cross-entropy. The weighting coefficients represent the Dice loss.

[0208] In one or more embodiments, the pixel-level multi-class cross-entropy and the Dice loss are obtained based on the following formulas:

[0209] ;

[0210] ;

[0211] In the formula, N represents the number of pixels in the remote sensing image; K represents the number of land cover categories; Let represent the true label of the i-th pixel, where if the i-th pixel belongs to the k-th land cover category, then... It equals 1, otherwise it equals 0; This represents the probability value of predicting that the i-th pixel belongs to the k-th land cover category.

[0212] It should be noted that during the training process, the network parameters θ of the multi-scale global-local fusion remote sensing segmentation network are iteratively updated using the backpropagation algorithm.

[0213] The specific update formula is as follows: ;

[0214] Where η represents the learning rate. This represents the gradient of the total loss function with respect to the parameter θ; and This represents the network parameters before and after the update. This process ensures that the network gradually learns the discrimination features of various land features in the remote sensing scene.

[0215] A pixel-level predicted probability map P can be obtained by using a well-trained multi-scale global-local fusion remote sensing segmentation network, and the class with the highest probability is selected from it. The final segmented image is obtained. ,in, Represents image segmentation The predicted land cover category of the pixel in the i-th row and j-th column; This represents the probability that the pixel in the i-th row and j-th column of the pixel-level prediction probability map P is predicted to be the k-th land cover category;

[0216] Image segmentation It can intuitively display the distribution of features such as buildings, roads, farmland, and water bodies, providing support for subsequent remote sensing interpretation and geographic information analysis.

[0217] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0218] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image segmentation method based on global dependency and local texture fusion attention, characterized in that, Includes the following steps: The remote sensing image to be segmented is input into a trained multi-scale global-local fusion remote sensing segmentation network to obtain a segmented image of the remote sensing image to be segmented; wherein, the multi-scale global-local fusion remote sensing segmentation network adopts a symmetrical encoder-decoder structure and introduces a global dependency and local texture fusion attention module in each scale layer; The remote sensing image to be segmented is input into a trained multi-scale global-local fusion remote sensing segmentation network to obtain a segmented image of the remote sensing image to be segmented. The specific steps include: inputting the remote sensing image to be segmented into an initial convolutional layer to obtain first-layer shallow features ; performing down-sampling on the first layer shallow feature to obtain a second layer shallow feature performing down-sampling on the first layer shallow feature to obtain a second layer shallow feature ; The first shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the first enhanced feature. ; For the second shallow layer features Perform downsampling to obtain the third shallow layer features. ; The second shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the second enhanced feature. ; For the third shallow layer features Perform downsampling to obtain the fourth shallow layer features. ; The third shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the third enhanced feature. ; For the fourth shallow layer features Perform upsampling to obtain the first upsampled feature. ; The first upsampled feature and the third enhancement feature By performing element-wise addition, the first fusion feature is obtained. ; For the first fusion feature Perform upsampling to obtain the second upsampled feature. ; The second upsampled feature and the second enhancement feature By performing element-wise addition, the second fusion feature is obtained. ; For the second fusion feature Perform upsampling to obtain the third upsampled feature. ; The third upsampling feature With the first enhanced feature By performing element-wise addition, the third fusion feature is obtained. ; Regarding the third fusion feature After upsampling, the image is input into the segmentation head to obtain a segmented image of the remote sensing image to be segmented; The first shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the first enhanced feature. Specifically, it includes the following steps: The first shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ; The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2. The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ; The segmentation features Normalized features are obtained after processing by the normalization layer. ; The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ; The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ; The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ; The rearrangement feature and the first shallow layer features By summing the residuals, intermediate features can be obtained. ; The intermediate features Normalized features are obtained after processing by the normalization layer. ; The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ; The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ; The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ; The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ; The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ; The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ; The intermediate features and the intermediate features The first enhanced feature is obtained by summing the residuals. ; The second shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the second enhanced feature. Specifically, it includes the following steps: The second shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ; The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2. The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ; The segmentation features Normalized features are obtained after processing by the normalization layer. ; The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ; The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ; The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ; The rearrangement feature and the second shallow layer features By summing the residuals, intermediate features can be obtained. ; The intermediate features Normalized features are obtained after processing by the normalization layer. ; The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ; The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ; The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ; The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ; The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ; The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ; The intermediate features and the intermediate features The residuals are summed to obtain the second enhanced feature. ; The third shallow layer features The input is processed by the global dependency and local texture fusion attention module to obtain the third enhanced feature. Specifically, it includes the following steps: The third shallow layer features Perform average segmentation along the channel dimension to obtain segmentation features. and segmentation features ; The segmentation features After being processed sequentially by batch normalization layers and convolutional layers, the results are then processed by N dilated convolutional layers of different scales to obtain N dilated convolution results of different scales; where N is greater than or equal to 2. The N dilated convolution results at different scales are concatenated and then processed sequentially through an activation function layer and a batch normalization layer to obtain local multi-scale features. ; The segmentation features Normalized features are obtained after processing by the normalization layer. ; The normalized features The weight matrix is ​​obtained by sequentially processing the layers through a depthwise separable convolutional layer, an activation function layer, an SS2D layer, and a normalization layer. ; The normalized features After processing by the activation function layer, and then with the weight matrix Perform matrix multiplication to obtain global features. ; The local multi-scale features and the global features Channel-level concatenation is performed followed by channel rearrangement to obtain rearranged features. ; The rearrangement feature and the third shallow layer features By summing the residuals, intermediate features can be obtained. ; The intermediate features Normalized features are obtained after processing by the normalization layer. ; The normalized features The weight matrix is ​​obtained after channel-by-channel global average pooling. ; The normalized features Intermediate features are obtained by processing with depthwise separable convolutional layers. ; The intermediate features The weight matrix is ​​obtained after channel-by-channel global average pooling. ; The weight matrix and the weight matrix The difference is then processed through an activation function layer to obtain the weight matrix. ; The weight matrix and the weight matrix After matrix multiplication, the matrix is ​​processed through an activation function layer to obtain the weight matrix. ; The weight matrix and the intermediate features After matrix multiplication, the data is processed through activation function layers and normalization layers to obtain intermediate features. ; The intermediate features and the intermediate features The third enhancement feature is obtained by summing the residuals. .

2. The remote sensing image segmentation method based on global dependency and local texture fusion attention as described in claim 1, characterized in that, The N dilated convolutional layers of different scales include 3*3 dilated convolutional layers, 5*5 dilated convolutional layers, and 7*7 dilated convolutional layers.

3. The remote sensing image segmentation method based on global dependency and local texture fusion attention as described in claim 1, characterized in that, The initial convolutional layer includes a convolutional layer, a normalization layer, and a SiLU activation function layer connected sequentially from input to output.

4. The remote sensing image segmentation method based on global dependency and local texture fusion attention as described in claim 1, characterized in that, The segmentation head includes convolutional layers and softmax function layers connected sequentially from input to output.

5. The remote sensing image segmentation method based on global dependency and local texture fusion attention as described in claim 1, characterized in that, The trained multi-scale global-local fusion remote sensing segmentation network is obtained based on the following total loss function: ; In the formula, Represents the total loss function; Represents pixel-level multi-class cross-entropy; Indicates Dice loss; The weight coefficients represent the pixel-level multi-class cross-entropy. The weighting coefficients represent the Dice loss.

6. The remote sensing image segmentation method based on global dependency and local texture fusion attention as described in claim 5, characterized in that, The pixel-level multi-class cross-entropy and the Dice loss are obtained based on the following formulas: ； ； In the formula, N represents the number of pixels in the remote sensing image; K represents the number of land cover categories; Let represent the true label of the i-th pixel, where if the i-th pixel belongs to the k-th land cover category, then... It equals 1 if it is equal to 1 otherwise it equals 0; This represents the probability value of predicting that the i-th pixel belongs to the k-th land cover category.

Images