Satellite remote sensing image semantic segmentation system and method

Abstract

This application relates to the field of artificial intelligence technology and discloses a semantic segmentation system and method for satellite remote sensing images. The system includes: an encoding module configured to extract multi-scale features of the target image; a feature fusion module configured to fuse the original feature maps at various scales through linear attention calculation to obtain a first fused feature, and then split it to generate multiple mixed feature maps with the same scale as the original feature maps; a collaborative decoding module configured to extract spatial domain features and frequency domain features from the original feature maps and the mixed feature maps of the same scale, and perform cross-attention calculation to iteratively fuse spatial domain information and frequency domain information to generate a first tensor; and an output module configured to process the first tensor to obtain the segmentation result of the target image. This system can improve the recognition accuracy of detailed textures and small targets in remote sensing images, and improve the accuracy of semantic segmentation of remote sensing images.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a satellite remote sensing image semantic segmentation system and method. Background Technology

[0002] Semantic segmentation of remote sensing images, a key task in remote sensing image understanding, assigns semantic categories to pixels in remote sensing images, enabling accurate identification and regional division of surface targets, and making the analysis of remote sensing data more intuitive and clear. In recent years, the introduction of the Transformer architecture, represented by Vision Transformer (ViT), has significantly improved the model's understanding of the global semantics of images.

[0003] Limited by the characteristics of the Transformer architecture, current semantic segmentation schemes for satellite remote sensing images struggle to distinguish between rooftops and roads with similar textures, as well as targets in shadow areas, in complex remote sensing urban scenes. They also lack the ability to recognize details such as object edges and textures, often resulting in blurred edges and segmentation errors. Furthermore, when target scales vary significantly, the recognition of small targets in remote sensing images is easily affected by noise, leading to misjudgments and impacting segmentation accuracy. Therefore, improving the recognition accuracy of detailed textures and small targets in remote sensing images, and thus enhancing the accuracy of semantic segmentation, is a problem that needs to be solved. Summary of the Invention

[0004] The purpose of this application is to provide a satellite remote sensing image semantic segmentation system and method to improve the recognition accuracy of detailed textures and small targets in remote sensing images and enhance the accuracy of semantic segmentation.

[0005] To achieve the above objectives, the technical solution of this application is as follows: In a first aspect, embodiments of this application provide a satellite remote sensing image semantic segmentation system, the system comprising: The encoding module is configured to extract multi-scale features from the target image to obtain multiple raw feature maps; The feature fusion module is configured to calculate and fuse the original feature maps at various scales through linear attention to obtain a first fused feature; and to split the first fused feature to generate multiple mixed feature maps with the same scale as the original feature maps. The collaborative decoding module is configured to extract spatial domain features and frequency domain features from each original feature map and a mixed feature map of the same scale; the spatial domain features include global features and local features; the frequency domain features include high-frequency features and low-frequency features; cross-attention calculation is performed on the spatial domain features and frequency domain features at each scale to iteratively fuse spatial domain information and frequency domain information to generate the first tensor; The output module is configured to process the first tensor to obtain the segmentation result of the target image.

[0006] Optionally, the feature fusion module includes: The pooling layer is configured to perform max pooling on each of the original feature maps; The connection layer is configured to recombine the pooling results of the individual original feature maps to obtain concatenated features; An attention fusion module is configured to perform linear attention calculation on the spliced ​​features to obtain a first fused feature; The splitting layer is configured to split the first fused feature based on the scale corresponding to each original feature map, so as to obtain a fused sub-feature map with the same scale as each original feature map. The gating layer is configured to perform weighted fusion of the original feature maps and fused sub-feature maps at various scales to generate a hybrid feature map at the corresponding scale.

[0007] Optionally, the attention fusion module is configured to perform linear attention calculation on the spliced ​​features to obtain a first fused feature, including: The concatenated features are mapped to Q vector, K vector, and V vector; where Q is the query, K is the key, and V is the value. The K vector and V vector are scaled down to shorten their length and increase the number of channels, resulting in compressed K vector and compressed V vector. Based on the Q vector, the compressed K vector, and the compressed V vector, according to The linear attention is calculated using a specific computational method to obtain intermediate results; The intermediate results are processed by a linear projection layer to recover the number of channels, thereby obtaining the first intermediate feature; Repeat the following steps to iteratively optimize the first intermediate feature: map the first intermediate feature to Q vector, K vector, and V vector, according to... The linear attention is calculated using a computational method to obtain optimized intermediate features; After multiple iterations and optimizations, the optimized intermediate feature obtained is determined as the first fusion feature.

[0008] Optionally, the collaborative decoding module includes: The frequency domain branch is configured to separate the feature information of the hybrid feature map in different frequency domains to obtain high-frequency features and low-frequency features; The global branch is configured to process the hybrid feature map through a sliding window, extract feature information in the spatial and channel dimensions, and fuse them to obtain global features in the spatial domain. Local branches are configured to iteratively extract local features of the spatial domain from the hybrid feature map using pooling windows of various sizes; The collaborative decoding module is configured to fuse the high-frequency features and the local features of the spatial domain through cross-attention calculation to obtain a first sub-feature; and to fuse the low-frequency features and the global features of the spatial domain through cross-attention calculation to obtain a second sub-feature; and to concatenate the first sub-feature and the second sub-feature in the channel dimension to obtain an optimized hybrid feature map.

[0009] Optionally, the frequency domain branch is configured to separate the feature information of the hybrid feature map in different frequency domains to obtain high-frequency features and low-frequency features, including: The hybrid feature map is separated in multiple frequency domains to obtain multiple high-frequency sub-features and one low-frequency sub-feature. All high-frequency sub-features are spliced ​​together to obtain the high-frequency signal; The high-frequency signal is subjected to nonlinear mapping processing to obtain the corresponding high-frequency features; and the low-frequency sub-features are subjected to nonlinear mapping processing to obtain the corresponding low-frequency features.

[0010] Optionally, the global branch is configured to process the hybrid feature map through a sliding window, extract feature information in the spatial and channel dimensions, and fuse them to obtain global features in the spatial domain, including: Based on a sliding window, the feature data in the hybrid feature map are linearly projected to obtain the corresponding Q matrix, K matrix, and V matrix; where Q is the query, K is the key, and V is the value. Based on the Q matrix, K matrix, and V matrix, according to The calculation method calculates the corresponding intermediate results; Calculate the product of the rows of the Q matrix and the columns of the K matrix generated by the linear projection; The product is subjected to max pooling and average pooling respectively, and the pooling results are fused by element-wise multiplication to obtain the channel weight map; The intermediate results are processed based on the channel weight map, and then processed through strip pooling and large kernel convolutional layers to perform long-distance information fusion between windows in the horizontal and vertical directions, respectively, to obtain global features in the spatial domain.

[0011] Optionally, the local branch is configured to iteratively extract local features in the spatial domain from the mixed feature map using pooling windows of various sizes, including: Basic features are extracted from the hybrid feature map through convolutional layers and normalization processing; The basic features are subjected to multi-scale max pooling operations through a pyramid pooling layer, and feature data from each scale are fused to obtain fused multi-scale local features. By optimizing the fused multi-scale local features through a bottleneck layer, the local features of the spatial domain corresponding to the hybrid feature map are obtained.

[0012] Optionally, the system further includes: The multidimensional feature refinement module is configured to fuse shallow and deep features in the first tensor based on multiple dimensions to obtain a second tensor; the multiple dimensions include at least two of the following dimensions: channel dimension, spatial dimension, and multi-scale dimension; The output module is further configured to process the second tensor to obtain the segmentation result of the target image.

[0013] Optionally, the multidimensional feature refinement module includes: The channel branch is configured to perform average pooling on the first tensor to generate a channel attention map; the channel attention map is then processed through multiple layers of convolution and activation, and a channel weight matrix is ​​generated by sigmoid activation; the channel weight matrix is ​​then multiplied element-wise with the first tensor to generate a first sub-feature map; The spatial branch is configured to perform average pooling and max pooling on the first tensor respectively, and concatenate the resulting spatial feature maps to generate a spatial attention map; the spatial attention map is then processed by convolution and sigmoid activation to obtain a spatial weight matrix; the spatial weight matrix and the first tensor are multiplied element-wise to generate a second sub-feature map. The multi-scale branch is configured to pass the first tensor through a convolutional layer to obtain the corresponding intermediate result; pass the intermediate result through three grouped convolutional layers with different dilation rates respectively, and fuse the multi-scale feature information to obtain the third sub-feature map; The multidimensional feature refinement module is configured to fuse the first sub-feature map, the second sub-feature map, and the third sub-feature map to obtain the second tensor.

[0014] Secondly, embodiments of this application provide a satellite remote sensing image semantic segmentation method, applied to the system provided in the first aspect of this application. The method includes: Extract multi-scale features from the target image to obtain multiple original feature maps; The first fused feature is obtained by calculating and fusing the original feature maps at various scales using linear attention; The first fused feature is split to generate multiple mixed feature maps with the same scale as the original feature maps; Spatial domain features and frequency domain features are extracted from each original feature map and a mixed feature map of the same scale; the spatial domain features include global features and local features; the frequency domain features include high-frequency features and low-frequency features. Cross-attention calculation is performed on spatial domain features and frequency domain features at various scales to iteratively fuse spatial domain information and frequency domain information to generate the first tensor; The first tensor is processed to obtain the segmentation result of the target image.

[0015] The satellite remote sensing image semantic segmentation system provided in this application extracts multi-scale features from the target image through an encoding module, obtaining multiple original feature maps at different scales. Then, before decoding, linear attention is used to fuse feature information from each level of the original feature maps, and the scale of the feature maps is restored. Compared with the traditional semantic segmentation scheme that fuses features through simple skip connections (concatenation or addition) between the encoder and decoder, this application uses linear attention to adaptively filter and fuse feature information at each level before decoding, thereby avoiding shallow noise from interfering with deep semantics and improving the model's accuracy in recognizing small targets.

[0016] Based on this, the collaborative decoding module extracts hybrid feature information from the spatial domain and frequency domain respectively, and uses the cross-attention mechanism to achieve complementary fusion of spatial domain and frequency domain features, thereby enhancing the model's ability to perceive and distinguish complex ground features and texture details, further improving the model's ability to distinguish edge details, targets in shadow areas and targets with similar textures, and improving the accuracy of semantic segmentation of remote sensing images. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of a satellite remote sensing image semantic segmentation system proposed in an embodiment of this application; Figure 2 This is an architecture diagram of a satellite remote sensing image semantic segmentation system according to an embodiment of this application; Figure 3 (a) and (b) are flowcharts of the attention fusion module calculating linear attention in one embodiment of this application, respectively; Figure 4 This is a flowchart of the collaborative decoding module performing feature extraction and fusion in the spatial and frequency domains in one embodiment of this application; Figure 5This is an architecture diagram of the global branch of the collaborative decoding module in one embodiment of this application; Figure 6 This is an architecture diagram of a partial branch of the collaborative decoding module in one embodiment of this application; Figure 7 This is an architecture diagram of the multi-dimensional feature refinement module in one embodiment of this application, which optimizes feature data through parallel branches; Figure 8 This is a flowchart of a satellite remote sensing image semantic segmentation method proposed in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0021] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects as detailed in this application.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0024] Current Transformer-based image semantic segmentation schemes are limited to focusing on low-frequency semantic information in the spatial domain during feature learning. When faced with features that have similar textures but vastly different frequency domain characteristics (such as rooftops and roads), their ability to distinguish between them is limited. Furthermore, the encoder and decoder of these models often use direct skip connections for feature fusion. While this approach can transmit shallow details, it also introduces a large amount of background noise and non-semantic information. Due to the lack of effective feature filtering, this noise interferes with the decoder's accurate feature recovery of multi-scale targets.

[0025] This application utilizes a feature fusion module to adaptively filter and fuse multi-scale features using linear attention, reducing noise interference in the feature map and achieving accurate reconstruction of small objects and edge details during the decoding stage. In the decoding stage, a collaborative decoding module captures spatial and frequency domain feature information from the multi-scale feature map. Leveraging the complementary advantages of spatial and frequency domain features, cross-attention is used to achieve efficient matching and semantic alignment of spatial and frequency domain features, improving the model's ability to perceive and distinguish similar textures and shadowed features.

[0026] The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] Figure 1 This is a schematic diagram of a satellite remote sensing image semantic segmentation system 100 according to an embodiment of this application. Figure 1 As shown, the system includes: Encoding module 101 is configured to extract multi-scale features from the target image to obtain multiple original feature maps; The feature fusion module 102 is configured to calculate and fuse the original feature maps at various scales through linear attention to obtain a first fused feature; and to split the first fused feature to generate multiple mixed feature maps with the same scale as the original feature maps. The collaborative decoding module 103 is configured to extract spatial domain features and frequency domain features from each original feature map and a mixed feature map of the same scale; the spatial domain features include global features and local features; the frequency domain features include high-frequency features and low-frequency features; cross-attention calculation is performed on the spatial domain features and frequency domain features at each scale to iteratively fuse spatial domain information and frequency domain information to generate the first tensor; The output module 104 is configured to process the first tensor to obtain the segmentation result of the target image.

[0028] In this embodiment, the system is based on the Transformer architecture and employs four encoding modules to extract continuous features from the target image. The system uses a feature fusion module for feature fusion. Unlike traditional simple feature concatenation, this embodiment uses linear attention computation to achieve adaptive filtering of multi-scale feature maps. After obtaining the first fused feature, it is then re-split into mixed feature maps of the same scale according to the scale of the original feature maps output by the encoding modules. Each mixed feature map generated after fusion contains relevant feature information from all scales (shallow to deep). Furthermore, in the subsequent decoding stage, the mixed feature maps are decoded, allowing the model to capture more detailed features than the original feature maps, effectively suppressing the influence of shallow noise on the model's perception of small targets, edges, textures, and other details.

[0029] During the decoding phase, the system iteratively processes the mixed feature maps at different scales using three collaborative decoding modules. The collaborative decoding module extracts global and local features in the spatial domain, as well as high-frequency and low-frequency features in the frequency domain, from the original and mixed feature maps, respectively. Leveraging the complementarity of global and low-frequency features, and local and high-frequency features, a cross-attention mechanism iteratively fuses the complementary feature information in the spatial and frequency domains to obtain the first tensor used for prediction. The output module assigns semantic categories to each pixel in the target image based on the first tensor, completing the semantic segmentation of the target image and outputting the segmentation result.

[0030] This embodiment integrates multi-scale features through a linear attention mechanism between the encoding and decoding stages. This enables the model to effectively capture small objects and edge details in the initial decoding stage, mitigating shallow noise interference and improving the recognition accuracy of small targets and edge details. In the decoding stage, this embodiment combines frequency domain features with spatial domain features, allowing the model to simultaneously capture target information from both the spatial and frequency domains. The capture of frequency domain features effectively distinguishes targets with similar textures or those in shadow areas, improving the boundary clarity of the model's semantic segmentation, reducing false positives, and enhancing semantic segmentation accuracy.

[0031] As one embodiment of this application, the feature fusion module includes: The pooling layer is configured to perform max pooling on each of the original feature maps; The connection layer is configured to recombine the pooling results of the individual original feature maps to obtain concatenated features; An attention fusion module is configured to perform linear attention calculation on the spliced ​​features to obtain a first fused feature; The splitting layer is configured to split the first fused feature based on the scale corresponding to each original feature map, so as to obtain a fused sub-feature map with the same scale as each original feature map. The gating layer is configured to perform weighted fusion of the original feature maps and fused sub-feature maps at various scales to generate a hybrid feature map at the corresponding scale.

[0032] Figure 2 This is an architecture diagram of a satellite remote sensing image semantic segmentation system according to one embodiment of this application. Figure 2 As shown, this system adopts the UNet structure. After inputting the target image, it first passes through 4 Swing Block encoding layers, and then uses a sliding window mechanism to continuously extract multi-scale features from the target image.

[0033] Subsequently, the Feature Fusion Module (MGB) takes the encoded features at each level as input, performs max pooling on each original feature map through pooling layers, and reassembles them through connection layers to obtain concatenated features. In the attention fusion module, normalization and linear attention are performed on the concatenated features to fuse shallow and deep feature information at different scales to obtain the first fused feature. Then, it is reassembled into multi-level fused sub-feature maps of the same scale as the original feature maps through splitting layers. The gating layer uses the same number of gating modules as the encoding module to adaptively weight and fuse the original feature maps and fused sub-feature maps at each scale to generate mixed feature maps of the corresponding scale for subsequent decoding stages.

[0034] In this embodiment, the gating module uses the fused feature map of the original feature map and the hybrid feature map as the hybrid feature map used in the subsequent decoding stage to mitigate semantic biases that may occur during the linear attention feature fusion process. Specifically, for each part of the first fused feature... Combining the multi-scale original feature maps output from each stage of the encoder By fusing the data using learnable weights, a corresponding hybrid feature map is generated. ,as follows: ; in, , , These are learnable weights; This is the feature map of the i-th scale output by the encoder; This is the fused sub-feature map of the i-th scale output by the split layer.

[0035] During the decoding phase, the system iteratively optimizes the hybrid feature maps at each level and their corresponding original feature maps through three collaborative decoding modules (SFCM modules). It utilizes cross-attention to capture complementary features in the spatial and frequency domains, ultimately aggregating them to obtain the first tensor. The output module predicts the category of each pixel in the target image based on the first tensor and generates semantic segmentation results.

[0036] In one embodiment of this application, the attention fusion module is configured to perform linear attention calculation on the spliced ​​features to obtain a first fused feature, including: The concatenated features are mapped to Q vector, K vector, and V vector; where Q is the query, K is the key, and V is the value. The K vector and V vector are scaled down to shorten their length and increase the number of channels, resulting in compressed K vector and compressed V vector. Based on the Q vector, the compressed K vector, and the compressed V vector, according to The linear attention is calculated using a specific computational method to obtain intermediate results; The intermediate results are processed by a linear projection layer to recover the number of channels, thereby obtaining the first intermediate feature; Repeat the following steps to iteratively optimize the first intermediate feature: map the first intermediate feature to Q vector, K vector, and V vector, according to... The linear attention is calculated using a computational method to obtain optimized intermediate features; After multiple iterations and optimizations, the optimized intermediate feature obtained is determined as the first fusion feature.

[0037] In one embodiment, the attention fusion module uses four cascaded attention computation modules to fuse information at various scales in the spliced ​​features. For example... Figure 2 As shown, each attention calculation module consists of a first normalization layer, a linear attention layer, a second normalization layer, and an FFN feedforward network. The FFN feedforward network consists of two fully connected layers and one non-linear activation function (such as ReLU), providing the model with non-linear capabilities. The first fully connected layer transforms the input features into high-dimensional features for deep feature extraction; the second fully connected layer transforms the high-dimensional features into the required dimensions. The linear attention layer employs a spatial dimensionality reduction Condensed Attention mechanism, first mapping the concatenated input features to a query vector. Key vector and value vector To reduce computational complexity, the K and V vectors are scaled before computation to obtain the compressed K vector. and the compressed V vector The details are as follows: ; .

[0038] Where r is the compression scale; C is the number of channels; and N is the sequence length. After compression, the length of the vector sequence changes from N to... At the same time, the number of channels increases from C to rC.

[0039] Figure 3 (a) and (b) are flowcharts of the attention fusion module calculating linear attention in one embodiment of this application. To further reduce the computational complexity of linear attention, in this embodiment, during the calculation of the first group of linear attention, as follows... Figure 3 As shown in (a), the compressed version is used. Vector and Vector, according to The order is to first aggregate the Q vector and... The vector is transposed, and the result is passed through a softmax layer before being joined with the vector. The vectors are used for calculation to obtain the first intermediate feature. After calculating the first intermediate feature using linear attention, the channel depth is restored to C through linear projection to ensure consistency in subsequent iterative calculations. The remaining three sets of linear attention calculations after the first set are as follows: Figure 3 As shown in (b), the intermediate features are used as input Q, and the uncompressed K vector and V vector are used according to... The calculation method first aggregates the transpose of the K vector and the V vector after the softmax layer. The result is then calculated with the Q vector after the softmax layer, outputting the optimized intermediate features as input to the next linear attention calculation module. Finally, after processing by four sets of attention calculation modules, the first fused feature is obtained. This calculation method reduces the complexity of linear attention calculation from... Reduce to This improves the efficiency of the model in semantic segmentation of target images.

[0040] As one embodiment of this application, the collaborative decoding module includes: The frequency domain branch is configured to separate the feature information of the hybrid feature map in different frequency domains to obtain high-frequency features and low-frequency features; The global branch is configured to process the hybrid feature map through a sliding window, extract feature information in the spatial and channel dimensions, and fuse them to obtain global features in the spatial domain. Local branches are configured to iteratively extract local features of the spatial domain from the hybrid feature map using pooling windows of various sizes; The collaborative decoding module is configured to fuse the high-frequency features and the local features of the spatial domain through cross-attention calculation to obtain a first sub-feature; and to fuse the low-frequency features and the global features of the spatial domain through cross-attention calculation to obtain a second sub-feature; and to concatenate the first sub-feature and the second sub-feature in the channel dimension to obtain an optimized hybrid feature map.

[0041] In one embodiment, the collaborative decoding module (SFCM module) leverages the complementary properties between the spatial and frequency domains and achieves efficient matching and semantic alignment of spatial and frequency domain features through cross-attention. This module employs a parallel three-branch architecture, including: a frequency domain branch using discrete wavelet transform, a global branch based on Swing Transformer, and a local branch.

[0042] Figure 4 This is a flowchart illustrating feature extraction and fusion in the spatial and frequency domains by the collaborative decoding module in one embodiment of this application. Figure 4 As shown, the frequency domain branch extracts high-frequency features from the input hybrid feature map through discrete wavelet transform. Low-frequency characteristics The global branch extracts spatial and channel dimension feature information from the highly mixed feature map through a sliding window mechanism and fuses them to obtain global features in the spatial domain. The local branch uses multiple pooling windows of different sizes to extract local features in the spatial domain from the input blended feature map. .

[0043] Subsequently, the collaborative decoding module utilizes a cross-attention mechanism to divide several features into two groups based on their complementary properties. Specifically, high-frequency features accurately reflect local changes, while local features contain rich feature information; the two complement each other in the frequency and spatial domains. The local features in the spatial domain... High-frequency characteristics in the frequency domain The features are grouped and fused to generate the first sub-feature. Low-frequency features retain global information but lack detail, while global features in the spatial domain contain rich boundary details. Therefore, the global features in the spatial domain are... Low-frequency characteristics in the frequency domain The features are grouped together and fused to generate a second sub-feature. Finally, the first and second sub-features are concatenated at the channel dimension through a connection layer to achieve efficient matching and semantic alignment of spatial and frequency domain features.

[0044] When performing cross-attention calculation on the spatial domain features and frequency domain features of each group, multi-scale convolution is used to map the two features to... , , ,as well as , , The cross-attention between the two is calculated to achieve cross-domain semantic alignment. The calculation process for cross-attention is as follows: ; .

[0045] Where d is the scale scaling factor. Where C is the number of channels of the input features for cross-attention; h is the number of cross-attention heads; , These are two attention maps calculated from spatial domain features and frequency domain features through cross-attention.

[0046] Subsequently, the two calculated attention maps are concatenated along the channel dimension to obtain an aligned output tensor, which serves as the input to the next SFCM module. After multiple iterations, the first tensor is finally generated.

[0047] In one embodiment of this application, the frequency domain branch is configured to separate the feature information of the hybrid feature map in different frequency domains to obtain high-frequency features and low-frequency features, including: The hybrid feature map is separated in multiple frequency domains to obtain multiple high-frequency sub-features and one low-frequency sub-feature. All high-frequency sub-features are spliced ​​together to obtain the high-frequency signal; The high-frequency signal is subjected to nonlinear mapping processing to obtain the corresponding high-frequency features; and the low-frequency sub-features are subjected to nonlinear mapping processing to obtain the corresponding low-frequency features.

[0048] In one embodiment, the frequency domain branch uses Haar wavelet transform to separate key information of the input feature map in different frequency domains, resulting in three high-frequency sub-features. , and and a low-frequency sub-feature Then, the three high-frequency sub-features are spliced ​​together to generate a high-frequency signal.

[0049] Subsequently, the high-frequency signal and low-frequency sub-features are respectively subjected to nonlinear mapping processing for feature reconstruction and channel compression, thereby filtering out redundant information and obtaining the final high-frequency features. and low frequency characteristics The calculation process is as follows: ; .

[0050] in, and Representing low-frequency and high-frequency features respectively, with a scale of ; Indicates batch normalization; Indicates the kernel size as Convolution operations; This indicates a channel splicing operation.

[0051] Optionally, in practical applications, the frequency domain branch can also use Fourier transform, Gabor, and other methods to extract high-frequency and low-frequency features.

[0052] In this embodiment, the frequency domain branch uses discrete wavelet transform to effectively decompose the input feature map into high-frequency and low-frequency components, thereby extracting edge texture and global structural information respectively. By deeply fusing the extracted frequency domain features with spatial domain features, the model's ability to perceive complex terrain boundaries and the overall semantic segmentation accuracy are enhanced.

[0053] In one embodiment of this application, the global branch is configured to process the hybrid feature map through a sliding window, extract feature information in the spatial and channel dimensions, and fuse them to obtain global features in the spatial domain, including: Based on a sliding window, the feature data in the hybrid feature map are linearly projected to obtain the corresponding Q matrix, K matrix, and V matrix; where Q is the query, K is the key, and V is the value. Based on the Q matrix, K matrix, and V matrix, according to The calculation method calculates the corresponding intermediate results; Calculate the product of the rows of the Q matrix and the columns of the K matrix generated by the linear projection; The product is subjected to max pooling and average pooling respectively, and the pooling results are fused by element-wise multiplication to obtain the channel weight map; The intermediate results are processed based on the channel weight map, and then processed through strip pooling and large kernel convolutional layers to perform long-distance information fusion between windows in the horizontal and vertical directions, respectively, to obtain global features in the spatial domain.

[0054] Figure 5 This is an architecture diagram of the global branch of the collaborative decoding module in one embodiment of this application. For example... Figure 5 As shown, the hybrid feature map is input to the PA layer, where positional encoding is performed using a pixel attention method. After passing through a sigmoid function, the output is multiplied element-wise with the original input. Intermediate data is batch normalized, and a sliding window mechanism is used to collect global features in the spatial domain. To further accelerate the computational efficiency of feature acquisition, this embodiment introduces linear multi-head self-attention in the global branch, mapping the feature sequence within the window to Q, K, and V, and then... Figure 3 As shown in (b) The computation is performed in the order of operations, thus reducing the computational complexity of window-based multi-head self-attention from... Reduce to , where N is the sequence length within the sliding window.

[0055] Considering that linear attention computation only learns the contextual relationships of positional information from the spatial dimension, this embodiment also introduces channel attention computation to supplement the dependencies between channels. While calculating linear attention using Q, K, and V, channel attention is also calculated based on Q and K. After obtaining the respective attention calculation results, the two attention calculation results are fused through element-wise multiplication. This allows the global branch to simultaneously capture rich feature information from both the spatial and channel dimensions of the input feature map, further improving the model's ability to capture details in the image. The channel attention calculation method is as follows: ; in, Indicates channel attention; This indicates max pooling; For average pooling; This is a projection operation along the row direction; This is a projection operation along the column direction; This represents element-wise addition. This represents element-wise multiplication; Let i be the query vector for the i-th position in the n-th channel; Let be the key vector at position i in channel n.

[0056] The fused attention calculation result is processed through a 1×1 convolution operation, followed by input into horizontal and vertical pooling layers to complete long-range information fusion. In this embodiment, a convolution method different from the complex SW-Trans module in traditional semantic segmentation schemes is adopted. More efficient strip pooling is used to perform long-range information fusion in both the horizontal (kernel size 1×S; S is the sliding window size) and vertical (kernel size S×1) directions, and fusion is further performed through a large-kernel convolutional layer (kernel size S×S), thereby achieving efficient contextual interaction between windows and improving the feature capture efficiency of global branches. The fused features output from the large-kernel convolutional layer are element-wise added to the feature data before attention calculation, then sequentially passed through a batch normalization layer and a multilayer perceptron (MLP), and finally fused with the feature data before batch normalization to obtain the global features in the spatial domain. .

[0057] As one embodiment of this application, the local branch is configured to iteratively extract local features of the spatial domain from the mixed feature map using pooling windows of various sizes, including: Basic features are extracted from the hybrid feature map through convolutional layers and normalization processing; The basic features are subjected to multi-scale max pooling operations through a pyramid pooling layer, and feature data from each scale are fused to obtain fused multi-scale local features. By optimizing the fused multi-scale local features through a bottleneck layer, the local features of the spatial domain corresponding to the hybrid feature map are obtained.

[0058] Figure 6 This is an architecture diagram of a partial branch of the collaborative decoding module in one embodiment of this application. For example... Figure 6 As shown, the local branches employ pooling windows of various sizes to capture multi-scale local details in the mixed feature map. First, the input mixed feature map undergoes a 3×3 convolutional layer and batch normalization to extract basic features. For the basic features... The bottleneck layer (BL layer) obtains intermediate results through convolution operations of two scales: 1×1 and 3×3. : ; The pyramid pooling layer (SSP layer) uses three different scale max pooling kernels (5×5, 9×9, and 13×13) to process the input data to extract local features. Multi-scale information fusion is achieved within the pyramid pooling layer, outputting intermediate results. : ; in, Indicates a channel splicing operation; Indicates the kernel size as Max pooling operation.

[0059] After the output of the pyramid pooling layer passes through a bottleneck layer, it is fused with the input data that has only undergone one 3×3 convolution layer through a convolutional layer to obtain local features in the spatial domain.

[0060] Local features of the output spatial domain in local branches The calculation is as follows: ; in, Indicates the bottleneck layer; This represents the spatial pyramid pooling operation; Indicates a channel splicing operation; Indicates the kernel size as Convolution operations; This indicates batch normalization.

[0061] As one embodiment of this application, the system further includes: The multidimensional feature refinement module is configured to fuse shallow and deep features in the first tensor based on multiple dimensions to obtain a second tensor; the multiple dimensions include at least two of the following dimensions: channel dimension, spatial dimension, and multi-scale dimension; The output module is further configured to process the second tensor to obtain the segmentation result of the target image.

[0062] Traditional image semantic segmentation schemes often use simple convolutional layers to map features in the output stage of the network. They lack in-depth refinement and optimization of the final feature map in multiple dimensions such as channels, space and multiple scales. Due to insufficient feature capture accuracy, it is difficult to fully correct pixel-level prediction errors, resulting in the segmentation results being not smooth at the edges of objects, and small targets are prone to breakage or misjudgment.

[0063] To improve the model's accuracy in recognizing and segmenting object edges and small targets, this embodiment introduces a multi-dimensional feature refinement module after the decoding stage. This module further refines the feature information in the first tensor based on at least two dimensions: channel dimension, spatial dimension, and multi-scale dimension. It performs weighted fusion of shallow and deep features, improving the model's ability to represent multi-scale ground features and obtaining a second tensor. Furthermore, the output module processes the second tensor to further improve segmentation accuracy.

[0064] In this embodiment, the system utilizes linear attention in the feature fusion module and a gating mechanism to effectively fuse multi-scale features extracted from each stage of the encoding module. This enhances the detailed information contained in the feature maps at each level during the decoding stage, significantly improving the model's ability to represent multi-scale ground features and providing more refined and richer remote sensing image features for the decoder stage. The collaborative decoding module acquires spatial and frequency domain features in parallel to fully exploit and utilize the complementarity of the two domains, effectively enhancing the model's target consistency and class discrimination capabilities in complex scenes and improving the recognition effect on complex ground features. The model further improves the accuracy and detail representation of the remote sensing image semantic segmentation results by performing weighted fusion and fine-grained optimization on the final generated feature tensor through a multi-dimensional feature refinement module.

[0065] As one embodiment of this application, the multidimensional feature refinement module includes: The channel branch is configured to perform average pooling on the first tensor to generate a channel attention map; the channel attention map is then processed through multiple layers of convolution and activation, and a channel weight matrix is ​​generated by sigmoid activation; the channel weight matrix is ​​then multiplied element-wise with the first tensor to generate a first sub-feature map; The spatial branch is configured to perform average pooling and max pooling on the first tensor respectively, and concatenate the resulting spatial feature maps to generate a spatial attention map; the spatial attention map is then processed by convolution and sigmoid activation to obtain a spatial weight matrix; the spatial weight matrix and the first tensor are multiplied element-wise to generate a second sub-feature map. The multi-scale branch is configured to pass the first tensor through a convolutional layer to obtain the corresponding intermediate result; pass the intermediate result through three grouped convolutional layers with different dilation rates respectively, and fuse the multi-scale feature information to obtain the third sub-feature map; The multidimensional feature refinement module is configured to fuse the first sub-feature map, the second sub-feature map, and the third sub-feature map to obtain the second tensor.

[0066] Figure 7 This is an architecture diagram of the multi-dimensional feature refinement module in one embodiment of this application, which optimizes feature data through parallel branching. For example... Figure 7 As shown, the multidimensional feature refinement module contains three parallel optimization branches: channel branch, spatial branch, and multi-scale branch. After inputting the first tensor, the multidimensional feature refinement module first performs weighted fusion of shallow and deep features to obtain the input features. Then, it is passed to three parallel branches to enhance the representation capabilities of channels, multi-scale and spatial dimensions, respectively.

[0067] The channel branch performs global average pooling on the input features to generate a channel attention map. Subsequently, the channel attention map is sequentially passed through a first 1×1 convolutional layer for dimensionality reduction, an RPReLU activation layer, and then through a second 1×1 convolutional layer to restore the original number of channels. Next, a sigmoid activation is used to generate a channel weight matrix, which is then multiplied element-wise with the input feature F to obtain the first sub-feature map with channel attention.

[0068] In the spatial branch, max pooling and average pooling are applied to the input feature F respectively to obtain two sets of spatial feature maps, which are then concatenated along the channel dimension to obtain the spatial attention map. Then, a spatial weight matrix is ​​obtained through 5×5 convolution and sigmoid activation. The spatial weight matrix is ​​then multiplied element-wise with the input feature F to generate a second sub-feature map that includes spatial attention.

[0069] In the multi-scale branch, 1×1 convolutions are first used for channel dimensionality reduction. Then, three groups of grouped convolutions with different dilation rates (1, 3, and 5) are used to achieve multi-scale feature fusion to reduce the number of parameters and computational overhead. The fused features are summed after passing through a ReLU activation layer, and then restored to the original channel dimension through 1×1 convolutions. They are then multiplied element-wise with the input feature F to obtain the third sub-feature map.

[0070] Finally, the outputs of the three parallel branches are fused by summing element-wise, and after passing through a 1×1 convolution, they are residually connected with the input features to obtain a better multi-scale feature representation, thus obtaining the final second tensor.

[0071] The following example illustrates the training and application process of this system. For the semantic segmentation task of remote sensing images of indoor scenes, this embodiment uses the ISPRS Potsdam dataset for training. The dataset covers various urban elements, divided into six categories: impermeable surfaces, buildings, low vegetation, trees, cars, and miscellaneous items / background. To enhance data diversity and improve the model's generalization ability, the training data was pre-enhanced before training the model through random cropping, horizontal flipping, contrast adjustment, and tone adjustment.

[0072] In the model building phase, this embodiment uses PyTorch 2.0.0 to implement the network structure, and uses both cross-entropy loss (CE) and Dice loss to train the model. The total loss L is calculated as follows: ; in, Cross-entropy loss; For Dice's loss; ; ; in, This represents the true label of the i-th pixel belonging to the c-th type of land cover; The representative model predicts the probability that the i-th pixel belongs to the c-th type of land cover. The smoothing term is a very small constant, mainly used to prevent the denominator from being zero when calculating the Dice loss, and to avoid gradient calculation errors when both the prediction set and the true set are empty.

[0073] Using a single-cycle learning rate strategy with an Adam optimizer, the learning rate is set at a rate of [missing value] in the first half. Growth to Then in the second half from Descending to During training, after each round of training, the model's performance on the test set is evaluated, and the corresponding weights are saved.

[0074] After training, the system receives user-input images (or images captured by a spaceborne camera) and loads them into memory. The input images are preprocessed, segmented into equal-sized pieces for model input, ensuring all image scales are multiples of 64. Edge regions smaller than 64 pixels are padded with pixels. The trained model is then loaded and used to infer the input images, producing predicted semantic segmentation results, which are then visualized on the user interface. Table 1 below shows the comparison results of this system with other systems in quantitative tests on the ISPRS Potsdam dataset.

[0075] Table 1: Quantitative comparison results on the ISPRS Potsdam dataset

[0076] As shown in Table 1, this scheme has a more accurate depth prediction effect compared with other methods. Compared with other image semantic segmentation methods, this scheme has achieved leading results in terms of precision and accuracy. For example, the average intersection-union ratio (mIoU) is 87.61%, the average F1 score (mF1) is 93.51%, and the overall accuracy (OA) is 92.08%, all of which are the best among all tested schemes.

[0077] Based on the same inventive concept, one embodiment of this application provides a method for semantic segmentation of satellite remote sensing images. Figure 8 This is a flowchart of a satellite remote sensing image semantic segmentation method proposed in an embodiment of this application. Figure 8 As shown, the method includes: S1: Extract multi-scale features from the target image to obtain multiple original feature maps; S2: The first fused feature is obtained by calculating and fusing the original feature maps at various scales through linear attention; S3: The first fused feature is split to generate multiple mixed feature maps with the same scale as each original feature map; S4: Extract spatial domain features and frequency domain features from each original feature map and the mixed feature map of the same scale; the spatial domain features include global features and local features; the frequency domain features include high-frequency features and low-frequency features; S5: Perform cross-attention calculation on spatial domain features and frequency domain features at various scales to iteratively fuse spatial domain information and frequency domain information to generate the first tensor; S6: Process the first tensor to obtain the segmentation result of the target image.

[0078] As one embodiment of this application, a first fused feature is obtained by fusing original feature maps at various scales through linear attention calculation, including: performing max pooling on each original feature map; recombining the pooling results of each original feature map to obtain a spliced ​​feature; and performing linear attention calculation on the spliced ​​feature to obtain the first fused feature. The first fusion feature is split to generate multiple mixed feature maps with the same scale as each original feature map, including: splitting the first fusion feature based on the scale corresponding to each original feature map to obtain fusion sub-feature maps with the same scale as each original feature map; and weighting and fusing the original feature maps and fusion sub-feature maps at each scale to generate mixed feature maps at the corresponding scale.

[0079] As one embodiment of this application, linear attention calculation is performed on the splicing features to obtain a first fusion feature, including: The concatenated features are mapped to Q vector, K vector, and V vector; where Q is the query, K is the key, and V is the value. The K vector and V vector are scaled down to shorten their length and increase the number of channels, resulting in compressed K vector and compressed V vector. Based on the Q vector, the compressed K vector, and the compressed V vector, according to The linear attention is calculated using a specific computational method to obtain intermediate results; The intermediate results are processed by a linear projection layer to recover the number of channels, thereby obtaining the first intermediate feature; Repeat the following steps to iteratively optimize the first intermediate feature: map the first intermediate feature to Q vector, K vector, and V vector, according to... The linear attention is calculated using a computational method to obtain optimized intermediate features; After multiple iterations and optimizations, the optimized intermediate feature obtained is determined as the first fusion feature.

[0080] As one embodiment of this application, spatial domain features and frequency domain features are extracted from various original feature maps and mixed feature maps of the same scale, including: separating the feature information of the mixed feature map in different frequency domains to obtain high-frequency features and low-frequency features; processing the mixed feature map through a sliding window to extract feature information in spatial dimension and channel dimension, and fusing them to obtain global features in the spatial domain; and iteratively extracting local features in the spatial domain from the mixed feature map using pooling windows of various sizes. Cross-attention calculation is performed on spatial domain features and frequency domain features at various scales, including: fusing the high-frequency features and local features of the spatial domain through cross-attention calculation to obtain a first sub-feature; and fusing the low-frequency features and global features of the spatial domain through cross-attention calculation to obtain a second sub-feature; and concatenating the first sub-feature and the second sub-feature along the channel dimension to obtain an optimized hybrid feature map.

[0081] As one embodiment of this application, the feature information of the hybrid feature map in different frequency domains is separated to obtain high-frequency features and low-frequency features, including: The hybrid feature map is separated in multiple frequency domains to obtain multiple high-frequency sub-features and one low-frequency sub-feature. All high-frequency sub-features are spliced ​​together to obtain the high-frequency signal; The high-frequency signal is subjected to nonlinear mapping processing to obtain the corresponding high-frequency features; and the low-frequency sub-features are subjected to nonlinear mapping processing to obtain the corresponding low-frequency features.

[0082] As one embodiment of this application, the hybrid feature map is processed through a sliding window to extract feature information in both spatial and channel dimensions, and then fused to obtain global features in the spatial domain, including: Based on a sliding window, the feature data in the hybrid feature map are linearly projected to obtain the corresponding Q matrix, K matrix, and V matrix; where Q is the query, K is the key, and V is the value. Based on the Q matrix, K matrix, and V matrix, according to The calculation method calculates the corresponding intermediate results; Calculate the product of the rows of the Q matrix and the columns of the K matrix generated by the linear projection; The product is subjected to max pooling and average pooling respectively, and the pooling results are fused by element-wise multiplication to obtain the channel weight map; The intermediate results are processed based on the channel weight map, and then processed through strip pooling and large kernel convolutional layers to perform long-distance information fusion between windows in the horizontal and vertical directions, respectively, to obtain global features in the spatial domain.

[0083] As one embodiment of this application, pooling windows of various sizes are used to iteratively extract local features of the spatial domain from the hybrid feature map, including: Basic features are extracted from the hybrid feature map through convolutional layers and normalization processing; The basic features are subjected to multi-scale max pooling operations through a pyramid pooling layer, and feature data from each scale are fused to obtain fused multi-scale local features. By optimizing the fused multi-scale local features through a bottleneck layer, the local features of the spatial domain corresponding to the hybrid feature map are obtained.

[0084] As one embodiment of this application, the method further includes: Based on multiple dimensions, the shallow and deep features in the first tensor are fused to obtain the second tensor; the multiple dimensions include at least two of the following dimensions: channel dimension, spatial dimension, and multi-scale dimension; The second tensor is processed to obtain the segmentation result of the target image.

[0085] As one implementation of this application, a second tensor is obtained by fusing shallow and deep features in the first tensor based on multiple dimensions, including: The first tensor is average pooled to generate a channel attention map; the channel attention map is then processed through multiple convolutions and activations, and a channel weight matrix is ​​generated by sigmoid activation; the channel weight matrix is ​​then multiplied element-wise with the first tensor to generate a first sub-feature map. The first tensor is subjected to average pooling and max pooling respectively, and the resulting spatial feature maps are concatenated to generate a spatial attention map; the spatial attention map is then processed by convolution and sigmoid activation to obtain a spatial weight matrix; the spatial weight matrix and the first tensor are multiplied element by element to generate a second sub-feature map. The first tensor is passed through a convolutional layer to obtain the corresponding intermediate result; the intermediate result is then passed through three grouped convolutional layers with different dilation rates, and multi-scale feature information is fused to obtain the third sub-feature map. The second tensor is obtained by fusing the first sub-feature map, the second sub-feature map, and the third sub-feature map.

[0086] The specific manner in which each step is performed in the methods described in the above embodiments has been described in detail in the relevant conventional embodiments, and will not be elaborated upon here. The above descriptions are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0087] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and components involved are not necessarily essential to this application.

[0088] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, this application is to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.

[0093] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0094] The above provides a detailed description of the satellite remote sensing image semantic segmentation system and method provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A semantic segmentation system for satellite remote sensing images, characterized in that, include: The encoding module is configured to extract multi-scale features from the target image to obtain multiple raw feature maps; The feature fusion module is configured to compute and fuse the original feature maps at various scales through linear attention to obtain the first fused feature. The first fused feature is split to generate multiple mixed feature maps with the same scale as the original feature maps; The collaborative decoding module is configured to extract spatial domain features and frequency domain features from each original feature map and a mixed feature map of the same scale; the spatial domain features include global features and local features; the frequency domain features include high-frequency features and low-frequency features; cross-attention calculation is performed on the spatial domain features and frequency domain features at each scale to iteratively fuse spatial domain information and frequency domain information to generate the first tensor; The output module is configured to process the first tensor to obtain the segmentation result of the target image.

2. The satellite remote sensing image semantic segmentation system according to claim 1, characterized in that, The feature fusion module includes: The pooling layer is configured to perform max pooling on each of the original feature maps; The connection layer is configured to recombine the pooling results of the individual original feature maps to obtain concatenated features; An attention fusion module is configured to perform linear attention calculation on the spliced ​​features to obtain a first fused feature; The splitting layer is configured to split the first fused feature based on the scale corresponding to each original feature map, so as to obtain a fused sub-feature map with the same scale as each original feature map. The gating layer is configured to perform weighted fusion of the original feature maps and fused sub-feature maps at various scales to generate a hybrid feature map at the corresponding scale.

3. The satellite remote sensing image semantic segmentation system according to claim 2, characterized in that, The attention fusion module is configured to perform linear attention calculation on the spliced ​​features to obtain a first fused feature, including: The concatenated features are mapped to Q vector, K vector, and V vector; where Q is the query, K is the key, and V is the value. The K vector and V vector are scaled down to shorten their length and increase the number of channels, resulting in compressed K vector and compressed V vector. Based on the Q vector, the compressed K vector, and the compressed V vector, according to The linear attention is calculated using a specific computational method to obtain intermediate results; The intermediate results are processed by a linear projection layer to recover the number of channels, thereby obtaining the first intermediate feature; Repeat the following steps to iteratively optimize the first intermediate feature: map the first intermediate feature to Q vector, K vector, and V vector, according to... The linear attention is calculated using a computational method to obtain optimized intermediate features; After multiple iterations and optimizations, the optimized intermediate feature obtained is determined as the first fusion feature.

4. The satellite remote sensing image semantic segmentation system according to claim 1, characterized in that, The collaborative decoding module includes: The frequency domain branch is configured to separate the feature information of the hybrid feature map in different frequency domains to obtain high-frequency features and low-frequency features; The global branch is configured to process the hybrid feature map through a sliding window, extract feature information in the spatial and channel dimensions, and fuse them to obtain global features in the spatial domain. Local branches are configured to iteratively extract local features of the spatial domain from the hybrid feature map using pooling windows of various sizes; The collaborative decoding module is configured to fuse the high-frequency features and the local features of the spatial domain through cross-attention calculation to obtain a first sub-feature; and to fuse the low-frequency features and the global features of the spatial domain through cross-attention calculation to obtain a second sub-feature; and to concatenate the first sub-feature and the second sub-feature in the channel dimension to obtain an optimized hybrid feature map.

5. The satellite remote sensing image semantic segmentation system according to claim 4, characterized in that, The frequency domain branch is configured to separate the feature information of the hybrid feature map in different frequency domains to obtain high-frequency features and low-frequency features, including: The hybrid feature map is separated in multiple frequency domains to obtain multiple high-frequency sub-features and one low-frequency sub-feature. All high-frequency sub-features are spliced ​​together to obtain the high-frequency signal; The high-frequency signal is subjected to nonlinear mapping processing to obtain the corresponding high-frequency features; and the low-frequency sub-features are subjected to nonlinear mapping processing to obtain the corresponding low-frequency features.

6. The satellite remote sensing image semantic segmentation system according to claim 4, characterized in that, The global branch is configured to process the hybrid feature map through a sliding window, extract feature information in both spatial and channel dimensions, and fuse them to obtain global features in the spatial domain, including: Based on a sliding window, the feature data in the hybrid feature map are linearly projected to obtain the corresponding Q matrix, K matrix, and V matrix; where Q is the query, K is the key, and V is the value. Based on the Q matrix, K matrix, and V matrix, according to The calculation method calculates the corresponding intermediate results; Calculate the product of the rows of the Q matrix and the columns of the K matrix generated by the linear projection; The product is subjected to max pooling and average pooling respectively, and the pooling results are fused by element-wise multiplication to obtain the channel weight map; The intermediate results are processed based on the channel weight map, and then processed through strip pooling and large kernel convolutional layers to perform long-distance information fusion between windows in the horizontal and vertical directions, respectively, to obtain global features in the spatial domain.

7. The satellite remote sensing image semantic segmentation system according to claim 4, characterized in that, The local branch is configured to use pooling windows of various sizes to iteratively extract local features in the spatial domain from the mixed feature map, including: Basic features are extracted from the hybrid feature map through convolutional layers and normalization processing; The basic features are subjected to multi-scale max pooling operations through a pyramid pooling layer, and feature data from each scale are fused to obtain fused multi-scale local features. By optimizing the fused multi-scale local features through a bottleneck layer, the local features of the spatial domain corresponding to the hybrid feature map are obtained.

8. The satellite remote sensing image semantic segmentation system according to claim 1, characterized in that, The system also includes: The multidimensional feature refinement module is configured to fuse shallow and deep features in the first tensor based on multiple dimensions to obtain a second tensor; the multiple dimensions include at least two of the following dimensions: channel dimension, spatial dimension, and multi-scale dimension; The output module is further configured to process the second tensor to obtain the segmentation result of the target image.

9. The satellite remote sensing image semantic segmentation system according to claim 8, characterized in that, The multidimensional feature refinement module includes: The channel branch is configured to perform average pooling on the first tensor to generate a channel attention map; the channel attention map is then processed through multiple layers of convolution and activation, and a channel weight matrix is ​​generated by sigmoid activation; the channel weight matrix is ​​then multiplied element-wise with the first tensor to generate a first sub-feature map; The spatial branch is configured to perform average pooling and max pooling on the first tensor respectively, and concatenate the resulting spatial feature maps to generate a spatial attention map; the spatial attention map is then processed by convolution and sigmoid activation to obtain a spatial weight matrix; the spatial weight matrix and the first tensor are multiplied element-wise to generate a second sub-feature map. The multi-scale branch is configured to pass the first tensor through a convolutional layer to obtain the corresponding intermediate result; pass the intermediate result through three grouped convolutional layers with different dilation rates respectively, and fuse the multi-scale feature information to obtain the third sub-feature map; The multidimensional feature refinement module is configured to fuse the first sub-feature map, the second sub-feature map, and the third sub-feature map to obtain the second tensor.

10. A semantic segmentation method for satellite remote sensing images, characterized in that, Applied to the system as described in any one of claims 1-9, comprising: Extract multi-scale features from the target image to obtain multiple original feature maps; The first fused feature is obtained by calculating and fusing the original feature maps at various scales using linear attention; The first fused feature is split to generate multiple mixed feature maps with the same scale as the original feature maps; Spatial domain features and frequency domain features are extracted from each original feature map and a mixed feature map of the same scale; the spatial domain features include global features and local features; the frequency domain features include high-frequency features and low-frequency features. Cross-attention calculation is performed on spatial domain features and frequency domain features at various scales to iteratively fuse spatial domain information and frequency domain information to generate the first tensor; The first tensor is processed to obtain the segmentation result of the target image.

Images