A mamba-based space-frequency domain neural network remote sensing image semantic segmentation method

Abstract

The application belongs to the technical field of remote sensing image processing and deep learning, and relates to a space-frequency domain neural network remote sensing image semantic segmentation method based on Mamba. The method introduces a space domain and frequency domain dual domain attention module, respectively uses a linear attention mechanism for space domain feature representation and an attention mechanism for frequency domain representation; a boundary perception head is designed to enhance the boundary representation with graphic information features in the feature extraction stage, and the perception ability of the model to the boundary is improved; and a supervised contrast learning strategy is combined to enhance the aggregation ability of semantic information in the feature representation in the decoder stage. Through the synergistic effect of the above modules, the application effectively overcomes the problems of single optimization method and fuzzy boundary segmentation of the existing Mamba model without modifying the hardware optimization code and increasing the model parameter quantity, and significantly improves the segmentation accuracy and running efficiency of the remote sensing image semantic segmentation model.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing and deep learning technology, and relates to a semantic segmentation method for remote sensing images based on Mamba spatial frequency domain neural networks. Background Technology

[0002] Semantic segmentation of remote sensing images is a core technology for geospatial applications, providing crucial support for land cover classification, ecological environment monitoring, urban planning, and dynamic change detection. With the rapid development of remote sensing platforms (such as UAVs and high-resolution satellites), the spatial resolution of the acquired image data has significantly improved, and the texture and structural details of ground features are richer. However, this also places more stringent technical requirements on semantic segmentation models in terms of refined feature extraction and efficient computation.

[0003] Currently, mainstream remote sensing image semantic segmentation methods are mainly based on convolutional neural networks (CNN) and Transformer architectures. Among them, CNN can effectively capture local spatial features due to its local connectivity and weight sharing characteristics, but its receptive field is limited, resulting in insufficient modeling ability of global context information and difficulty in handling long-distance pixel dependencies. In contrast, Transformer can establish global feature associations through self-attention mechanism, but it has quadratic computational complexity, making it difficult to adapt to large-scale remote sensing image processing. It can be seen that existing traditional models generally have the following technical bottlenecks: (1) It is difficult to balance low computational complexity and long-distance information learning; (2) For the characteristics of large intra-class differences, high inter-class similarity and blurred boundaries that are common in remote sensing images, traditional models are prone to problems such as class confusion and inaccurate segmentation boundary localization.

[0004] In recent years, the state-space model Mamba has demonstrated its application potential in computer vision due to its linear complexity sequence modeling capabilities. Its linear computational complexity and powerful long sequence modeling capabilities provide a new technical approach for processing large-format remote sensing images. However, existing solutions for applying Mamba to remote sensing segmentation still have several key shortcomings: 1. Mamba efficiency optimization mainly focuses on optimizing the GPU computation process, and this optimization is based on the coordination of hardware and software algorithms. Subsequent innovations in algorithm theory are limited by hardware compatibility; 2. Existing models are mostly limited to feature learning in the spatial domain, failing to effectively mine and utilize frequency domain information. This results in insufficient extraction capabilities for frequency domain features such as periodic textures and edges of ground objects in remote sensing images, leading to a single feature representation dimension; 3. There is a lack of targeted enhancement mechanisms for ground object boundary features, making it difficult to meet the urgent need for refined segmentation boundaries in high-resolution remote sensing images.

[0005] In view of this, the present invention is hereby proposed. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a spatial frequency domain neural network-based semantic segmentation method for remote sensing images based on Mamba. This method aims to break the traditional trade-off between model size, computational efficiency and segmentation accuracy, and achieve high-precision pixel-level semantic segmentation while maintaining a lightweight model structure.

[0007] To achieve the above objectives, the present invention provides the following technical solution: On the one hand, this invention provides a spatial-frequency domain neural network-based semantic segmentation method for remote sensing images based on Mamba. It is based on an image segmentation model of dual-domain encoder and similarity fusion decoder, extracts multi-scale features through dual-domain attention module, and improves segmentation accuracy by combining boundary awareness and similarity comparison learning, and finally outputs the semantic segmentation prediction results of remote sensing images.

[0008] Specifically, the method includes the following steps: Step 1: Acquire remote sensing image data; Step 2: Perform preliminary feature transformation on the image data; Step 3: Extract spatial frequency domain features in parallel based on multiple dual-domain attention modules. The output of each dual-domain attention module is connected to the boundary sensing head. The boundary sensing head is used to learn the edge information of objects in remote sensing image data, and its output is used to calculate the contrastive learning loss. Step 4, Multi-scale Feature Interaction and Boundary Constraints: The similarity fusion decoder uses hierarchical dilated convolution and a multi-scale similarity calculation module to constrain the features of different levels of the dual-domain encoder with the features of the decoder, thereby achieving efficient fusion of cross-scale features; and combined with the multi-scale similarity calculation module, it optimizes the edge segmentation results to improve the integrity and accuracy of the segmented contours; Step 5: Achieve high-precision image semantic segmentation by jointly optimizing the contrastive learning loss and the boundary constraint loss.

[0009] Furthermore, in step 3, the dual-domain attention module employs a dual-domain attention mechanism, which includes a frequency domain attention mechanism and a spatial domain Mamba attention mechanism; wherein, The frequency domain attention mechanism transforms the feature representation into the frequency domain, then performs spatial attention processing on the frequency domain signal, and performs an inverse Fourier transform on the resulting signal to obtain the feature representation after frequency domain attention. The spatial domain Mamba attention mechanism, based on a simplified Mamba-inspired linear attention mechanism, reduces the computational complexity of self-attention from quadratic to linear, thereby achieving efficient capture of global features in the spatial domain.

[0010] Specifically, the parallel extraction of spatial-frequency domain features based on multiple dual-domain attention modules includes: First, the input data X after feature transformation in After normalization, a Fast Fourier Transform is performed to obtain the frequency domain feature representation X. FFT ; X FFT =FFT(norm(X in )) Secondly, the separation amplitude spectrum X Amplitude and phase spectrum X Phase , where Im() and Re() represent the real and imaginary parts of the Fourier transform, respectively; Then, frequency domain attention mechanisms are applied, frequency domain features are reconstructed by inverse Fourier transform, and regularized scaling activation functions are introduced to alleviate the long-tail problem of feature distribution. Finally, based on a simplified Mamba-inspired linear self-attention mechanism, the computational complexity of self-attention is reduced from quadratic to linear, enabling efficient capture of global features in the spatial domain.

[0011] It should be noted that in the normal Mamba linear attention mechanism, the positional information of word vectors is composed of conditional positional encoding and rotational positional encoding. In this invention, considering that the backbone network has already added frequency domain information learning, in order to reduce the complexity of positional encoding, Mamba is simplified to only use rotational positional encoding.

[0012] Furthermore, in step 3, the calculation process of the contrastive learning loss specifically includes: Step 3.1: Select the feature representation output by the similarity fusion decoder as the input to the contrastive learning module. Let the feature of the i-th layer decoder be A. i The category label map of the original remote sensing image is scaled to match A. i The same size ensures a one-to-one correspondence between the category labels and the spatial locations of the feature maps; Step 3.2: Based on the scaled label image, filter features by pixel location; for the image of the j-th land cover (there are N land cover types in total), extract A. i The corresponding label is the pixel feature of the j-th type of land cover, which constitutes the feature vector set v of this type. i j ; Step 3.3: From the feature vector set v of each category i jRandomly select n vectors as representative anchor points a for the current category. i j The representative anchor point a i j Used to reflect the distribution of the core features of this category; Step 3.4, in the feature vector set v i j In, select the one with a i j Other feature vectors (non-anchor points) belonging to the j-th class are used as positive samples V + The remaining feature vectors that do not belong to the j-th class are negative samples V. - ; Step 3.5: Use the contrastive loss function (InfoNCE, Info Noise Contrastive Estimation) to measure the anchor point a. i j With positive sample V + negative sample V - Feature similarity; Step 3.6: Introduce a coefficient γ that dynamically increases with the training process to avoid interference from incorrect anchor points in the early stages of training. Among them, the contrastive learning module (CLM) is a key module in Mamba-SFNet used to enhance the feature aggregation capability of land cover categories and reduce the misclassification rate of similar categories, and optimizes the feature representation of the decoder through a supervised contrastive learning mechanism.

[0013] Furthermore, in step 3.5, the process of obtaining the feature similarity is as follows: Where τ is a temperature coefficient used to control the smoothness of the similarity distribution. , These are the feature dot products of the anchor point and the positive and negative samples, respectively, used to measure feature similarity.

[0014] Specifically, in step 3.6, the formula for calculating the coefficient γ is as follows: in, For the current training round, This represents the total number of training rounds.

[0015] Specifically, in step 4, the boundary constraint loss output by the multi-scale similarity calculation module includes: Step 4.1: Process the label image using a Laplacian filter to extract the boundary information of ground features and generate the corresponding boundary label map; Step 4.2: Input the shallow features of the encoder (which have richer details and are more suitable for boundary learning) into the boundary-aware head, and project the output features into a binary boundary mask map with the same size as the input features through a convolutional layer; Step 4.3: In view of the extremely low proportion of boundary pixels (sparse distribution) in remote sensing images, the original boundary labels extracted in step 4.1 are dilated using a dilation operator to expand the pixel range of the boundary region, thereby increasing the proportion of boundary labels in the overall image. Step 4.4: Use the binary cross-entropy loss function to calculate the difference between the boundary mask map in Step 4.2 and the optimized boundary label in Step 4.3.

[0016] Furthermore, the formula for calculating the difference in step 4.4 is as follows: Where α is the weight hyperparameter for boundary pixels, and β is the weight hyperparameter for non-boundary pixels. For the category ground truth of a single pixel, This represents the predicted class value for a single pixel. This represents the number of pixels.

[0017] On the other hand, the present invention also provides a spatial-frequency domain neural network remote sensing image semantic segmentation model based on Mamba, the model being used to implement some or all of the semantic segmentation methods described above, and the model comprising: The basic transformation block is used to perform preliminary feature transformation on the input remote sensing image data; Multiple dual-domain attention blocks of different scales are repeatedly stacked. Each dual-domain attention block includes a frequency domain attention block and a spatial domain attention block. Multi-scale features are extracted from the frequency domain and the spatial domain, respectively, and then fused by a feature fusion module. The boundary-aware head module is connected to the output of dual-domain attention blocks of different scales to learn edge information of remote sensing image data, and the output features are used to calculate the contrastive learning loss. The similarity fusion decoder is used to connect to the output of the dual-domain attention block. It adopts hierarchical dilated convolution and multi-scale similarity calculation module to constrain the features of different levels of the dual-domain encoder with the features of the decoder, so as to achieve efficient fusion of cross-scale features. The segmentation head module is used to fuse features at all scales and output a semantic segmentation prediction map; The multi-scale similarity calculation module is also used to output boundary representation maps, which, combined with boundary constraint loss, optimize edge segmentation results and improve the integrity and accuracy of segmentation contours.

[0018] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects: 1) By designing a dual-domain attention module (DDA) based on simplified Mamba-inspired linear attention (sMILA), the quadratic complexity (O(N2)) of traditional Transformer self-attention is reduced to linear complexity (O(N)), thus reducing computational burden while avoiding performance loss.

[0019] 2) For the first time, frequency domain attention (FDA) and spatial domain Mamba attention (SDMA) are integrated in a remote sensing segmentation model. The amplitude spectrum (reflecting signal intensity) and phase spectrum (reflecting structural details) of the image are separated by FFT. Combined with spatial domain global feature learning, the collaborative extraction of "frequency domain details + spatial context" is achieved.

[0020] 3) By leveraging the synergistic effect of the Boundary Aware Head (BAH) and the Contrastive Learning Module (CLM), the core pain points of "blurred boundaries and similar categories" in remote sensing images are specifically addressed. A pixel-level (PW) and channel-level (CW) hierarchical feature fusion module is designed to achieve an adaptive balance between spatial frequency domain features and encoder-decoder features. Attached Figure Description

[0021] The accompanying drawings are incorporated in and form part of this specification, and together with the description serve to explain the principles of the invention.

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the implementation framework of a spatial frequency domain neural network-based semantic segmentation method for remote sensing images according to the present invention. Figure 2 A flowchart of a spatial frequency domain neural network-based semantic segmentation method for remote sensing images provided by the present invention; Figure 3 This is a structural diagram of the frequency domain feature extraction module in this invention; Figure 4 This is a structural diagram of the spatial domain feature extraction module in this invention; Figure 5 This is the segmentation result of an embodiment of the present invention on a standard dataset. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples consistent with some aspects of the invention as detailed in the appended claims.

[0025] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0026] Example 1 This embodiment provides a spatial frequency domain neural network-based semantic segmentation method for remote sensing images based on Mamba. It is an image segmentation model based on a dual-domain encoder and a similarity fusion decoder. Multi-scale features are extracted through a dual-domain attention module, and boundary awareness and similarity comparison learning are combined to improve segmentation accuracy. Finally, the semantic segmentation prediction results of the remote sensing images are output.

[0027] Specifically, the semantic segmentation method includes the following steps: Step 1: Acquire remote sensing image data; Step 2: Perform preliminary feature transformation on the image data; Step 3: Extract spatial frequency domain features in parallel based on multiple dual-domain attention modules. The output of each dual-domain attention module is connected to the boundary sensing head. The boundary sensing head is used to learn the edge information of objects in remote sensing image data, and its output is used to calculate the contrastive learning loss. Step 4, Multi-scale Feature Interaction and Boundary Constraints: The similarity fusion decoder uses hierarchical dilated convolution and a multi-scale similarity calculation module to constrain the features of different levels of the dual-domain encoder with the features of the decoder, thereby achieving efficient fusion of cross-scale features; and combined with the multi-scale similarity calculation module, it optimizes the edge segmentation results to improve the integrity and accuracy of the segmented contours; Step 5: Achieve high-precision image semantic segmentation by jointly optimizing the contrastive learning loss and the boundary constraint loss.

[0028] Furthermore, in step 3, the dual-domain attention module employs a dual-domain attention mechanism, which includes a frequency domain attention mechanism and a spatial domain Mamba attention mechanism; wherein, The frequency domain attention mechanism transforms the feature representation into the frequency domain, then performs spatial attention processing on the frequency domain signal, and performs an inverse Fourier transform on the resulting signal to obtain the feature representation after frequency domain attention. The spatial domain Mamba attention mechanism, based on a simplified Mamba-inspired linear attention mechanism, reduces the computational complexity of self-attention from quadratic to linear, thereby achieving efficient capture of global features in the spatial domain.

[0029] Specifically, the parallel extraction of spatial-frequency domain features based on multiple dual-domain attention modules includes: First, the input data X after feature transformation in After normalization, a Fast Fourier Transform is performed to obtain the frequency domain feature representation X. FFT ; X FFT =FFT(norm(X in )) Secondly, the separation amplitude spectrum X Amplitude and phase spectrum X Phase , where Im() and Re() represent the real and imaginary parts of the Fourier transform, respectively; Then, frequency domain attention mechanisms are applied, frequency domain features are reconstructed by inverse Fourier transform, and regularized scaling activation functions are introduced to alleviate the long-tail problem of feature distribution. Finally, based on a simplified Mamba-inspired linear self-attention mechanism, the computational complexity of self-attention is reduced from quadratic to linear, enabling efficient capture of global features in the spatial domain.

[0030] It should be noted that in the normal Mamba linear attention mechanism, the positional information of word vectors is composed of conditional positional encoding and rotational positional encoding. In this invention, considering that the backbone network has already added frequency domain information learning, in order to reduce the complexity of positional encoding, Mamba is simplified to only use rotational positional encoding.

[0031] Furthermore, in step 3, the calculation process of the contrastive learning loss specifically includes: Step 3.1: Select the feature representation output by the similarity fusion decoder as the input to the contrastive learning module. Let the feature of the i-th layer decoder be A. i The category label map of the original remote sensing image is scaled to match A. i The same size ensures a one-to-one correspondence between the category labels and the spatial locations of the feature maps; Step 3.2: Based on the scaled label map, filter features by pixel location; for the j-th type of land cover image, extract A. iThe corresponding label is the pixel feature of the j-th type of land cover, which constitutes the feature vector set v of this type. i j ; Step 3.3: From the feature vector set v of each category i j Randomly select n vectors as representative anchor points a for the current category. i j The representative anchor point a i j Used to reflect the distribution of the core features of this category; Step 3.4, in the feature vector set v i j In, select the one with a i j Feature vectors belonging to the j-th class are used as positive samples V + The remaining feature vectors that do not belong to the j-th class are negative samples V. - ; Step 3.5: Use the contrastive loss function to measure anchor point a i j With positive sample V + negative sample V - Feature similarity; Step 3.6: Introduce a coefficient γ that dynamically increases with the training process to avoid interference from incorrect anchor points in the early stages of training. Furthermore, in step 3.5, the process of obtaining the feature similarity is as follows: Where τ is a temperature coefficient used to control the smoothness of the similarity distribution. , These are the feature dot products of the anchor point and the positive and negative samples, respectively, used to measure feature similarity.

[0032] Specifically, in step 3.6, the formula for calculating the coefficient γ is as follows: in, For the current training round, This represents the total number of training rounds.

[0033] Specifically, in step 4, the boundary constraint loss output by the multi-scale similarity calculation module includes: Step 4.1: Process the label image using a Laplacian filter to extract the boundary information of ground features and generate the corresponding boundary label map; Step 4.2: Input the shallow features of the encoder into the boundary sensing head, and project the output features into a binary boundary mask map with the same size as the input features through a convolutional layer; Step 4.3: Use the dilation operator to dilate the original boundary labels extracted in Step 4.1 to expand the pixel range of the boundary region and increase the proportion of the boundary labels in the overall image. Step 4.4: Use the binary cross-entropy loss function to calculate the difference between the boundary mask map in Step 4.2 and the optimized boundary label in Step 4.3.

[0034] Furthermore, the formula for calculating the difference in step 4.4 is as follows: Where α is the weight hyperparameter for boundary pixels, and β is the weight hyperparameter for non-boundary pixels. For the category ground truth of a single pixel, This represents the predicted class value for a single pixel. This represents the number of pixels.

[0035] Example 2 This embodiment provides a spatial frequency domain neural network-based semantic segmentation model for remote sensing images based on Mamba. The model is used to implement some or all of the semantic segmentation methods described above. The specific steps in building this model include: (a) First, prepare remote sensing image data: The present invention was experimentally verified on the Vaihingen high-resolution remote sensing dataset to demonstrate the effectiveness and feasibility of the method: the Vaihingen dataset contains 6 classes: Building, High veg, Imper surf, Low veg, Cars, and Tree; the images were uniformly cropped to a size of 512×512 pixels.

[0036] (II) Model Building (Model Parameter Settings and Reasons) 1) Number of training rounds: 200 rounds; 2) Input remote sensing image size: 512×512 pixels (adapted to the dataset cropping size, balancing resolution and computational cost); 3) Learning rate: The backbone network learning rate is 0.001 (the backbone network parameters need to be fine-tuned to avoid destroying the pre-trained features). 4) Weight decay: 1e-4 (regularization to prevent model overfitting); 5) Optimizer: SGD (suitable for deep learning models, convergent and stable, adapted to semantic segmentation tasks).

[0037] (III) Model Training and Evaluation This method selects Precision, Recall, IoU, and F1 score as evaluation metrics within the task and mIoU and mF1 as accuracy metrics between tasks to evaluate the performance of this invention in incremental learning semantic segmentation tasks. The specific formulas are as follows: Where P is the percentage of correct positive predictions, reflecting the reliability of the detection results; TP is the number of correctly detected real targets; FP is the number of background regions misidentified as targets; recall R represents the proportion of real targets correctly detected by the model, reflecting the model's ability to avoid missed detections; and FN in the formula represents the number of undetected real targets.

[0038] Where N represents the number of categories, and mIoU and mF1 represent the IoU and F1 scores of the i-th category.

[0039] (iv) Experimental Results and Comparison The method of this invention was compared with existing technologies (such as MCSNet, AFENet, A2FPN) on the Vaihingen dataset. The main evaluation metrics include IoU (Intersection over Union), mIoU (mean Intersection over Union), and mF1 (mean F1 score).

[0040] Table 1 Comparison results of the Vaihingen dataset As shown in Table 1, the semantic segmentation method provided by this invention has strong generalization ability: it performs well on datasets of different geographical regions and resolutions, adapts to different land cover types and remote sensing image characteristics, and has a wide range of practical application scenarios.

[0041] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention.

[0042] It should be understood that the present invention is not limited to the content already described above, and various modifications and changes can be made without departing from its scope. The scope of the present invention is limited only by the appended claims.

Claims

1. A semantic segmentation method for remote sensing images based on Mamba spatial-frequency domain neural networks, characterized in that, The image segmentation model based on dual-domain encoder and similarity fusion decoder extracts multi-scale features through dual-domain attention module and combines boundary awareness and similarity contrast learning to improve segmentation accuracy, and finally outputs semantic segmentation prediction results of remote sensing images.

2. The spatial-frequency domain neural network-based remote sensing image semantic segmentation method according to claim 1, characterized in that, Includes the following steps: Step 1: Acquire remote sensing image data; Step 2: Perform preliminary feature transformation on the image data; Step 3: Extract spatial frequency domain features in parallel based on multiple dual-domain attention modules. The output of each dual-domain attention module is connected to the boundary sensing head. The boundary sensing head is used to learn the edge information of objects in remote sensing image data, and its output is used to calculate the contrastive learning loss. Step 4, Multi-scale Feature Interaction and Boundary Constraints: The similarity fusion decoder uses hierarchical dilated convolution and a multi-scale similarity calculation module to constrain the features of different levels of the dual-domain encoder with the features of the decoder, thereby achieving efficient fusion of cross-scale features; and combined with the multi-scale similarity calculation module, it optimizes the edge segmentation results to improve the integrity and accuracy of the segmented contours; Step 5: Achieve high-precision image semantic segmentation by jointly optimizing the contrastive learning loss and the boundary constraint loss.

3. The spatial-frequency domain neural network-based remote sensing image semantic segmentation method according to claim 2, characterized in that, In step 3, the dual-domain attention module adopts a dual-domain attention mechanism, which includes a frequency domain attention mechanism and a spatial domain Mamba attention mechanism. The frequency domain attention mechanism transforms the feature representation into the frequency domain, performs spatial attention processing on the frequency domain signal, and performs an inverse Fourier transform on the resulting signal to obtain the feature representation after frequency domain attention. The spatial domain Mamba attention mechanism, based on a simplified Mamba-inspired linear attention mechanism, reduces the computational complexity of self-attention from quadratic to linear, thereby achieving efficient capture of global features in the spatial domain.

4. The spatial-frequency domain neural network-based remote sensing image semantic segmentation method according to claim 3, characterized in that, The simplified Mamba uses only rotational position encoding.

5. The spatial-frequency domain neural network-based remote sensing image semantic segmentation method according to claim 2, characterized in that, Step 3, the calculation process of the contrastive learning loss specifically includes: Step 3.1: Select the feature representation output by the similarity fusion decoder as the input to the contrastive learning module. Let the feature of the i-th layer decoder be A. i The category label map of the original remote sensing image is scaled to match A. i The same size ensures a one-to-one correspondence between the category labels and the spatial locations of the feature maps; Step 3.2: Based on the scaled label map, filter features by pixel location; for the j-th type of land cover image, extract A. i The corresponding label is the pixel feature of the j-th type of land cover, which constitutes the feature vector set v of this type. i j ; Step 3.3: From the feature vector set v of each category i j Randomly select n vectors as representative anchor points a for the current category. i j The representative anchor point a i j Used to reflect the distribution of the core features of this category; Step 3.4, in the feature vector set v i j In, select the one with a i j Feature vectors belonging to the j-th class are used as positive samples V + The remaining feature vectors that do not belong to the j-th class are negative samples V. - ; Step 3.5: Use the contrastive loss function to measure anchor point a i j With positive sample V + negative sample V - Feature similarity; Step 3.6: Introduce a coefficient γ that dynamically increases with the training process to avoid interference from incorrect anchor points in the early stages of training.

6. The spatial-frequency domain neural network-based remote sensing image semantic segmentation method according to claim 5, characterized in that, In step 3.5, the process of obtaining the feature similarity is as follows: Where τ is a temperature coefficient used to control the smoothness of the similarity distribution. , These are the feature dot products of the anchor point and the positive and negative samples, respectively, used to measure feature similarity.

7. The spatial-frequency domain neural network-based remote sensing image semantic segmentation method according to claim 5, characterized in that, In step 3.6, the formula for calculating the coefficient γ is as follows: in, For the current training round, This represents the total number of training rounds.

8. The spatial-frequency domain neural network-based remote sensing image semantic segmentation method according to claim 2, characterized in that, In step 4, the boundary constraint loss output by the multi-scale similarity calculation module specifically includes: Step 4.1: Process the label image using a Laplacian filter to extract the boundary information of ground features and generate the corresponding boundary label map; Step 4.2: Input the shallow features of the encoder into the boundary sensing head, and project the output features into a binary boundary mask map with the same size as the input features through a convolutional layer; Step 4.3: Use the dilation operator to dilate the original boundary labels extracted in Step 4.1 to expand the pixel range of the boundary region and increase the proportion of the boundary labels in the overall image. Step 4.4: Use the binary cross-entropy loss function to calculate the difference between the boundary mask map in Step 4.2 and the optimized boundary label in Step 4.

3.

9. The spatial-frequency domain neural network-based remote sensing image semantic segmentation method according to claim 8, characterized in that, The formula for calculating the difference in step 4.4 is as follows: Where α is the weight hyperparameter for boundary pixels, and β is the weight hyperparameter for non-boundary pixels. For the category ground truth of a single pixel, This represents the predicted class value for a single pixel. This represents the number of pixels.

10. A spatial-frequency domain neural network-based semantic segmentation model for remote sensing images, characterized in that, The model is used to implement the semantic segmentation method according to any one of claims 1 to 9, and the model includes: The basic transformation block is used to perform preliminary feature transformation on the input remote sensing image data; Multiple dual-domain attention blocks of different scales are repeatedly stacked. Each dual-domain attention block includes a frequency domain attention block and a spatial domain attention block. Multi-scale features are extracted from the frequency domain and the spatial domain, respectively, and then fused by a feature fusion module. The boundary-aware head module is connected to the output of dual-domain attention blocks of different scales to learn edge information of remote sensing image data, and the output features are used to calculate the contrastive learning loss. The similarity fusion decoder is used to connect to the output of the dual-domain attention block. It adopts hierarchical dilated convolution and multi-scale similarity calculation module to constrain the features of different levels of the dual-domain encoder with the features of the decoder, so as to achieve efficient fusion of cross-scale features. The segmentation head module is used to fuse features at all scales and output a semantic segmentation prediction map; The multi-scale similarity calculation module is also used to output boundary representation maps, which, combined with boundary constraint loss, optimize the segmentation results and improve the integrity and accuracy of the segmentation contours.

Images