An image segmentation method and system

Abstract

The application relates to the technical field of image processing, and particularly discloses an image segmentation method and system, which comprises the following steps: acquiring a plurality of encoding feature maps with different spatial resolutions; inputting the deepest encoding feature map into a frequency domain enhancement module; the frequency domain enhancement module performs spatial domain processing and frequency domain processing in parallel, and outputs bottleneck fusion features; inputting the bottleneck fusion features into a decoder; at each hop connection, a current encoding feature map and a current decoding feature map are subjected to channel-space double gate fusion through an adaptive fusion gate module; and inputting the feature map finally output by the decoder into an uncertainty quantization head module, to output the predicted probability of each pixel belonging to each category and the uncertainty value of each pixel. The application introduces a self-defined frequency domain enhancement module to enhance the spectral distinguishing capability by using frequency domain analysis and cross attention mechanism, so that the low-precision problem caused by spectral confusion in traditional methods and convolutional neural network models is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an image segmentation method and system. Background Technology

[0002] In remote sensing image analysis tasks, semantic segmentation is a core technology for achieving pixel-level recognition of ground features. With the development of high-resolution multispectral remote sensing satellites (such as Landsat 8-9), remote sensing images exhibit typical image characteristics of being "multi-channel, large-scale, and containing mixed ground features." Especially in land cover classification tasks, image segmentation in complex surface environments faces the following common challenges at the image processing level:

[0003] Poor class separability due to spectral feature mixing: In image regions where saline-alkali land, cultivated land, grassland, and other land cover types coexist, the responses of different land cover types on multiple spectral bands highly overlap, exhibiting typical spectral confusion. Traditional methods based on spectral indices only utilize single or a few band combinations, lacking the ability to model the complex interactions between multiple channels, making it difficult to construct discriminative feature representations, resulting in numerous misclassifications in image segmentation results.

[0004] Blurred segmentation boundaries and artifact interference: Because the edges of features in remote sensing images often exhibit gradual transitions rather than abrupt changes, especially in areas where saline-alkali land meets cultivated land or bare land, existing image segmentation models are prone to producing "spot-like" missegmentation or boundary drift. This problem essentially stems from the model's insufficient ability to model local contextual information and edge structure information. In particular, during multi-scale feature fusion, simple stitching operations introduce semantically inconsistent features, reducing the accuracy of segmentation boundary localization.

[0005] Insufficient generalization ability in cross-regional image segmentation: Remote sensing images from different geographical regions exhibit significant differences in imaging conditions, surface composition, and noise distribution. Existing segmentation models (especially convolutional neural networks) show a significant decrease in segmentation performance when transferred to new regions after training in one region, exhibiting structural failure due to feature distribution shift. This reflects the model's lack of ability to model frequency-domain invariant features or globally context-stable features.

[0006] Image segmentation results lack uncertainty assessment mechanisms: In high-risk decision-making scenarios such as ecological monitoring, image segmentation results not only require high-precision classification but also need to output the model's confidence level in its own predictions. Most existing image segmentation methods adopt deterministic outputs, which cannot generate pixel-level uncertainty maps. This makes it impossible for users to determine which regions have unreliable segmentation results, limiting the interpretability and robustness of the model in practical deployment.

[0007] To address the aforementioned image processing challenges, researchers have introduced the Transformer architecture in recent years to enhance global modeling capabilities. However, its training cost is high and it lacks targeted optimization for spectral obfuscation tasks. Uncertainty modeling methods attempt to introduce probability outputs, but they are difficult to effectively model the uncertainties under complex spectral distributions in remote sensing images.

[0008] Therefore, there is an urgent need to propose a remote sensing image segmentation algorithm for complex terrain scenes that can perform collaborative optimization in the three key image processing stages of feature extraction, multi-scale fusion, and output modeling, thereby improving segmentation accuracy, boundary clarity, and model interpretability. Summary of the Invention

[0009] The purpose of this invention is to provide an image segmentation method and system to solve the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] An image segmentation method, the method comprising:

[0012] A multispectral remote sensing image to be segmented is acquired, and an encoder is used to extract multi-scale features from the multispectral remote sensing image to obtain multiple encoded feature maps with different spatial resolutions.

[0013] The deepest encoded feature map is input into the frequency domain enhancement module, which performs spatial domain processing and frequency domain processing in parallel, and fuses spatial features and frequency domain enhanced features through a cross-attention mechanism to output bottleneck fusion features.

[0014] The bottleneck fusion features are input into the decoder, which upsamples step by step and performs channel-space dual-gated fusion on the current encoded feature map and the current decoded feature map through an adaptive fusion gating module at each jump connection, outputting multi-level fusion features;

[0015] The feature map output by the decoder is input into the uncertainty quantization head module. The uncertainty quantization head module maps the convolution output to the parameters of the Dirichlet distribution based on evidence deep learning, and outputs the predicted probability of each pixel belonging to each category and the uncertainty value of each pixel, thus obtaining the segmentation prediction map and the uncertainty map.

[0016] As a further improvement of the present invention, the frequency domain enhancement module performs the following operations:

[0017] In the spatial domain path, the input features are convolved with 1×1 to obtain spatial features;

[0018] In the frequency domain path, a two-dimensional fast Fourier transform is performed on the input features. The transformed frequency domain features are multiplied element-wise with the learnable frequency mask, and then restored to the spatial domain through a two-dimensional inverse fast Fourier transform. After passing through a 1×1 convolution, the frequency domain enhanced features are obtained.

[0019] The spatial features and the frequency domain enhancement features are flattened and then input into a multi-head cross-attention layer, where the query vector comes from the spatial features and the key vector and value vector come from the frequency domain enhancement features, to obtain the attention fusion features;

[0020] The attention fusion feature and the spatial feature are added element-wise and then residually connected to output the bottleneck fusion feature.

[0021] As a further improvement of the present invention, the adaptive fusion gating module performs the following operations:

[0022] Receive the encoded feature map from the i-th layer of the encoder and the upsampled decoded feature map from the (i+1)-th layer of the decoder;

[0023] The encoded feature maps are subjected to global average pooling and global variance pooling respectively. The pooling results are concatenated along the channel dimension and input into a multilayer perceptron. The channel weights are generated by mapping through the Sigmoid activation function.

[0024] Perform a 1×1 convolution on the decoded feature map to generate spatial query features. Multiply the spatial query features element-wise with the encoded feature map and then map them using the Sigmoid activation function to generate a spatial weight map.

[0025] The encoded feature map is then multiplied element-wise with the channel weight vector and the spatial weight map to obtain the gated encoded feature map.

[0026] The gated encoded feature map and the decoded feature map are concatenated along the channel dimension to output a multi-level fused feature.

[0027] As a further improvement of the present invention, the uncertainty quantization head module performs the following operations:

[0028] The feature map output by the decoder is compressed to the number of classes K by 1×1 convolution to obtain the original log odds vector.

[0029] The original log-odds vector is input into the Softplus activation function to convert it into non-negative evidence quantities. Each piece of evidence quantity is incremented by 1 to obtain the parameters of the Dirichlet distribution. ;

[0030] Calculate the total evidence strength S, and then calculate the expected probability of each pixel belonging to the k-th class as follows: Output segmentation prediction map;

[0031] The uncertainty value of each pixel is calculated as follows: Output the uncertainty graph.

[0032] As a further improvement to the present invention, a training step is also included:

[0033] Obtain a training dataset, which contains multispectral remote sensing image samples and corresponding pixel-level labels;

[0034] The encoder, frequency domain enhancement module, adaptive fusion gating module, decoder, and uncertainty quantization head module are trained end-to-end using an evidence deep learning loss function, which includes a cross-entropy loss term and a KL divergence regularization term.

[0035] The present invention also provides an image segmentation system, the system comprising:

[0036] The encoder unit is used to acquire the multispectral remote sensing image to be segmented, and to extract multi-scale features from the multispectral remote sensing image using the encoder to obtain multiple coded feature maps with different spatial resolutions.

[0037] The frequency domain enhancement unit is used to input the deepest encoded feature map into the frequency domain enhancement module. The frequency domain enhancement module performs spatial domain processing and frequency domain processing in parallel, and fuses spatial features and frequency domain enhanced features through a cross-attention mechanism to output bottleneck fusion features.

[0038] The decoder unit is used to input the bottleneck fusion feature into the decoder. The decoder performs upsampling step by step, and at each hop connection, the adaptive fusion gating module performs channel-space dual gating fusion on the current encoded feature map and the current decoded feature map to output multi-level fusion features.

[0039] The uncertainty quantization head unit is used to input the feature map finally output by the decoder into the uncertainty quantization head module. The uncertainty quantization head module maps the convolution output to the parameters of the Dirichlet distribution based on evidence deep learning, outputs the predicted probability of each pixel belonging to each category and the uncertainty value of each pixel, and obtains the segmentation prediction map and the uncertainty map.

[0040] As a further improvement of the present invention, the frequency domain enhancement unit includes:

[0041] The spatial domain processing subunit is used to perform 1×1 convolution on the input features to obtain spatial features;

[0042] The frequency domain processing subunit is used to sequentially perform two-dimensional fast Fourier transform, element-wise multiplication with a learnable frequency mask, two-dimensional inverse fast Fourier transform, and 1×1 convolution on the input features to obtain frequency domain enhanced features.

[0043] The cross-attention fusion subunit is used to input the spatial features and the frequency domain enhancement features into the multi-head cross-attention layer, and add the attention output to the spatial feature residual to output the bottleneck fusion feature.

[0044] Compared with existing technologies, the advantages of this invention are as follows: By introducing a custom frequency domain enhancement module, it leverages frequency domain analysis and cross-attention mechanisms to enhance spectral discrimination capabilities, thus solving the low-precision problem caused by spectral obfuscation in traditional methods and convolutional neural network models. Simultaneously, an adaptive fusion gating module is designed to dynamically fuse multi-scale features, optimizing the segmentation of complex texture regions and reducing boundary errors. Furthermore, by designing an uncertainty quantization head module combined with evidence-based deep learning loss, pixel-level uncertainty maps are generated, providing crucial reliability assessments for high-risk ecological decisions. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0046] Figure 1 The diagram shows the structure of the FAU-SwinUNet model provided in this embodiment of the invention.

[0047] Figure 2 This is a structural diagram of the frequency domain enhancement module provided in an embodiment of the present invention.

[0048] Figure 3 This is a structural diagram of the adaptive fusion gating module provided in an embodiment of the present invention.

[0049] Figure 4 This is a structural diagram of the uncertainty quantization head module provided in an embodiment of the present invention.

[0050] Figure 5 This is a visualization of comparative images on a dataset from a plain in Northeast China, provided as an embodiment of the present invention.

[0051] Figure 6 This is a visualization of comparative images on a basin geomorphology dataset provided in an embodiment of the present invention.

[0052] Figure 7 This is a visualization comparison of ablation results for a dataset from a plain in Northeast China, provided as an embodiment of the present invention.

[0053] Figure 8This is a visualization comparison of the ablation results of the basin geomorphology dataset provided in this embodiment of the invention. Detailed Implementation

[0054] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0055] In this embodiment of the invention, an image segmentation method is provided, the method comprising:

[0056] A multispectral remote sensing image to be segmented is acquired, and an encoder is used to extract multi-scale features from the multispectral remote sensing image to obtain multiple encoded feature maps with different spatial resolutions.

[0057] The deepest encoded feature map is input into the frequency domain enhancement module, which performs spatial domain processing and frequency domain processing in parallel, and fuses spatial features and frequency domain enhanced features through a cross-attention mechanism to output bottleneck fusion features.

[0058] The bottleneck fusion features are input into the decoder, which upsamples step by step and performs channel-space dual-gated fusion on the current encoded feature map and the current decoded feature map through an adaptive fusion gating module at each jump connection, outputting multi-level fusion features;

[0059] The feature map output by the decoder is input into the uncertainty quantization head module. The uncertainty quantization head module maps the convolution output to the parameters of the Dirichlet distribution based on evidence deep learning, and outputs the predicted probability of each pixel belonging to each category and the uncertainty value of each pixel, thus obtaining the segmentation prediction map and the uncertainty map.

[0060] In this embodiment, the image segmentation model in this invention is denoted as FAU-SwinUNet, and the image segmentation model is as follows: Figure 1 As shown, it accepts 6-channel Landsat raster images (256*256 pixels, bands 2-7) as input. The model is based on a symmetrical encoder-decoder structure, where the encoder adopts the Swin-Transformer coding style and the decoder adopts the UNet coding style, and integrates three core innovative modules: a frequency domain enhancement module (such as...) Figure 2 As shown), adaptive fusion gating module (such as...) Figure 3 (as shown) and uncertainty quantization head module (such as Figure 4 (As shown).

[0061] Multi-scale coding feature extraction: A six-channel Landsat raster image with a size of 256*256 is fed into a pre-trained encoder. After the encoder divides the image into patches, it sequentially performs feature extraction and downsampling in four stages. Each stage uses sliding window attention and fixed window attention mechanisms to continuously extract and downsample features at multiple scales, ultimately outputting four coded feature maps with different spatial resolutions and channel depths.

[0062] Bottleneck layer feature processing based on the frequency domain enhancement module: The feature map output from the fourth stage of the encoder is input to the frequency domain enhancement module connecting the encoder and decoder. This module processes the input features in parallel spatial and spectral domains. In the frequency domain path, the spatial domain is transformed to the spectral domain through a two-dimensional fast Fourier transform, and frequency components are filtered by multiplying with a learnable mask. Then, the filtered features are restored to the spatial domain through an inverse transform and obtained by 1×1 convolution to obtain the frequency-enhanced feature F. freq The output is then fed into a multi-head cross-attention mechanism for fusion to obtain the fused feature F. fused Output. Finally, by comparing with the original spatial features F spatial The element-wise addition forms the residual connection, and the output bottleneck fusion feature X is output. out .

[0063] Decoding and Multi-Scale Fusion Based on Adaptive Fusion Gating: Output features enter the decoder. The decoder consists of four upsampling stages, gradually restoring the feature map resolution through patch expansion. At each skip connection of upsampling, there is an adaptive fusion gating module. This module receives upsampled high semantic features from the decoder and high-resolution features from the corresponding encoder layer. Specifically, this module receives the high-resolution feature encoder input from the i-th stage of the encoder and the upsampled feature decoder input from the (i+1)-th stage of the decoder, and processes them strictly according to the following parallel path: the encoder input is fed in parallel into global average pooling and variance pooling for processing, the pooling results are concatenated, and the output channel weights W are nonlinearly mapped through a multilayer perceptron. c The decoder input is fed separately into a 1×1 convolutional layer to generate spatial query features Q. Then, the query features Q are multiplied element-wise with the original encoder input, and the result is fed into a sigmoid activation function to generate spatial weights W. s The generated weights are used to perform gating filtering of encoder features, that is, the original encoder input is sequentially compared with the channel weights W. c and spatial weight W s Perform element-wise multiplication to obtain E i ’ Finally, E i ’The input to the decoder is concatenated along the channel dimension to generate a multi-level fused feature S. i This multi-level fusion feature S i After four complete upsampling and gating fusion operations, the feature map is restored to the original input resolution of 256*256.

[0064] The probability and uncertainty output based on the uncertainty quantization head: The feature map output by the decoder is fed into the uncertainty quantization head module for distribution parameter calculation. The specific processing flow is as follows: First, the feature map output by the decoder is input into the convolutional layer, and the number of channels is compressed to be the same as the number of categories K in the classification task (K=7), obtaining the original log odds; then, the result is input into the Softplus activation function, forcibly converting it into non-negative evidence values; subsequently, in order to ensure that each evidence value is not 0, the evidence values ​​are incremented by 1, mapping to the parameters of the Dirichlet distribution. ,make sure Furthermore, for parameters of all categories The summation operation is performed to obtain the total evidence strength S. Finally, based on the parameter α and the total strength S, the model simultaneously outputs two branches: one branch outputs the segmentation prediction map of the seven land cover types, and the other branch outputs the uncertainty map representing the model's confidence level.

[0065] In a preferred embodiment of the present invention, the frequency domain enhancement module performs the following operations:

[0066] In the spatial domain path, the input features are convolved with 1×1 to obtain spatial features;

[0067] In the frequency domain path, a two-dimensional fast Fourier transform is performed on the input features. The transformed frequency domain features are multiplied element-wise with the learnable frequency mask, and then restored to the spatial domain through a two-dimensional inverse fast Fourier transform. After passing through a 1×1 convolution, the frequency domain enhanced features are obtained.

[0068] The spatial features and the frequency domain enhancement features are flattened and then input into a multi-head cross-attention layer, where the query vector comes from the spatial features and the key vector and value vector come from the frequency domain enhancement features, to obtain the attention fusion features;

[0069] The attention fusion feature and the spatial feature are added element-wise and then residually connected to output the bottleneck fusion feature.

[0070] like Figure 2 As shown, in this embodiment, the custom frequency domain enhancement module in the bottleneck layer effectively enhances the spectral discrimination ability between saline-alkali land and other land features by processing spatial and frequency domain information in parallel and fusing them using a cross-attention mechanism. Specifically, in the spatial domain path, the input features are processed through a 1×1 convolution to obtain spatial features F. spatialThis operation not only completes channel alignment, but more importantly, it refines local information from the original features to generate an optimized F. spatial This is used for query projection in the subsequent cross-attention mechanism. In the frequency domain processing path, the input is first subjected to a two-dimensional fast Fourier transform to convert it to the frequency domain. Subsequently, it is multiplied element-wise with a learnable frequency mask M to achieve adaptive enhancement or suppression of specific frequency components.

[0071] Then, the features are restored back to the spatial domain using a two-dimensional inverse fast Fourier transform, and after a 1×1 convolution, the frequency domain enhanced features F are obtained. freq Finally, F spatial and F freq After flattening, the input is deeply fused using a multi-head cross-attention mechanism, and finally, through integration with F... spatial The residual connections yield bottleneck fusion features X out .

[0072] This process can be summarized by the following formula:

[0073] ;

[0074] Among them, X out This represents the bottleneck layer feature tensor, i.e., the bottleneck fusion feature, output by the frequency domain enhancement module after deep fusion with cross-attention and residual addition; Softmax represents the normalized exponential activation function, used to map the similarity score after the dot product of the query vector and the key vector to a probability distribution in the interval (0,1), thereby generating the attention weight matrix; Q(F spatial ) represents the linear projection operation of the query vector, with spatial features F spatial As the basis for the query; K(F) freq ) represents the linear projection operation on the key vector to enhance the frequency domain features F. freq As a key vector; This represents taking the square root with respect to the dimension; : Represents a linear projection operation on the value vector to enhance the frequency domain features F freq As a value vector; This indicates an element-wise addition operation.

[0075] In a preferred embodiment of the present invention, the adaptive fusion gating module performs the following operations:

[0076] Receive the encoded feature map from the i-th layer of the encoder and the upsampled decoded feature map from the (i+1)-th layer of the decoder;

[0077] The encoded feature maps are subjected to global average pooling and global variance pooling respectively. The pooling results are concatenated along the channel dimension and input into a multilayer perceptron. The channel weights are generated by mapping through the Sigmoid activation function.

[0078] Perform a 1×1 convolution on the decoded feature map to generate spatial query features. Multiply the spatial query features element-wise with the encoded feature map and then map them using the Sigmoid activation function to generate a spatial weight map.

[0079] The encoded feature map is then multiplied element-wise with the channel weight vector and the spatial weight map to obtain the gated encoded feature map.

[0080] The gated encoded feature map and the decoded feature map are concatenated along the channel dimension to output a multi-level fused feature.

[0081] like Figure 3 As shown, in this embodiment, the core design purpose of the custom adaptive fusion gating module at the skip connection is to solve the problem of noise or redundant information that may be introduced due to semantic mismatch when directly concatenating the high-resolution features (encoded feature map) of the deep encoder with the high semantic features (upsampled decoded feature map) of the shallow decoder. This module receives the high-resolution feature E from the i-th layer of the encoder. i (Encoded feature map) and upsampled high semantic features D from the (i+1)th layer of the subsequent decoder i+1 The input is the upsampled decoded feature map, and its core operation is a parallel dual-weight mechanism.

[0082] Considering the spectral complexity and uncertainty of features in saline-alkali land tasks, the channel weighting component not only employs traditional global average pooling to obtain channel average information but also introduces global variance pooling to measure the spatial discreteness or heterogeneity of features, thereby enhancing the perception of texture differences. The results from both methods are concatenated and input into a lightweight multilayer perceptron to generate channel weights W. c :

[0083] ;

[0084] The high semantic features D returned by the spatial weighted decoder i+1 The query feature Q is generated through a 1×1 convolution. Then, Q is combined with the encoded feature map E. i Element-wise multiplication followed by Sigmoid activation generates a spatial weight map W. s This operation ensures that only feature points relevant to the current reconstruction area are enhanced.

[0085] ;

[0086] ;

[0087] Ultimately, the channel weight W c With spatial weight W S Together they act on the original encoded feature map E i The gated encoded feature map E is obtained. i ’ Subsequently, E i ’ With upsampled decoding feature map D i+1 The splicing process forms a multi-level fusion feature S i And pass it to the subsequent double convolution module:

[0088] ;

[0089] ;

[0090] Among them, E i This represents the shallow, high-resolution features from the i-th layer of the encoder, i.e., the encoded feature map; D i+1 represents the deep upsampled high semantic features from the (i+1)th layer of the decoder, i.e., the upsampled decoded feature map; MLP stands for Lightweight Multilayer Perceptron; Sigmoid represents the Sigmoid activation function, used to nonlinearly map numerical values ​​to the (0,1) interval, thereby generating continuous gating coefficients; W c Represents the generated channel weight vector; Concat represents the feature concatenation operation along the channel dimension; AvgPool represents the global average pooling operation; VarPool represents the global variance pooling operation; Q represents the query feature generated by the convolution operation on the features input from the decoder; Conv represents the 1×1 convolution operation; W s This represents the generated spatial weight vector; Represents element-wise multiplication of matrices; This represents the gated encoded feature map obtained after double weighting by channel and spatial weights; S i This represents the multi-level fused feature output after skip connection processing.

[0091] In a preferred embodiment of the present invention, the uncertainty quantization head module performs the following operations:

[0092] The feature map output by the decoder is compressed to the number of classes K by 1×1 convolution to obtain the original log odds vector.

[0093] The original log-odds vector is input into the Softplus activation function to convert it into non-negative evidence quantities. Each piece of evidence quantity is incremented by 1 to obtain the parameters of the Dirichlet distribution. ;

[0094] Calculate the total evidence strength S, and then calculate the expected probability of each pixel belonging to the k-th class as follows: Output segmentation prediction map;

[0095] The uncertainty value of each pixel is calculated as follows: Output the uncertainty graph.

[0096] like Figure 4 As shown in this embodiment, a custom uncertainty quantization head module is used in the model output stage. This module receives the fused features F from the decoder end. out As input, the channels are compressed to the number of classes K (K=7) through 1×1 convolution, generating the original network log odds output (logits), which is the original log odds vector, denoted as Z. k The result is then input into the Softplus activation function, forcing it to be converted into a non-negative evidence quantity. Where the evidence of the k-th class is e... k The calculation is as follows:

[0097] ;

[0098] Evidence e k Parameters of the enhanced Dirichlet distribution , representing the confidence strength of category k. Its parameter set To ensure that each piece of evidence is not zero, the total amount of evidence is incremented by 1 to control the probability of the model assigning each category.

[0099] ;

[0100] Then, the total evidence strength S is obtained by summing the parameters of all categories; based on these parameters, the expected prediction probability P of class k is obtained. k The overall model uncertainty U is:

[0101] ;

[0102] ;

[0103] Among them, Z k This represents the original network log-odds (Logits) output for the k-th land cover class after the network reduces the dimensionality of the input features through convolutional layers. This value can be positive or negative and has not yet been normalized; K represents the total number of classes set in the segmentation task (corresponding to 7 land cover classes in this invention), and also represents the "total ignorance" of the system in uncertainty calculation; e k This represents the amount of non-negative evidence extracted based on features regarding that the pixel belongs to the k-th type of land cover. It reflects the strength of the supporting features that the model actually observes from the image. P represents the mapped Dirichlet distribution parameters, indicating the model's confidence or belief strength in the k-th class; k The semantic segmentation prediction map output by the first branch of the model is composed of the expected probabilities of each pixel; U is the uncertainty quantization map output by the second branch of the model, which gathers the uncertainty values ​​of all pixels and is used to intuitively indicate the uncertainty of the model's prediction. That is, the higher the uncertainty, the more hesitant the model tends to be when making predictions, and vice versa.

[0104] As a preferred embodiment of the present invention, a training step is also included:

[0105] Obtain a training dataset, which contains multispectral remote sensing image samples and corresponding pixel-level labels;

[0106] The encoder, frequency domain enhancement module, adaptive fusion gating module, decoder, and uncertainty quantization head module are trained end-to-end using an evidence deep learning loss function, which includes a cross-entropy loss term and a KL divergence regularization term.

[0107] In this embodiment, to support model training and validation, the present invention selects a publicly available 30-meter land use dataset and combines it with Landsat 8-9 images of a core plain area and basin topography in Northeast China. A saline-alkali land segmentation dataset is constructed using the professional remote sensing image processing software QGIS. This dataset covers seven land cover types, including saline-alkali land. The specific data acquisition, processing, and construction process is as follows:

[0108] Remote sensing imagery data was acquired from publicly available satellite data websites. High-quality images with cloud cover less than 15% acquired between June and September 2023 were selected to ensure data integrity and reliability. The ground truth label dataset used was the aforementioned publicly available 30-meter land use dataset. The acquisition and production period of this dataset corresponds precisely to the time period of the remote sensing imagery dataset, and this period conforms to the optimal phenological period principle, corresponding to the peak vegetation growth season. This maximizes the spectral separability among various land cover types, facilitating accurate differentiation of target categories.

[0109] First, Landsat 8 / 9 OLI imagery covering two regions was imported into QGIS. Multi-band composite operations (bands 2 to 7) were performed on each image to construct a six-dimensional spectral feature space. To eliminate data redundancy and ensure spatial continuity, image stitching technology was used to fuse multiple images into a complete regional-scale image. Subsequently, using the precise administrative region vector boundaries of a plain in Northeast China and the northern edge of the Tarim Basin as a mask, a strict cropping operation was performed to limit the data range to the study area, effectively removing irrelevant background information. Simultaneously, ground truth label data was spatially registered and synchronized with the remote sensing imagery. The original complex multi-level classification system in the ground truth label data was converted into a label system suitable for semantic segmentation tasks. Specifically, cultivated land, forest land, grassland, water area, residential area, and other land types were retained as basic categories, while saline-alkali land was extracted separately from "other land types" as a seventh independent target category, thus maximizing the highlighting of the core research object. The final standard label system includes seven categories: cultivated land, forest land, grassland, water area, residential area, saline-alkali land, and other land types. After spatial and spectral normalization, a sliding window cropping technique was used to construct the final dataset. All images and their corresponding labels were cropped into 256×256 pixel sample blocks. To ensure sample quality, a strict quality control mechanism was implemented: first, samples with a valid pixel ratio of less than 90% were removed; then, manual random sampling was conducted to delete low-quality image-label pairs. Afterward, according to a random allocation principle, the qualified samples were divided into training, validation, and test sets to ensure statistical independence.

[0110] Training hardware configuration: Intel i9-14900HX processor, 16GB RAM, and NVIDIA 5060 GPU with 8GB VRAM.

[0111] Training software environment: FAU-SwinUNet model built on the Python-based PyTorch deep learning framework, and other comparative models.

[0112] Comparative Experiments: To demonstrate the performance of our model, we compared it with seven semantic segmentation models, using mean intersection-over-union ratio (mIoU), overall accuracy (OA), Kappa coefficient, and user accuracy (UA) as the main evaluation metrics. Furthermore, we introduced the APU metric in the ablation experiments to specifically evaluate the effectiveness of the uncertainty quantification mechanism. Figure 5 , Figure 6 Tables 1 and 2 show the results of the comparative experiments on the two datasets, along with visual comparison images.

[0113] Table 1. Results of the comparative experiment on a dataset from a plain in Northeast China.

[0114]

[0115] Table 2. Results of the comparative experiment on the basin geomorphology dataset.

[0116]

[0117] Ablation Experiments: To fully verify the independent contributions and synergistic effects of the proposed frequency domain enhancement module and adaptive fusion gating module, and to evaluate the impact of the uncertainty quantization mechanism, we conducted a series of systematic ablation experiments on two datasets. Five different configurations were designed to rigorously separate the performance gains brought by each component module:

[0118] M0: The original Swin-UNet, trained using standard cross-entropy loss. The purpose of introducing M0 is to verify whether the transition from a traditional learning framework to an evidence-based learning framework will affect the basic segmentation capabilities of the backbone network.

[0119] M1: An uncertainty quantization head is introduced on top of Swin-UNet, and it is trained using evidence-based learning loss. This model serves as a baseline for our proposed method. By comparing M1 and M0, the specific impact of the uncertainty estimation mechanism on segmentation accuracy can be quantified.

[0120] M2: Based on M1, a frequency domain enhancement module is introduced to evaluate the effect of frequency domain feature enhancement in suppressing spectral noise.

[0121] M3: Based on M1, an adaptive fusion gating module is introduced to verify the effect of the attention-based feature selection mechanism in boundary refinement.

[0122] FAU-SwinUNet: This paper proposes a complete model that integrates two modules on top of the M1 framework. By comparing it with M2 and M3, the synergistic gains brought about by the combined use of the two modules can be verified.

[0123] As shown in Tables 3 and 4 Figure 7 and Figure 8 As shown, the comparison results and visualizations of ablation experiments on two datasets are presented. APU is an uncertainty value used to evaluate the model; the smaller the value, the more confident the model's predictions are.

[0124] Table 3 Ablation results of a dataset from a plain in Northeast China

[0125]

[0126] Table 4 Ablation results of the basin geomorphology dataset

[0127]

[0128] This invention also provides an image segmentation system, the system comprising:

[0129] The encoder unit is used to acquire the multispectral remote sensing image to be segmented, and to extract multi-scale features from the multispectral remote sensing image using the encoder to obtain multiple coded feature maps with different spatial resolutions.

[0130] The frequency domain enhancement unit is used to input the deepest encoded feature map into the frequency domain enhancement module. The frequency domain enhancement module performs spatial domain processing and frequency domain processing in parallel, and fuses spatial features and frequency domain enhanced features through a cross-attention mechanism to output bottleneck fusion features.

[0131] The decoder unit is used to input the bottleneck fusion feature into the decoder. The decoder performs upsampling step by step, and at each hop connection, the adaptive fusion gating module performs channel-space dual gating fusion on the current encoded feature map and the current decoded feature map to output multi-level fusion features.

[0132] The uncertainty quantization head unit is used to input the feature map finally output by the decoder into the uncertainty quantization head module. The uncertainty quantization head module maps the convolution output to the parameters of the Dirichlet distribution based on evidence deep learning, outputs the predicted probability of each pixel belonging to each category and the uncertainty value of each pixel, and obtains the segmentation prediction map and the uncertainty map.

[0133] In a preferred embodiment of the present invention, the frequency domain enhancement unit includes:

[0134] The spatial domain processing subunit is used to perform 1×1 convolution on the input features to obtain spatial features;

[0135] The frequency domain processing subunit is used to sequentially perform two-dimensional fast Fourier transform, element-wise multiplication with a learnable frequency mask, two-dimensional inverse fast Fourier transform, and 1×1 convolution on the input features to obtain frequency domain enhanced features.

[0136] The cross-attention fusion subunit is used to input the spatial features and the frequency domain enhancement features into the multi-head cross-attention layer, and add the attention output to the spatial feature residual to output the bottleneck fusion feature.

[0137] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image segmentation method, characterized in that, The method includes: A multispectral remote sensing image to be segmented is acquired, and an encoder is used to extract multi-scale features from the multispectral remote sensing image to obtain multiple encoded feature maps with different spatial resolutions. The deepest encoded feature map is input into the frequency domain enhancement module, which performs spatial domain processing and frequency domain processing in parallel, and fuses spatial features and frequency domain enhanced features through a cross-attention mechanism to output bottleneck fusion features. The bottleneck fusion features are input into the decoder, which upsamples step by step and performs channel-space dual-gated fusion on the current encoded feature map and the current decoded feature map through an adaptive fusion gating module at each jump connection, outputting multi-level fusion features; The feature map output by the decoder is input into the uncertainty quantization head module. The uncertainty quantization head module maps the convolution output to the parameters of the Dirichlet distribution based on evidence deep learning, and outputs the predicted probability of each pixel belonging to each category and the uncertainty value of each pixel, thus obtaining the segmentation prediction map and the uncertainty map.

2. The image segmentation method according to claim 1, characterized in that, The frequency domain enhancement module performs the following operations: In the spatial domain path, the input features are convolved with 1×1 to obtain spatial features; In the frequency domain path, a two-dimensional fast Fourier transform is performed on the input features. The transformed frequency domain features are multiplied element-wise with the learnable frequency mask, and then restored to the spatial domain through a two-dimensional inverse fast Fourier transform. After passing through a 1×1 convolution, the frequency domain enhanced features are obtained. The spatial features and the frequency domain enhancement features are flattened and then input into a multi-head cross-attention layer, where the query vector comes from the spatial features and the key vector and value vector come from the frequency domain enhancement features, to obtain the attention fusion features; The attention fusion feature and the spatial feature are added element-wise and then residually connected to output the bottleneck fusion feature.

3. The image segmentation method according to claim 1, characterized in that, The adaptive fusion gating module performs the following operations: Receive the encoded feature map from the i-th layer of the encoder and the upsampled decoded feature map from the (i+1)-th layer of the decoder; The encoded feature maps are subjected to global average pooling and global variance pooling respectively. The pooling results are concatenated along the channel dimension and input into a multilayer perceptron. The channel weights are generated by mapping through the Sigmoid activation function. Perform a 1×1 convolution on the decoded feature map to generate spatial query features. Multiply the spatial query features element-wise with the encoded feature map and then map them using the Sigmoid activation function to generate a spatial weight map. The encoded feature map is then multiplied element-wise with the channel weight vector and the spatial weight map to obtain the gated encoded feature map. The gated encoded feature map and the decoded feature map are concatenated along the channel dimension to output a multi-level fused feature.

4. The image segmentation method according to claim 1, characterized in that, The uncertainty quantization header module performs the following operations: The feature map output by the decoder is compressed to the number of classes K by 1×1 convolution to obtain the original log odds vector. The original log-odds vector is input into the Softplus activation function to convert it into non-negative evidence quantities. Each piece of evidence quantity is incremented by 1 to obtain the parameters of the Dirichlet distribution. ; Calculate the total evidence strength S, and then calculate the expected probability of each pixel belonging to the k-th class as follows: Output segmentation prediction map; The uncertainty value of each pixel is calculated as follows: Output the uncertainty graph.

5. The image segmentation method according to claim 1, characterized in that, It also includes training steps: Obtain a training dataset, which contains multispectral remote sensing image samples and corresponding pixel-level labels; The encoder, frequency domain enhancement module, adaptive fusion gating module, decoder, and uncertainty quantization head module are trained end-to-end using an evidence deep learning loss function, which includes a cross-entropy loss term and a KL divergence regularization term.

6. An image segmentation system for implementing the image segmentation method according to any one of claims 1-5, characterized in that, The system includes: The encoder unit is used to acquire the multispectral remote sensing image to be segmented, and to extract multi-scale features from the multispectral remote sensing image using the encoder to obtain multiple coded feature maps with different spatial resolutions. The frequency domain enhancement unit is used to input the deepest encoded feature map into the frequency domain enhancement module. The frequency domain enhancement module performs spatial domain processing and frequency domain processing in parallel, and fuses spatial features and frequency domain enhanced features through a cross-attention mechanism to output bottleneck fusion features. The decoder unit is used to input the bottleneck fusion feature into the decoder. The decoder performs upsampling step by step, and at each hop connection, the adaptive fusion gating module performs channel-space dual gating fusion on the current encoded feature map and the current decoded feature map to output multi-level fusion features. The uncertainty quantization head unit is used to input the feature map finally output by the decoder into the uncertainty quantization head module. The uncertainty quantization head module maps the convolution output to the parameters of the Dirichlet distribution based on evidence deep learning, outputs the predicted probability of each pixel belonging to each category and the uncertainty value of each pixel, and obtains the segmentation prediction map and the uncertainty map.

7. The image segmentation system according to claim 6, characterized in that, The frequency domain enhancement unit includes: The spatial domain processing subunit is used to perform 1×1 convolution on the input features to obtain spatial features; The frequency domain processing subunit is used to sequentially perform two-dimensional fast Fourier transform, element-wise multiplication with a learnable frequency mask, two-dimensional inverse fast Fourier transform, and 1×1 convolution on the input features to obtain frequency domain enhanced features. The cross-attention fusion subunit is used to input the spatial features and the frequency domain enhancement features into the multi-head cross-attention layer, and add the attention output to the spatial feature residual to output the bottleneck fusion feature.

Images