A remote sensing image ground object segmentation method based on Fourier space-channel interaction

Abstract

The application discloses a kind of based on Fourier space-channel interaction remote sensing image ground feature segmentation method, belong to remote sensing image processing field, including: obtaining target area contains multiple ground feature types of to-be-segmented remote sensing image and extracting initial feature are preprocessed;Initial feature is handled, and the processing result is input into the four Fourier space-channel interaction Mamba module connected in turn pre-constructed;In four Mamba module, by sequentially carrying out frequency domain transformation and channel interaction processing to input feature, output deep feature representation;Deep feature representation is fused with initial feature by residual connection layer, and global feature compression is carried out to fused feature by pooling layer;Compressed feature is output ground feature segmentation result of remote sensing image by segmentation head.The method obtains the feature information of remote sensing image in frequency domain, spatial domain and channel domain by Fourier space-channel interaction Mamba module, improves the expression ability of model, realizes the ground feature segmentation of remote sensing image.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and more specifically to a method for land feature segmentation in remote sensing images based on Fourier space-channel interaction. Background Technology

[0002] Remote sensing image segmentation is one of the key tasks in remote sensing image analysis. Its main purpose is to perform pixel-level classification of remote sensing data in order to extract information on land cover categories.

[0003] Because remote sensing images typically have characteristics such as hyperspectral, multi-scale, and multi-sensor, and are affected by factors such as noise, illumination changes, and the diversity of land cover types, traditional segmentation methods have certain limitations in terms of accuracy, robustness, and generalization ability.

[0004] In recent years, deep learning technologies, especially convolutional neural networks (CNNs) and self-attention mechanisms (Transformers), have made significant progress in the field of remote sensing image processing. However, traditional CNNs have bottlenecks in handling long-range dependencies and global feature extraction, while Transformers, due to their high computational complexity, still face challenges in their application to high-resolution remote sensing images.

[0005] Mamba is a novel neural network architecture based on the Selective State Space Model. While maintaining approximately linear scalability to sequence length, it provides modeling capabilities comparable to Transformers; in image processing, it selectively focuses on what must be focused on and filters out what can be ignored; and Mamba provides a generalization of all preceding content at each step, combining efficiency in both training and inference.

[0006] Therefore, constructing an efficient, robust, and applicable method for land cover segmentation based on Mamba is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a remote sensing image land cover segmentation method based on Fourier space-channel interaction, which combines frequency domain transformation, channel interaction and hierarchical feature extraction mechanism to improve the feature expression capability of remote sensing images and improve the land cover segmentation accuracy.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A remote sensing image feature segmentation method based on Fourier space-channel interaction includes the following steps:

[0010] S1. Obtain remote sensing images of the target area containing multiple land cover types, perform preprocessing, and extract initial features;

[0011] S2. Normalize and perform nonlinear activation processing on the initial features, and input the processing results into four pre-constructed, sequentially connected Fourier space-channel interactive Mamba modules;

[0012] S3. In the four Fourier space-channel interaction Mamba modules, the input features are sequentially transformed by frequency domain and processed by channel interaction. The preceding output features are then fused by adding them point by point between modules to gradually enhance and deepen the features and output a deep feature representation.

[0013] S4. The deep feature representation is fused with the initial feature through a residual connection layer, and the fused feature is globally compressed through a pooling layer; the compressed feature is then processed by a segmentation head to output the land feature segmentation result of the remote sensing image; the land feature segmentation result includes buildings, roads, water bodies, vegetation and bare land.

[0014] Furthermore, it also includes:

[0015] S5. Use a composite loss function for model training, which includes the weighted cross-entropy loss function, the Dice loss function, and the boundary-aware loss function; optimize the ground feature segmentation performance.

[0016] Furthermore, step S1 includes:

[0017] Acquire remote sensing images of the target area containing multiple land cover types, and perform radiometric correction, geometric correction, and denoising processing.

[0018] Preliminary features are extracted from the denoised image by employing block embedding and a series of convolutional processes; the preliminary features include texture features and structural features related to land cover types.

[0019] Furthermore, the Fourier space-channel interaction Mamba module includes: a normalization layer, a frequency domain module, a spatial module, a fusion and splicing module, a channel module, and a fusion output module;

[0020] The input features are normalized through the normalization layer to obtain a normalized feature representation;

[0021] The frequency domain module performs a Fourier transform on the normalized feature representation, converts it to the frequency domain for frequency scanning and feature interaction, and obtains an enhanced frequency domain feature representation through an inverse Fourier transform.

[0022] The spatial module performs convolutional dimensionality reduction, spatial scanning, and convolution operations on the normalized feature representation to obtain an enhanced spatial feature representation.

[0023] The fusion and splicing module fuses the enhanced spatial feature representation and input features, and fuses the enhanced frequency domain feature representation and normalized feature representation; the two fusion results are spliced ​​together, and after convolution operation, the enhanced spliced ​​feature representation is obtained;

[0024] The channel module performs global feature extraction on the enhanced splicing feature representation to obtain enhanced global features; then, through channel frequency domain transformation, it obtains the spatial features of the channel dimension.

[0025] The fusion output module activates the enhanced global features, multiplies the activation result by the spatial features of the channel dimension, and multiplies it element-wise by the enhanced concatenation feature representation to obtain the output features.

[0026] Furthermore, the frequency domain module performs a Fourier transform on the normalized feature representation, converting it to the frequency domain for frequency scanning and feature interaction, and obtains an enhanced frequency domain feature representation through an inverse Fourier transform; specifically including:

[0027] The frequency domain module performs a Fourier transform on the normalized feature representation to convert it to the frequency domain, thereby obtaining spatial amplitude spectrum features and spatial phase spectrum features.

[0028] A selective scanning mechanism is used to perform frequency scanning on the spatial amplitude spectrum features and spatial phase spectrum features respectively; the scanning results are used for feature interaction through depthwise separable convolution and SiLU activation function;

[0029] The feature interaction results are transformed back to the spatial domain through inverse Fourier transform to obtain an enhanced frequency domain feature representation.

[0030] Furthermore, the spatial module performs convolutional dimensionality reduction, spatial scanning, and convolution operations on the normalized feature representation to obtain an enhanced spatial feature representation; specifically including:

[0031] The spatial module performs dimensionality reduction mapping on the normalized feature representation through convolutional layers;

[0032] A selective scanning mechanism is used to perform spatial scanning on the dimensionality reduction mapping results to generate enhanced features;

[0033] The normalized feature representation is transformed by the SiLU activation function and then multiplied element-wise with the enhanced feature to obtain the spatial feature interaction result.

[0034] The spatial feature interaction results are processed by depthwise separable convolution and standard convolution to obtain an enhanced spatial feature representation.

[0035] Furthermore, the channel module performs global feature extraction on the enhanced splicing feature representation to obtain enhanced global features; then, through channel frequency domain transformation, it obtains the spatial features of the channel dimension; specifically including:

[0036] The channel module performs global feature extraction on the enhanced splicing feature representation to obtain enhanced global features;

[0037] Then, through channel Fourier transform, the channel amplitude spectrum characteristics and channel phase spectrum characteristics are obtained;

[0038] A selective scanning mechanism is used to perform frequency scanning on the channel amplitude spectrum features and channel phase spectrum features respectively; the scanning results are used for feature interaction through depthwise separable convolution and SiLU activation function;

[0039] After feature interaction is completed, the channel dimension is recovered by inverse Fourier transform to obtain the spatial features of the channel dimension.

[0040] Furthermore, step S3 includes:

[0041] In the four Fourier space-channel interaction Mamba modules, the processing result is input into the first Fourier space-channel interaction Mamba module, and then processed through frequency domain transformation and channel interaction.

[0042] The first output feature representation is used as the input feature of the second Fourier space-channel interactive Mamba module; after frequency domain transformation and channel interaction processing, the second output feature representation is obtained.

[0043] The first output feature representation and the second output feature representation are added and fused point by point to obtain an enhanced feature representation, which is used as the input feature of the third Fourier space-channel interactive Mamba module; after frequency domain transformation and channel interaction processing, the third output feature representation is obtained.

[0044] The second and third output feature representations are added and fused point by point to obtain an enhanced feature representation, which is used as the input feature of the fourth Fourier space-channel interactive Mamba module. After frequency domain transformation and channel interaction processing, the fourth output feature representation is obtained as the deep feature representation.

[0045] The deep feature representation includes semantic boundary information of ground features, cross-scale structural features of ground features, and composite features after fusion of multi-source information.

[0046] Furthermore, in step S4, the segmentation head includes an upsampling layer, a convolutional layer, a normalization layer, a ReLU activation function, a convolutional layer, and a Softmax activation function connected in sequence.

[0047] Furthermore, in step S5:

[0048] The weighted cross-entropy loss function is as follows:

[0049]

[0050] Where C is the number of categories; Y c and P c w represents the true label and the model predicted probability of category c, respectively; c f represents the weight of category c; c This indicates the frequency of category c in the dataset;

[0051] The Dice loss function is as follows:

[0052]

[0053] Where ∈ is a smoothing term to prevent the denominator from being zero;

[0054] The boundary-aware loss function is as follows:

[0055] L Boundary =-∑B Y log B P +(1-B Y log(1-B) P )

[0056]

[0057] Among them, B Y B represents the boundary map after calculating the gradient of the true label Y; P This represents the boundary map after calculating the gradient of the segmentation result P.

[0058] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a remote sensing image feature segmentation method based on Fourier space-channel interaction, which has the following beneficial effects:

[0059] This invention employs the Fourier spatial-channel interactive Mamba module, combined with frequency domain transformation, spatial scanning, and channel enhancement mechanisms, enabling the model to make fuller use of global structural information and local details, thereby improving the ability to distinguish complex land cover categories and enhancing the segmentation effect of remote sensing images.

[0060] This invention combines a multi-scale feature interaction mechanism and a hierarchical fusion strategy, enabling the network to adapt to remote sensing images acquired from different resolutions and sensors. Simultaneously, the optimized loss function, incorporating cross-entropy, Dice loss, and boundary-aware loss, improves the model's ability to capture boundary details while maintaining the integrity of the target region, thereby enhancing overall segmentation performance. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0062] Figure 1 A flowchart of a remote sensing image feature segmentation method based on Fourier space-channel interaction is provided for an embodiment of the present invention.

[0063] Figure 2 The diagram shows the structure of the Fourier space-channel interaction Mamba module provided in this embodiment of the invention.

[0064] Figure 3 This is a structural diagram of a remote sensing image ground feature segmentation model based on Fourier space-channel interaction, provided for an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] This invention discloses a method for land cover segmentation in remote sensing images based on Fourier space-channel interaction, referring to... Figure 1 As shown, it includes the following steps:

[0067] S1. Obtain remote sensing images of the target area containing multiple land cover types, perform preprocessing, and extract initial features;

[0068] S2. Normalize and perform nonlinear activation processing on the initial features, and input the processing results into four pre-constructed, sequentially connected Fourier space-channel interactive Mamba modules;

[0069] S3. In the four Fourier space-channel interaction Mamba modules, the input features are sequentially transformed by frequency domain and processed by channel interaction. The preceding output features are then fused by adding them point by point between modules to gradually enhance and deepen the features and output a deep feature representation.

[0070] S4. The deep feature representation is fused with the initial feature through a residual connection layer, and the fused feature is globally compressed through a pooling layer; the compressed feature is then processed by a segmentation head to output the land feature segmentation result of the remote sensing image; the land feature segmentation result includes buildings, roads, water bodies, vegetation and bare land.

[0071] S5. Use a composite loss function for model training, which includes the weighted cross-entropy loss function, the Dice loss function, and the boundary-aware loss function; optimize the ground feature segmentation performance.

[0072] In this embodiment of the invention, according to the above steps, a remote sensing image of the target city is first obtained, such as the image including clouds, buildings, roads, water bodies, vegetation, vehicles and pedestrians, and then image processing and initial feature extraction are performed.

[0073] Secondly, a Fourier spatial-channel interactive Mamba module is constructed to fully integrate the feature information of remote sensing images in the frequency, spatial, and channel domains. Four Fourier spatial-channel interactive Mamba modules are sequentially connected to perform hierarchical information exchange. This ensures that deep features not only retain the key information of the initial features but also enhance the ability of remote sensing images to distinguish land cover categories through multi-layered interaction. Furthermore, a residual connection mechanism is employed to maintain information flow between features at different levels, thereby improving the stability and generalization ability of the segmentation network. Finally, a composite loss function is used for model training and optimization.

[0074] After land cover segmentation using the method of this invention, each pixel can be assigned to its corresponding land cover category. Land cover segmentation yields an image labeled with each land cover category, where each pixel is given a category label. This fine-grained classification provides foundational data for subsequent quantitative analysis, such as calculating the proportion of various land uses, estimating building area, and assessing urban expansion trends. Furthermore, high-quality land cover segmentation results are a key support for intelligent decision-making in multiple fields, including precision agriculture, environmental protection, and disaster response.

[0075] The steps described above are explained in detail below:

[0076] Step S1 specifically includes:

[0077] High-resolution remote sensing images of the target city were acquired using the GF-2 satellite. The input images had a spatial resolution of 0.8 meters, a size of 512×512 pixels, and included R, G, and B bands. The scenes covered typically included buildings, roads, water bodies, vegetation, and bare land. Radiometric correction, geometric correction, and denoising were performed on the remote sensing images to improve data quality and reduce noise interference with the segmentation task.

[0078] Radiometric correction converts the raw digital values ​​recorded by remote sensing sensors into physically meaningful radiance or reflectance to eliminate errors caused by inconsistent sensor responses, atmospheric effects, and other factors. This embodiment performs sensor correction, atmospheric correction, and solar angle correction; it eliminates internal sensor noise and nonlinear response; and it corrects for variations caused by solar angle, atmospheric absorption, and scattering. This improves the comparability of data from different times and different sensors.

[0079] Geometric correction is the process of transforming remote sensing images from an image coordinate system to a geographic coordinate system, so that each pixel in the image corresponds to a real geographical location. This embodiment eliminates geometric distortions caused by factors such as sensor attitude changes, terrain undulations, Earth curvature, and atmospheric refraction through internal geometric correction, external geometric correction, and resampling operations, ensuring accurate spatial positioning of the image and facilitating overlay analysis with other geographic information data.

[0080] Denoising processing aims to reduce random noise in an image, improve the signal-to-noise ratio, and thus enhance image quality and the accuracy of subsequent feature extraction. This embodiment employs Wiener filtering or Lee filtering to reduce noise interference and improve recognition performance.

[0081] The preprocessed remote sensing image is used as input to the neural network. It first undergoes patch embedding and a series of convolutional processes to extract the initial feature representation X. o .

[0082] The preprocessed remote sensing image is divided into several non-overlapping or partially overlapping patches. Then, each image patch is mapped to a fixed-dimensional vector (embedding vector) using linear projection or positional encoding.

[0083] After completing the block embedding, this embodiment captures low-level features such as edges and textures in the image through standard convolutional layers to obtain the initial feature representation X. o .

[0084] The initial features in this embodiment not only include low-level visual features of the image but also various feature representations highly correlated with land cover types. For example, for hyperspectral remote sensing images, the initial features include spectral features extracted by principal component analysis (PCA) or convolution operations, reflecting the differences in reflectance of land cover in different bands; for SAR images, they include texture features extracted by gray-level co-occurrence matrix (GLCM) to distinguish land cover with different roughness and scattering characteristics. Furthermore, structural features such as edge gradients, shape contours, and spatial relationships can be combined to enhance the model's ability to perceive typical land cover such as buildings, roads, water bodies, and vegetation. These features collectively form the basis for the neural network to perform deep semantic understanding in subsequent stages.

[0085] Step S2 specifically includes:

[0086] Reference Figure 2 As shown, a Fourier spatial-channel interactive Mamba module is constructed, including a normalization layer, a frequency domain module, a spatial module, a fusion and stitching module, a channel module, and a fusion output module. By combining the information interaction mechanism of the frequency domain, spatial domain, and channel domain, the feature representation capability of remote sensing images is enhanced, thereby improving the segmentation accuracy.

[0087] This embodiment normalizes and performs nonlinear activation on the initial features, and then inputs them into the Fourier space-channel interactive Mamba module. Let the input features be... First, the normalized feature representation is obtained by entering the row normalization layer. Normalization helps stabilize feature distribution and improves the training convergence and generalization ability of the model.

[0088] Secondly, a frequency domain module is constructed to extract information from remote sensing images in the frequency domain using Fourier transform, thereby enhancing the ability to analyze features at different scales.

[0089] This embodiment uses Fourier transform to normalize the features. Applying a Fourier transform to the frequency domain yields the amplitude spectrum characteristics. and phase spectrum characteristics in, It describes the energy distribution of a signal in the frequency domain and mainly captures global structural information. It describes the phase information of the signal while preserving local edge and contour features.

[0090] Subsequently, a selective scanning mechanism was used to analyze the amplitude spectrum. and phase spectrum Perform an ordered scan to extract key frequency information, and obtain... and Improve the spatial detail recovery capability of remote sensing images. The scanning results are used for feature interaction through depthwise separable convolution and SiLU activation function; enhance the expressive power of frequency domain features.

[0091] The formula is expressed as follows:

[0092]

[0093] Result of feature interaction processing By transforming back to the spatial domain using inverse Fourier transform, an enhanced frequency domain feature representation is obtained.

[0094] Secondly, spatial modules are constructed; in the spatial domain, the spatial representation capability of remote sensing images is enhanced through spatial feature interaction mechanisms.

[0095] This embodiment uses convolutional layers to normalize features. Dimensionality reduction mapping is performed to obtain intermediate feature representations; and a selective scanning mechanism is used to process spatial features step by step, capturing long-distance dependencies and generating enhanced features.

[0096] Then, the normalized features were activated using the SiLU activation function. Perform transformations and combine them with enhanced features. Element-wise multiplication yields the spatial feature interaction result; further processing of the spatial feature interaction result using depthwise separable convolution and standard convolution results results in the enhanced spatial feature representation F2, as follows:

[0097]

[0098] Here, DWConv represents depthwise separable convolution, which is used to further extract local spatial features, while Conv represents standard convolution, which is used to perform channel blending and improve feature representation capabilities.

[0099] Furthermore, a fusion splicing module is constructed to integrate spatial and frequency domain features.

[0100] This embodiment will first... Compared with the original input features Point-by-point addition is performed to fuse spatial enhancement information. Then, the fused features are combined with the frequency domain processed features. Channel concatenation is performed and the data is fed into a convolutional layer to generate an enhanced concatenated feature representation.

[0101] Furthermore, a channel module is constructed; in the channel dimension, the feature representation is further optimized to better adapt to the multispectral characteristics of remote sensing images.

[0102] The channel module in this embodiment is for special purposes. Apply Global Average Pooling (GAP) to extract enhanced global features. Used for channel attention calculation. Perform a Fourier transform to obtain the amplitude spectrum. and phase spectrum

[0103] Then, a selective frequency domain scanning mechanism is used to perform an ordered scan, utilizing the DWConv and SiLU activation functions, followed by feature interaction for feature enhancement; the formula is:

[0104]

[0105] Afterwards, the feature interaction was completed and obtained Spatial features of the channel dimension are recovered using inverse Fourier transform.

[0106] Finally, the fusion output module is constructed.

[0107] This embodiment first... Apply the SiLU activation function and, with Multiply by this to generate enhanced channel features. Then multiply by... Element-wise multiplication yields the final output features:

[0108]

[0109] The module's final output As a high-level feature representation for subsequent remote sensing image segmentation tasks.

[0110] Step S3, refer to Figure 3 As shown, it specifically includes:

[0111] First, connect four Fourier space-channel interactive Mamba modules in sequence.

[0112] In the first Fourier space-channel interaction Mamba module, the input features are After frequency domain transformation and channel interaction processing, the first output feature is represented as follows: In this embodiment, the global structural features of the image, such as the overall outline of buildings and the connectivity of water bodies, are extracted through the frequency domain amplitude spectrum in the first Fourier space-channel interactive Mamba module.

[0113] Further input to the second Fourier space-channel interactive Mamba module generates a second output feature representation. In this embodiment, the long-distance spatial dependencies of features such as roads and vegetation are captured through a selective scanning mechanism of the spatial domain in the second Fourier space-channel interaction Mamba module.

[0114] Secondly, feature fusion and hierarchical interaction are performed; firstly, and The enhanced feature representation is obtained by point-by-point addition and fusion, and then input into the third Fourier space-channel interactive Mamba module to output the third output feature. In the third Fourier space-channel interaction Mamba module, the output features of the first two Fourier space-channel interaction Mamba modules are combined to further enhance the ability to distinguish complex features (such as dense building clusters).

[0115] Subsequently, and The features are further added and fused, and then input into the fourth Fourier space-channel interaction Mamba module to obtain the fourth output feature representation as the final deep feature representation. In the fourth Fourier space-channel interaction Mamba module, the characteristics of deep channel frequency are combined to enhance the representational differences of different types of land features in the channel dimension.

[0116] The deep feature representation in this embodiment not only includes fused multi-scale information but also reflects the contextual semantic enhancement capabilities brought about by joint modeling of the frequency domain and channel dimensions. Specifically, deep features can include the following important information related to remotely sensed ground features: semantic boundary information of ground features, such as building edges and water body outlines, which can be enhanced by high-frequency components in the frequency domain; cross-scale structural features of ground features, such as the distribution characteristics of large areas of forest and point vegetation; the ability to recognize complex texture patterns, which enhances the distinction between surface types such as farmland, sandy land, and bare land through channel interaction; and high-order feature representations after the fusion of multi-source information, such as composite features constructed by combining spectrum and texture, which helps improve the model's discrimination ability in complex backgrounds. The final output deep features provide high-quality semantic support for subsequent decoding, segmentation, or classification modules.

[0117] Step S4 specifically includes:

[0118] The final features are connected through a residual connection layer. With initial feature X o Residual connections are performed to preserve initial feature information while enhancing deep representation capabilities. Global information is incorporated.

[0119] The fused features are compressed globally using an average pooling layer, and then input into the segmentation head for pixel-level classification. The segmentation head in this embodiment consists of an upsampling layer (transposed convolution), a convolutional layer, a normalization layer, a ReLU activation function, another convolutional layer, and a Softmax activation function, ultimately outputting a segmentation result P.

[0120] In this embodiment, the segmentation result P is a pixel-level classification map of the same size as the input image, where each pixel is labeled with its corresponding land feature category. Typical land feature categories include, but are not limited to, the following: buildings (e.g., residential buildings, industrial buildings, commercial areas, etc.); roads (e.g., urban roads, highways, rural paths, etc.); water bodies (e.g., rivers, lakes, canals, etc.); vegetation (e.g., forests, grasslands, farmland, green belts, etc.); bare land (e.g., sandy land, bare rock, construction areas, etc.); wetlands / swamps; vehicles and man-made structures (e.g., bridges, containers, towers, etc., optional); shaded areas and other special land feature categories.

[0121] Step S5 specifically includes:

[0122] Construct a loss function optimization strategy for remote sensing image segmentation.

[0123] In this embodiment, a multi-scale, multi-class awareness loss function optimization strategy is constructed for remote sensing image segmentation tasks to improve segmentation accuracy. This loss function comprehensively considers class imbalance, edge detail preservation, and target region integrity, combining cross-entropy loss, Dice loss, and boundary-aware loss to enhance the segmentation performance of remote sensing images.

[0124] First, we construct a weighted cross-entropy loss.

[0125] Cross-entropy loss is widely used in classification tasks; it measures the similarity between the predicted probability distribution and the true class. For a segmentation prediction P and the true label Y of an input remote sensing image, cross-entropy loss is defined as follows:

[0126]

[0127] Where C is the number of categories, Y c and P c These represent the true label and the model predicted probability of category c, respectively.

[0128] To further improve the model's robustness in class imbalance situations, this embodiment uses weighted cross-entropy:

[0129]

[0130] Wherein, weight w c The frequency f of category c in the dataset c calculate:

[0131]

[0132] This weighted strategy can reduce the neglect of categories with a smaller proportion and improve the segmentation accuracy of small target regions.

[0133] Secondly, construct the Dice loss.

[0134] Since cross-entropy loss may lead to prediction bias in small target regions under class imbalance, this embodiment introduces Dice loss to improve the integrity of the target region.

[0135] Dice loss is defined as follows:

[0136]

[0137] Here, ∈ is a smoothing term to prevent the denominator from being zero.

[0138] The purpose of Dice loss is to measure the degree of overlap between the predicted result and the true label. It can effectively optimize the integrity and boundary information of the target area, and is particularly suitable for segmentation tasks of complex features in remote sensing images.

[0139] Next, construct a boundary-aware loss function.

[0140] Land features in remote sensing images typically have clear boundaries. Therefore, this embodiment introduces boundary-aware loss to enhance the model's ability to learn edge information.

[0141] In this embodiment, the gradient of the true label Y is first calculated to obtain the boundary map B. Y :

[0142]

[0143] Similarly, the boundary map B is calculated for the segmentation result P. P :

[0144]

[0145] Then, in this embodiment, the boundary-aware loss is calculated using binary cross-entropy:

[0146] L Boundary =-∑B Y log B P +(1-B Y log(1-B) P )

[0147] This loss can prompt the model to pay more attention to the boundary region, thereby improving the detail segmentation of ground feature boundaries in remote sensing images.

[0148] Finally, a composite loss function is constructed.

[0149] This embodiment combines the three loss functions mentioned above with weights to obtain the overall optimization objective:

[0150] L=λ1L WCE +λ2L Dice +λ3L Boundary

[0151] Wherein, λ1, λ2, and λ3 are the weight parameters for the weighted cross-entropy loss, Dice loss, and boundary-aware loss, respectively, controlling the degree of influence of each part of the loss. The specific values ​​of the weights can be adjusted according to experiments to obtain the best segmentation performance.

[0152] During training, this embodiment uses the AdamW optimizer to optimize the composite loss function:

[0153]

[0154] Where η is the learning rate and β is the weight decay coefficient. Gradient descent is used to update the network parameters, allowing the model to gradually learn the optimal feature representation and improve the segmentation accuracy of remote sensing images.

[0155] The loss optimization strategy in this embodiment can effectively optimize three aspects: class balance, boundary detail preservation, and target region integrity, thereby improving the overall performance of remote sensing image segmentation.

[0156] This invention first preprocesses the input remote sensing imagery to improve data quality and reduce noise interference in the segmentation task. Then, through block embedding, the input image is converted into a high-dimensional feature representation and input into the hierarchical feature extraction module. To enhance the global perception and cross-layer information interaction capabilities of the features, this invention introduces the Fourier space-channel interaction Mamba mechanism, transforming and fusing features through a series of Fourier space-channel interaction Mamba modules. In this process, deep features not only retain the key information of the initial features but also enhance the ability to distinguish land cover categories in the remote sensing imagery through multi-layer interaction. Furthermore, this invention employs a residual connection mechanism to maintain information flow between features at different levels, thereby improving the stability and generalization ability of the segmentation network.

[0157] The Fourier spatial-channel interactive Mamba module of this invention fully utilizes the feature information of remote sensing images in the frequency, spatial, and channel domains to improve the model's expressive power. First, this module stabilizes the input feature distribution through a normalization layer to enhance model convergence. Then, in the frequency domain module, Fourier transform is used to extract the amplitude and phase spectrum features of the remote sensing images. Amplitude spectrum information is mainly used to capture the global structure of the image, while phase spectrum information can preserve edge and local features, thereby enhancing the model's ability to resolve land cover categories. To further extract key frequency information, this invention introduces a selective scanning mechanism to improve the recovery ability of land cover at different scales, and combines depthwise separable convolution (DWConv) and SiLU activation functions to enhance the frequency domain feature representation.

[0158] In the spatial module, this invention uses convolutional layers to project input features in a dimensionality reduction manner and employs a spatial scanning mechanism to extract long-range dependencies, thereby enhancing the spatial consistency of land cover categories. Simultaneously, the SiLU activation function is used to transform the features, and depthwise separable convolution is used to further extract local feature information. Furthermore, to optimize the representation of channel information, this invention designs a global feature extraction and channel frequency domain transformation mechanism in the channel module. First, global average pooling (GAP) is used to extract global information of the features, and a Fourier transform is applied to obtain channel frequency features. Subsequently, by combining selective scanning and DWConv operations, the model can extract richer information in the channel dimension, thereby improving the multispectral feature analysis capability of remote sensing images.

[0159] Ultimately, this invention fuses spatial, frequency, and channel information, and generates high-level feature representations through convolutional layers and stitching operations, which serve as input for remote sensing image segmentation tasks. Compared to traditional CNN or Transformer methods, the Fourier spatial-channel interactive Mamba module proposed in this invention can improve segmentation accuracy while reducing computational complexity, especially demonstrating superior performance in complex land cover categories, multi-scale scenes, and high-resolution remote sensing images.

[0160] In summary, by combining Fourier transform, channel interaction, and hierarchical feature extraction mechanisms, this invention improves the accuracy of remote sensing image segmentation while enhancing the model's generalization ability and adaptability, enabling it to be more effectively applied to various remote sensing image processing tasks.

[0161] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0162] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for land cover segmentation in remote sensing images based on Fourier space-channel interaction, characterized in that, Includes the following steps: S1. Obtain remote sensing images of the target area containing multiple land cover types, perform preprocessing, and extract initial features; S2. Normalize and perform nonlinear activation processing on the initial features, and input the processing results into four pre-constructed, sequentially connected Fourier space-channel interactive Mamba modules; S3. In the four Fourier space-channel interaction Mamba modules, the input features are sequentially transformed by frequency domain and processed by channel interaction. The preceding output features are then fused by adding them point by point between modules to gradually enhance and deepen the features and output a deep feature representation. S4. The deep feature representation is fused with the initial feature through a residual connection layer, and the fused feature is globally compressed through a pooling layer; the compressed feature is then processed by a segmentation head to output the land feature segmentation result of the remote sensing image; the land feature segmentation result includes buildings, roads, water bodies, vegetation, and bare land; The Fourier space-channel interaction Mamba module includes: a normalization layer, a frequency domain module, a spatial module, a fusion and splicing module, a channel module, and a fusion output module. The input features are normalized through the normalization layer to obtain a normalized feature representation; The frequency domain module performs a Fourier transform on the normalized feature representation, converts it to the frequency domain for frequency scanning and feature interaction, and obtains an enhanced frequency domain feature representation through an inverse Fourier transform. The spatial module performs convolutional dimensionality reduction, spatial scanning, and convolution operations on the normalized feature representation to obtain an enhanced spatial feature representation. The fusion and splicing module fuses the enhanced spatial feature representation and input features, and fuses the enhanced frequency domain feature representation and normalized feature representation; the two fusion results are spliced ​​together, and after convolution operation, the enhanced spliced ​​feature representation is obtained; The channel module performs global feature extraction on the enhanced splicing feature representation to obtain enhanced global features; then, through channel frequency domain transformation, it obtains the spatial features of the channel dimension. The fusion output module activates the enhanced global features, multiplies the activation result by the spatial features of the channel dimension, and multiplies it element-wise by the enhanced concatenation feature representation to obtain the output features.

2. The remote sensing image feature segmentation method based on Fourier space-channel interaction as described in claim 1, characterized in that, Also includes: S5. Use a composite loss function for model training, which includes the weighted cross-entropy loss function, the Dice loss function, and the boundary-aware loss function; optimize the ground feature segmentation performance.

3. The remote sensing image feature segmentation method based on Fourier space-channel interaction as described in claim 1, characterized in that, Step S1 includes: The remote sensing image of the target area containing multiple land cover types is acquired and subjected to radiometric correction, geometric correction and denoising. Preliminary features are extracted from the denoised image by employing block embedding and a series of convolutional processes; the preliminary features include texture features and structural features related to land cover types.

4. The remote sensing image feature segmentation method based on Fourier space-channel interaction as described in claim 1, characterized in that, The frequency domain module performs a Fourier transform on the normalized feature representation, converting it to the frequency domain for frequency scanning and feature interaction, and obtains an enhanced frequency domain feature representation through an inverse Fourier transform; specifically including: The frequency domain module performs a Fourier transform on the normalized feature representation to convert it to the frequency domain, thereby obtaining spatial amplitude spectrum features and spatial phase spectrum features. A selective scanning mechanism is used to perform frequency scanning on the spatial amplitude spectrum features and spatial phase spectrum features respectively; the scanning results are used for feature interaction through depthwise separable convolution and SiLU activation function; The feature interaction results are transformed back to the spatial domain through inverse Fourier transform to obtain an enhanced frequency domain feature representation.

5. The remote sensing image feature segmentation method based on Fourier space-channel interaction as described in claim 1, characterized in that, The spatial module performs convolutional dimensionality reduction, spatial scanning, and convolution operations on the normalized feature representation to obtain an enhanced spatial feature representation; specifically including: The spatial module performs dimensionality reduction mapping on the normalized feature representation through convolutional layers; A selective scanning mechanism is used to perform spatial scanning on the dimensionality reduction mapping results to generate enhanced features; The normalized feature representation is transformed by the SiLU activation function and then multiplied element-wise with the enhanced feature to obtain the spatial feature interaction result. The spatial feature interaction results are processed by depthwise separable convolution and standard convolution to obtain an enhanced spatial feature representation.

6. The remote sensing image feature segmentation method based on Fourier space-channel interaction as described in claim 1, characterized in that, The channel module performs global feature extraction on the enhanced splicing feature representation to obtain enhanced global features; Then, through channel frequency domain transformation, the spatial characteristics of the channel dimension are obtained; specifically including: The channel module performs global feature extraction on the enhanced splicing feature representation to obtain enhanced global features; Then, through channel Fourier transform, the channel amplitude spectrum characteristics and channel phase spectrum characteristics are obtained; A selective scanning mechanism is used to perform frequency scanning on the channel amplitude spectrum features and channel phase spectrum features respectively; the scanning results are used for feature interaction through depthwise separable convolution and SiLU activation function; After feature interaction is completed, the channel dimension is recovered by inverse Fourier transform to obtain the spatial features of the channel dimension.

7. The remote sensing image feature segmentation method based on Fourier space-channel interaction as described in claim 1, characterized in that, Step S3 includes: In the four Fourier space-channel interaction Mamba modules, the processing result is input into the first Fourier space-channel interaction Mamba module, and then processed through frequency domain transformation and channel interaction. The first output feature representation is used as the input feature of the second Fourier space-channel interactive Mamba module; after frequency domain transformation and channel interaction processing, the second output feature representation is obtained. The first output feature representation and the second output feature representation are added and fused point by point to obtain an enhanced feature representation, which is used as the input feature of the third Fourier space-channel interactive Mamba module; after frequency domain transformation and channel interaction processing, the third output feature representation is obtained. The second and third output feature representations are added and fused point by point to obtain an enhanced feature representation, which is used as the input feature of the fourth Fourier space-channel interactive Mamba module. After frequency domain transformation and channel interaction processing, the fourth output feature representation is obtained as the deep feature representation. The deep feature representation includes semantic boundary information of ground features, cross-scale structural features of ground features, and composite features after the fusion of multi-source information.

8. The remote sensing image feature segmentation method based on Fourier space-channel interaction as described in claim 7, characterized in that, In step S4, the segmentation head includes an upsampling layer, a convolutional layer, a normalization layer, a ReLU activation function, a convolutional layer, and a Softmax activation function connected in sequence.

9. The remote sensing image feature segmentation method based on Fourier space-channel interaction as described in claim 2, characterized in that, In step S5: The weighted cross-entropy loss function is as follows: Where C is the number of categories; Y c and P c w represents the true label and the model predicted probability of category c, respectively; c f represents the weight of category c; c This indicates the frequency of category c in the dataset; The Dice loss function is as follows: Where ∈ is a smoothing term to prevent the denominator from being zero; The boundary-aware loss function is as follows: L Boundary =-ΣB Y log B P +(1-B Y )log(1-B P ) Among them, B Y B represents the boundary map after calculating the gradient of the true label Y; P This represents the boundary map after calculating the gradient of the segmentation result P.

Images