A remote sensing image segmentation method integrating frequency modulation and spatial awareness

Abstract

This invention discloses a remote sensing image segmentation method that integrates frequency modulation and spatial awareness. The method acquires and preprocesses the original remote sensing image to generate a standardized input image. This image is then input into a multi-scale frequency domain enhancement feature extraction network, which extracts features at multiple feature levels. Each level achieves frequency-adaptive semantic enhancement through frequency domain modulation transformation and spatial feature fusion, followed by modeling and residual connection output via a feedforward neural network. Residual fusion is introduced across levels to enhance deep feature representation. A decoding module decodes the final multi-scale features to restore spatial resolution and generate pixel-level segmentation results. A composite loss function, including classification error, boundary awareness, and frequency consistency terms, is constructed and used to optimize and train the network. This method fully leverages the semantic complementarity of remote sensing images in the frequency and spatial domains, thereby improving the segmentation accuracy and robustness of ground targets in complex scenes, and possesses good generalization ability and engineering practicality.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and more specifically to a remote sensing image segmentation method that integrates frequency modulation and spatial perception. Background Technology

[0002] Remote sensing image segmentation is a key technology in fields such as geographic information analysis, land use planning, urban monitoring, and agricultural management. It aims to perform pixel-level semantic classification of ground features (such as buildings, roads, water bodies, and farmland) in remote sensing imagery. With the continuous improvement of remote sensing imaging resolution, images exhibit more complex ground feature structures, small-scale targets, and fine-grained texture features. This places higher demands on the feature representation capabilities, boundary recognition capabilities, and multi-scale modeling capabilities of segmentation models.

[0003] Traditional remote sensing image segmentation methods largely rely on convolutional neural networks (CNNs) for spatial domain feature extraction. These methods can capture local structural information in images, such as edges and textures, to a certain extent. However, due to the inherent receptive field limitations of CNNs, they perform poorly when handling large-scale contexts, fine-grained structures, or long-range dependencies, especially in complex scenes such as densely built-up areas, target occlusion, or blurred boundaries. This often results in problems such as missed detection of small targets, unclear boundaries, or category confusion. Furthermore, existing methods generally suffer from insufficient multi-scale information fusion and loss of local details when dealing with terrain features that exhibit drastic scale variations, severely restricting the accuracy and generalization ability of remote sensing image segmentation models.

[0004] In recent years, the Transformer architecture has been increasingly applied to remote sensing image analysis due to its excellent global modeling capabilities. However, pure Transformer models suffer from high computational overhead and high training data requirements, making them unsuitable for direct deployment in remote sensing tasks with limited data. Other works have attempted to incorporate frequency domain information into the model structure, utilizing Fourier transforms to capture the global semantic structure of images. However, these methods mostly remain at the shallow operation stage, lacking frequency modulation, spatial interaction, and semantic enhancement mechanisms, making it difficult to fully unleash the potential of frequency domain information in segmentation tasks.

[0005] Therefore, there is an urgent need for a remote sensing image segmentation method that can fully exploit the semantic complementarity of remote sensing images in the frequency and spatial domains. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a remote sensing image segmentation method that integrates frequency modulation and spatial perception. Based on fully absorbing the advantages of existing spatial modeling methods, it innovatively integrates frequency domain modeling and spatial domain modulation mechanisms, fully explores the semantic complementarity of remote sensing images in the frequency and spatial domains, provides a new technical path for remote sensing image segmentation tasks, and has good segmentation accuracy, generalization ability and engineering practicality. It is especially suitable for the need for fine analysis of ground objects in complex scenarios.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] This invention first provides a remote sensing image segmentation method that integrates frequency modulation and spatial awareness, comprising the following steps:

[0009] S1: Acquire the raw remote sensing image and preprocess it to generate a standardized input image;

[0010] S2: The standardized input image is input into a multi-scale frequency domain enhancement feature extraction network, and multi-stage feature extraction is performed sequentially according to multiple feature levels. In each feature level, the input features of the current feature level are subjected to frequency domain modulation transformation, and the modulated frequency domain features are fused with the input features to obtain frequency-adaptive semantic enhancement features. The frequency-adaptive semantic enhancement features are then subjected to nonlinear modeling and semantic transformation of the channel dimension through a feedforward neural network and residual connection is performed with the frequency-adaptive semantic enhancement features to form the output features of the current feature level, which serve as the input features of the next feature level. A cross-layer residual fusion mechanism is introduced between at least two feature levels to fuse output features of different depths, and the final multi-scale frequency domain enhancement features are output at the end feature level.

[0011] S3: The segmentation decoding module decodes the final multi-scale frequency domain enhancement features obtained through multi-stage progressive modeling to restore spatial resolution and generate pixel-level segmentation results.

[0012] S4: Construct a composite supervised loss function and jointly optimize and train the multi-scale frequency domain enhanced feature extraction network and the segmentation decoding module to perform segmentation operations on the remote sensing image to be processed according to S1 to S3.

[0013] Preferably, before inputting the standardized input image into the multi-scale frequency domain enhancement feature extraction network in step S2, the method further includes: first passing the standardized input image through a convolutional layer to extract initial spatial texture features, and then using the initial spatial texture features as input to the multi-scale frequency domain enhancement feature extraction network.

[0014] Preferably, in step S2, performing frequency domain transformation modulation on the input features of the current feature level and fusing the modulated frequency domain features with the input features to obtain frequency-adaptive semantic enhancement features includes the following steps:

[0015] S21: Input features for the current feature level After performing a Fast Fourier Transform (FFT), a subsequent FFT shift is performed to move the zero-frequency components from the four corners to the center of the spectrum. The resulting features are the same as the input features. Features are obtained by performing residual connections

[0016] S22: Features After normalization, the data is sequentially input into the first convolutional layer and the depthwise separable convolutional layer to extract local and spread frequency information. Based on a lightweight gating mechanism, the gated fusion features corresponding to the local and spread frequency information are output.

[0017] S23: Gated fusion features based on lightweight channel attention mechanism Perform channel feature enhancement to obtain channel features

[0018] S24: Channel Features The system is divided into multiple spatial sub-regions. Based on the positional relationship of each spatial sub-region relative to the spectral center, a position code characterizing its frequency spatial distribution is generated. A kernel-bias generator is used to dynamically generate parameters for feature modulation using these position codes, and the frequency domain features of each spatial sub-region are weighted and biased to obtain the frequency position modulated output features.

[0019] S25: Determine the channel features and the output characteristics after frequency position modulation The modulation is achieved by performing addition and fusion, followed by sequential passing through a third convolutional layer, inverse fast Fourier transform shift, and inverse fast Fourier transform to obtain the modulated frequency domain features.

[0020] Preferably, the lightweight gating mechanism in S22 includes:

[0021] S221: The output features of the depth-separable convolutional layer are divided into two sub-feature maps by channel segmentation;

[0022] S222: The two sub-feature maps are respectively input to two convolutional layer branches with different kernel sizes, and the outputs of the two convolutional layer branches are multiplied element-wise to obtain the gated fusion features.

[0023] Preferably, the lightweight channel attention mechanism in S23 includes:

[0024] S231: The gated fusion feature The data is fed into a global average pooling layer for spatial information aggregation to obtain global statistics for each channel.

[0025] S232: A second convolutional layer processes global statistics to generate weights for each channel;

[0026] S233: Apply the calculated weights to the gated fusion features The channels are recalibrated to obtain channel characteristics.

[0027] Preferably, step S24 includes the following steps:

[0028] S241: Channel characteristics Divide into several non-overlapping sub-blocks;

[0029] S242: Calculate sub-block features The distance d between the center of the sub-block and the center of the spectrum ij Used for frequency position coding:

[0030] d ij =||(x ij y ij )-(x c ,y c )||;

[0031] Where ||·|| represents the Euclidean distance, i represents the index of the sub-block in the row direction, j represents the index of the sub-block in the column direction, and x... ij Let xij represent the coordinates of the sub-block at position i in row i and column j on the X-axis, and yij represent the coordinates of the sub-block at position i in row i and column j on the Y-axis. c It is the coordinate value of the center point of the spectrum on the X-axis, y c It is the coordinate value of the center point of the spectrum on the Y-axis;

[0032] S243: Construct kernel-bias generators, each kernel-bias generator comprising a multilayer perceptron, the kernel-bias generator encoding d according to position. ij Dynamically generate the weight w for each sub-block ij and bias b ij ;

[0033] S244: Sub-block features With the corresponding weight w ij Perform a weighted summation to obtain an intermediate feature, and then apply the bias b. ij After being added to the intermediate feature, it is compared with the original sub-block feature. Performing element-wise multiplication, we obtain the sub-block modulation features:

[0034]

[0035] S245: After fusing all sub-block modulation features, the output features are obtained by convolutional compression.

[0036] Preferably, the step in S2 of performing channel-dimensional nonlinear modeling and semantic transformation of the frequency-adaptive semantic enhancement features using a feedforward neural network includes:

[0037] The frequency-adaptive semantic enhancement features are expanded in channel dimension through a linear transformation layer. A non-linear activation operation is applied to the expanded features, and then a feature compression layer is used to restore the channel dimension to the original dimension of the frequency-adaptive semantic enhancement features.

[0038] Preferably, in step S4, a composite supervised loss function is constructed based on the pixel-level classification difference, boundary region alignment, and frequency domain consistency between the real label and the segmentation result, and the multi-scale frequency domain enhanced feature extraction network and the segmentation decoding module are jointly optimized and trained.

[0039] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a remote sensing image segmentation method that integrates frequency modulation and spatial awareness, which has the following beneficial effects:

[0040] 1) This invention introduces a "frequency-spatial dual-domain fusion" mechanism for the first time in remote sensing image segmentation tasks, effectively solving the problems of small targets and blurred boundaries.

[0041] Traditional remote sensing segmentation models often rely on spatial domain feature extraction, making it difficult to capture the semantic information of ground objects at the frequency level. This results in unclear edge contours and weak fine-grained structure recognition. In this invention, the frequency-adaptive semantic enhancement operation utilizes Fast Fourier Transform (FFT-Shift), channel attention, and position modulation mechanisms to capture long-range dependencies in the frequency domain and enhance local detail representation in the spatial domain, significantly improving the model's segmentation performance for key areas such as building edges and road intersections.

[0042] 2) This invention achieves dynamic interaction and collaborative optimization between the frequency domain and the spatial domain, improving the robustness and semantic consistency of feature representation.

[0043] This invention constructs a bidirectional feedback mechanism between spatial and frequency domain features. Complementary enhancement of information from both domains is achieved through residual connections. This mechanism enhances the model's robustness to complex environmental interference and improves the semantic consistency and stability of the segmentation results.

[0044] 3) Design a lightweight frequency and position modulation module to realize dynamic modeling and fine adjustment of multi-scale ground structures.

[0045] This invention utilizes a position-encoded learnable modulation kernel (MLP generating weights and biases) to locally reconstruct frequency sub-blocks, effectively improving the network's adaptability to scale-sensitive regions (such as large-scale farmland versus small-scale vehicles and bridges). This mechanism not only maintains model computational efficiency but also enables fine-tuning of frequency response at different scale levels, overcoming the insufficient expressive power of existing methods in cases of uneven scale and complex target distribution, thus improving overall segmentation accuracy and generalization performance.

[0046] 4) Significantly reduces the model's dependence on large-scale labeled data, improving learning efficiency in scenarios with small sample sizes.

[0047] Because this invention introduces frequency consistency loss as an auxiliary supervision signal, this loss term provides additional structured prior information by constraining the similarity between the predicted results and the true labels in the frequency domain. Compared to traditional methods that rely solely on pixel-level cross-entropy loss, this invention can guide the model to learn more generalizable global structural features even with limited training samples. It exhibits superior convergence speed and segmentation accuracy compared to pure spatial domain models, making it suitable for practical remote sensing applications where annotation costs are high. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 A schematic diagram of the remote sensing image segmentation method that integrates frequency modulation and spatial perception provided by the present invention.

[0050] Figure 2 This is a schematic diagram of the architecture of the frequency adaptive semantic analysis module provided by the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] This invention discloses a remote sensing image segmentation method that integrates frequency modulation and spatial perception, aiming to solve the problems of difficulty in identifying small targets, blurred edge contours, and significant inter-class interference in remote sensing images.

[0053] This embodiment first constructs a multi-scale frequency domain enhanced feature extraction network, introducing a frequency-adaptive semantic analysis module into the traditional convolutional extraction framework. This effectively integrates low-frequency structural and high-frequency detail information, enhancing the ability to represent key structures such as building edges and road intersections through frequency awareness. Simultaneously, this invention designs a lightweight feedforward neural network and a residual progressive structure to progressively pass and fuse intermediate semantic information, strengthening the unified modeling capability for multi-scale ground objects (such as urban building clusters and rural farmland).

[0054] Figure 1 This diagram illustrates the logical principle of a remote sensing image segmentation method that integrates frequency modulation and spatial awareness. This step aims to address the significant differences in spatial morphology and frequency structure of multi-scale ground objects (such as buildings, roads, farmland, and water bodies) in remote sensing images by designing a feature extraction network with frequency domain modeling capabilities and a residual progressive structure. This network combines convolutional modeling, a frequency-adaptive semantic enhancement module, and a lightweight feedforward neural network to effectively extract deep semantic information while maintaining shallow spatial resolution, laying a solid feature foundation for subsequent segmentation tasks. The specific steps are as follows:

[0055] S1: Acquire the raw remote sensing image and preprocess it to generate a standardized input image.

[0056] In one embodiment, the raw remote sensing images acquired by the device are preprocessed, including geometric correction, radiometric normalization, noise filtering (such as Wiener filtering or Lee filtering), and scale normalization, to obtain a clear and uniformly scaled network input image I, ensuring the robustness and consistency of downstream feature extraction.

[0057] S2: The standardized input image is input into a multi-scale frequency domain enhancement feature extraction network, and multi-stage feature extraction is performed sequentially according to multiple feature levels. In each feature level, the input features of the current feature level are subjected to frequency domain modulation transformation, and the modulated frequency domain features are fused with the input features to obtain frequency-adaptive semantic enhancement features. The frequency-adaptive semantic enhancement features are then subjected to nonlinear modeling and semantic transformation of the channel dimension through a feedforward neural network and residual connection with the frequency-adaptive semantic enhancement features to form the output features of the current feature level, which serve as the input features of the next feature level. A cross-layer residual fusion mechanism is introduced between at least two feature levels to fuse output features of different depths, and the final multi-scale frequency domain enhancement features are output at the end feature level.

[0058] In one embodiment, before inputting the standardized input image into the multi-scale frequency domain enhanced feature extraction network in S2, the following steps are included: the input image I first passes through a standard 3×3 convolutional layer to extract initial spatial texture features, denoted as X0. Subsequently, this feature is input into the frequency adaptive semantic analysis module.

[0059] In one embodiment, the constructed frequency-adaptive semantic analysis module combines FFT-Shift, frequency modulation gating, channel attention, and position-coded modulation mechanisms. While maintaining model computational efficiency, it achieves dynamic selection and fine-grained adjustment of frequency information, thereby effectively adapting to complex terrain and changes in target distribution. Specifically, in S2, such as... Figure 2 As shown, the process of performing frequency domain transformation modulation on the input features of the current feature level and fusing the modulated frequency domain features with the input features to obtain frequency-adaptive semantic enhancement features includes the following steps:

[0060] S21: Input features for the current feature level After performing a Fast Fourier Transform (FFT), a Fast Fourier Transform Shift (FFT-Shift) is performed to move the zero-frequency components from the four corners to the center of the spectrum. The resulting features are the same as the input features. Features are obtained by performing residual connections

[0061] It should be noted that after performing a Fast Fourier Transform (FFT), the zero-frequency components of the frequency domain data are usually located at the four corners of the spectrum. However, since the four corners of the spectrum represent low-frequency information and the central region represents high-frequency information, this arrangement is not intuitive for frequency feature extraction. To make the spectrum more intuitive, this embodiment moves the zero-frequency components to the center of the spectrum in S21; this is the function of FFT-Shift. For a two-dimensional signal (image), FFT-Shift performs one-dimensional FFT-Shift operations in both the X and Y axes, moving the zero-frequency components from the four corners to the center of the spectrum. The resulting features are in line with... Features are obtained by performing residual connections

[0062] S22: Features After normalization, a 1×1 convolutional layer and a 3×3 depthwise separable convolutional layer are sequentially input to extract local and spread frequency information. Based on a lightweight gating mechanism, the gated fusion features corresponding to the local and spread frequency information are output.

[0063] In this embodiment, the lightweight gating mechanism in S22 includes:

[0064] S221: The output features of the depthwise separable convolutional layer are divided into two sub-feature maps by channel segmentation, denoted as follows: and

[0065] S222: The two sub-feature maps are input to two convolutional layer branches with different kernel sizes, and the following is executed:

[0066]

[0067] The outputs of the two convolutional layer branches are multiplied element-wise to obtain gated fusion features, thereby achieving gated modulation:

[0068]

[0069] This process enhances semantic expressiveness through element-level modulation and suppresses redundant frequency interference.

[0070] S23: Gated fusion features based on lightweight channel attention mechanism Perform channel feature enhancement to obtain channel features

[0071] In this embodiment, the lightweight channel attention mechanism in S23 is used to enhance the network's attention to important channel features while suppressing unimportant channel features. It includes:

[0072] S231: Gated Fusion Features The data is fed into a global average pooling layer for spatial information aggregation to obtain global statistics for each channel.

[0073] S232: Global statistics are processed through a 1×1 convolutional layer to generate weights for each channel;

[0074] S233: Apply the calculated weights to the gated fusion features The channels are recalibrated to obtain channel characteristics.

[0075] S24: Channel Features The system is divided into multiple spatial sub-regions. Based on the positional relationship of each spatial sub-region relative to the spectral center, a positional code characterizing its frequency spatial distribution is generated. A kernel-bias generator is used to dynamically generate parameters for feature modulation using the positional code, and the frequency domain features of each spatial sub-region are weighted and biased to obtain the frequency position-modulated output features.

[0076] In this embodiment, S24 is used to construct adaptive frequency position modulation, including the following steps:

[0077] S241: Perform feature segmentation: divide channel features Divide into several non-overlapping sub-blocks, each sub-block having a size of p×p;

[0078] S242: Perform position encoding: calculate sub-block features The distance d between the center of the sub-block and the center of the spectrum ij Used for frequency position coding:

[0079] d ij =||(x ij ,y ij )-(x c ,y c )||;

[0080] Where ||·|| represents the Euclidean distance, i represents the index of the sub-block in the row direction, j represents the index of the sub-block in the column direction, and x... ij Let xij represent the coordinates of the sub-block at position i in row i and column j on the X-axis, and yij represent the coordinates of the sub-block at position i in row i and column j on the Y-axis. c It is the coordinate value of the center point of the spectrum on the X-axis, y c It is the coordinate value of the center point of the spectrum on the Y-axis;

[0081] S243: Constructing the kernel-bias generator: Each kernel-bias generator consists of a multilayer perceptron (MLP) containing two linear layers and a GELU activation function. The kernel-bias generator encodes d according to position. ij Dynamically generate the weight w for each sub-block ij and bias b ij ;

[0082] S244: Perform feature modulation and weighted summation: combine sub-block features With the corresponding weight w ij Perform a weighted summation to obtain an intermediate feature, and then apply the bias b. ij After being added to the intermediate feature, it is compared with the original sub-block feature. Performing element-wise multiplication, we obtain the sub-block modulation features:

[0083]

[0084] S245: After fusing all sub-block modulation features, perform 1×1 convolution compression to obtain the frequency position modulated output features:

[0085]

[0086] Where ∪ represents the set union operation, which performs a union operation on all elements at position i in row j. Do the union.

[0087] S25: Channel Features Output characteristics after frequency and position modulation The modulation is achieved by performing addition and fusion, followed by sequential passing through a 1×1 convolutional layer, inverse fast Fourier transform shift, and inverse fast Fourier transform to obtain the modulated frequency domain features.

[0088] The frequency-adaptive semantic analysis module enhances the segmentation capability of remote sensing images through a combination of frequency domain sensing enhancement and frequency-position adaptive modulation. On one hand, it utilizes Fast Fourier Transform shifting and residual connection operations to effectively extract global frequency distribution features from the image, enhancing the model's ability to represent small targets and edge contours. On the other hand, it introduces a learnable modulation kernel based on position coding to achieve fine-tuning of the feature space, dynamically adapting to frequency components at different scales, thereby better characterizing the complex and diverse land cover structures in remote sensing images.

[0089] In one embodiment, the steps in S2 of performing channel-dimensional nonlinear modeling and semantic transformation of the frequency-adaptive semantic enhancement features using a feedforward neural network include:

[0090] The frequency-adaptive semantic enhancement features are expanded in channel dimension through a linear transformation layer. A non-linear activation operation is applied to the expanded features, and then a feature compression layer is used to restore the channel dimension to the original dimension of the frequency-adaptive semantic enhancement features.

[0091] In practice, this embodiment aims to perform nonlinear modeling and semantic transformation on the extracted spatial features to enhance the expressive power of the feature representation. For example... Figure 1 As shown, the first feature level is used as an example for explanation: Let the input feature be X1, which comes from the preliminary encoded features of the previous frequency adaptive semantic analysis module and contains key information about multi-scale targets and texture boundaries in remote sensing images.

[0092] Feedforward neural networks mainly consist of the following parts:

[0093] Linear Transformation Layer: First, the input feature X1 is expanded in dimension through a 1×1 convolutional layer to improve the model's non-linear modeling capability.

[0094] X a =Conv(X1);

[0095] Here, Conv represents a convolutional layer.

[0096] Activation function (non-linear transformation): for feature X a Applying nonlinear activation operations (such as GELU or ReLU) enhances the model's ability to represent complex ground features in remotely sensed images.

[0097] X b =GELU(X a );

[0098] Dropout regularization: To prevent overfitting, the Dropout operation can be added, which is particularly suitable for scenarios with small sample remote sensing datasets.

[0099] X c =Dropout(X b );

[0100] It should be noted that the Dropout regularization operation can be omitted according to actual needs and does not affect the functionality of the feedforward neural network.

[0101] Feature compression layer: A 1×1 convolution is then used to restore the channel dimensions to their original dimensions, forming the output features.

[0102] X′1=Conv(X c );

[0103] This feedforward network structure helps to enhance the representation of ground objects in remote sensing images without introducing too many parameters, especially demonstrating good modeling ability for complex structures such as blurred boundaries, small-scale buildings, farmland, and roads. Finally, feature X′1 and feature X1 are residually connected to obtain feature X″1.

[0104] The following explains feature-based progressive modeling and multi-stage residual fusion:

[0105] To further enhance the model's ability to model ground features, a hierarchical feature extraction process is designed. Each stage repeats the "frequency enhancement + feedforward network" processing logic, and a cross-layer residual fusion mechanism is introduced at key levels to strengthen information flow. The specific implementation process is as follows: Figure 1 As shown:

[0106] The input image I first passes through a standard 3×3 convolutional layer to extract initial spatial texture features X0. The input frequency adaptive semantic analysis module then obtains enhanced features that incorporate global information from the frequency domain. The two are fused element-wise to achieve frequency spatial complementarity, resulting in feature X1. This operation improves the network's ability to perceive small-scale targets (such as rooftops and bridges) and edge structures (such as road boundaries and field dividing lines). Feature X1 is fed forward into a feedforward neural network to form feature X′1. Feature X′1 and feature X1 are residually connected to obtain the output feature X″1 of the first feature layer, which is then used as the input of the second feature layer.

[0107] The feature X2 is obtained through the same processing flow of the frequency adaptive semantic analysis module in the second feature level. Then, the output feature X″2 of the second feature level is obtained through the same processing flow of the feedforward neural network and residual connection in the second feature level, and is used as the input of the third feature level.

[0108] The feature X3 is obtained through the same processing flow of the frequency adaptive semantic analysis module in the third feature level. Then, the output feature X″3 of the third feature level is obtained through the same processing flow of the feedforward neural network and residual connection in the third feature level, and is used as the input of the third feature level.

[0109] After undergoing the same processing flow as the frequency adaptive semantic analysis module in the fourth feature level, feature X4 is obtained. After going through the same processing flow as the feedforward neural network and residual connection in the fourth feature level, this level introduces a cross-layer residual fusion mechanism, which performs residual connection on the output features X″2, X4, and X′4 of the second feature level to obtain feature X″4, which is then used as the input of the fifth feature level.

[0110] After undergoing the same processing flow as the frequency adaptive semantic analysis module in the fifth feature level, feature X5 is obtained. After going through the same processing flow as the feedforward neural network and residual connection in the fifth feature level, this level also introduces a cross-layer residual fusion mechanism. The output features X″1, X5, and X′5 of the first feature level are residually connected to obtain feature X″5, which is then used as the input of the sixth feature level.

[0111] The feature X6 is obtained by going through the same process as the frequency adaptive semantic analysis module in the sixth feature level. Then, the feature X′6 is obtained by going through the same process as the feedforward neural network and residual connection in the sixth feature level, and is used as the input of the segmentation head.

[0112] This implementation process stacks and fuses intermediate semantic features layer by layer to improve the unified recognition capability of large-scale terrain and small-scale buildings in remote sensing images; it also mitigates information loss and improves the segmentation accuracy of fine-grained targets through cross-stage feature fusion.

[0113] S3: The segmentation and decoding module decodes the final multi-scale frequency domain enhancement features obtained through multi-stage progressive modeling, restores the spatial resolution, and generates pixel-level segmentation results.

[0114] This step aims to convert the deeply extracted multi-scale semantic features X′6 into pixel-level remote sensing image segmentation results. The process includes three parts: decoding and restoration, segmentation prediction, and loss function supervision, comprehensively improving the model's ability to model and accurately locate invariant features such as edge contours, small targets, and ground object boundaries in remote sensing images.

[0115] In this embodiment, S3 is used to perform feature decoding and resolution restoration: To gradually restore the spatial resolution and enhance the supporting role of location information for segmentation, the finally extracted deep feature X′6 is first input into a segmentation head module. This module uses bilinear interpolation or transposed convolution to upsample the feature map to the original image size and then connects it to a 1×1 convolutional layer for channel compression to obtain the predicted segmentation probability map.

[0116] P = Softmax(Conv) 1×1 (Upsample(X′6)));

[0117] in, C represents the number of categories, indicating the probability distribution of each pixel belonging to each category.

[0118] S4: Construct a composite supervised loss function and jointly optimize and train the multi-scale frequency domain enhanced feature extraction network and the segmentation and decoding module to perform segmentation operations on the remote sensing image to be processed based on S1 to S3.

[0119] In one embodiment, during the decoding stage, the present invention introduces a multi-task supervision mechanism based on frequency consistency. Based on the pixel-level classification difference between the real label and the segmentation result, the degree of alignment of the boundary region, and the frequency domain consistency, a composite supervision loss function is constructed to jointly optimize and train the multi-scale frequency domain enhanced feature extraction network and the segmentation decoding module.

[0120] In this embodiment, to improve the segmentation accuracy of fine structures in remote sensing images, a comprehensive supervised loss function is constructed. It integrates classic cross-entropy loss, boundary-aware loss, and frequency consistency loss, as detailed below:

[0121] Cross-entropy Loss:

[0122] Used to optimize the difference between pixel-level classification probabilities and ground truth labels, defined as:

[0123]

[0124] Where G is the true label image, P i,c This represents the predicted probability that the i-th pixel belongs to the c-th class.

[0125] Boundary-Aware Loss:

[0126] To enhance the perception of the edges of ground features (such as building boundaries and road boundaries) in remote sensing images, a boundary loss function based on edge gradient weighting is introduced:

[0127]

[0128] Among them, D i Assigning importance weights to boundary regions (such as those generated based on edge detection) can effectively improve the model's sensitivity to edge transition regions.

[0129] Frequency consistency loss:

[0130] To consider the frequency feature consistency of remote sensing images, an L2 consistency term in the frequency space is introduced as an auxiliary supervision:

[0131]

[0132] The final loss function is a weighted sum of all components:

[0133]

[0134] Where λ1, λ2, and λ3 are the weight hyperparameters of the loss term, which can be flexibly adjusted according to different data scenarios.

[0135] This step further improves the model's accuracy in recognizing fine ground features and its responsiveness to boundary transition regions by combining cross-entropy loss, boundary-aware loss, and frequency consistency loss. Through a multi-layered decoding structure and multi-dimensional loss design, it not only achieves accurate mapping from deep features to pixel-level segmentation maps, but also further enhances the accuracy and robustness of remote sensing image segmentation through boundary enhancement and frequency consistency supervision. It is particularly suitable for remote sensing image segmentation tasks in complex scenes where ground features are sparsely distributed, have blurred boundaries, or suffer from severe inter-class interference.

[0136] In summary, the embodiments of the present invention construct a multi-scale frequency domain enhanced feature extraction network, combine a frequency adaptive semantic analysis module and a decoding supervision mechanism, and fully explore the semantic correlation characteristics of remote sensing images in the frequency domain and spatial domain, thereby improving the segmentation accuracy and robustness of ground targets in complex scenes.

[0137] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0138] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image segmentation method integrating frequency modulation and spatial awareness, characterized in that, Includes the following steps: S1: Acquire the raw remote sensing image and preprocess it to generate a standardized input image; S2: The standardized input image is input into a multi-scale frequency domain enhancement feature extraction network, and multi-stage feature extraction is performed sequentially according to multiple feature levels; in each feature level, the input features of the current feature level are subjected to frequency domain modulation transformation, and the modulated frequency domain features are fused with the input features to obtain frequency adaptive semantic enhancement features; after performing nonlinear modeling and semantic transformation of the channel dimension of the frequency adaptive semantic enhancement features through a feedforward neural network, a residual connection is formed with the frequency adaptive semantic enhancement features to form the output features of the current feature level, which are used as the input features of the next feature level; A cross-layer residual fusion mechanism is introduced between at least two feature levels to fuse output features at different depths, and the final multi-scale frequency domain enhanced features are output at the end feature level; wherein, the steps to obtain frequency-adaptive semantic enhanced features include: S21: Input features for the current feature level After performing a Fast Fourier Transform (FFT), a subsequent FFT shift is performed to move the zero-frequency components from the four corners to the center of the spectrum. The resulting features are the same as the input features. Features are obtained by performing residual connections ; S22: Features After normalization, the data is sequentially input into the first convolutional layer and the depthwise separable convolutional layer to extract local and spread frequency information. Based on a lightweight gating mechanism, the gated fusion features corresponding to the local and spread frequency information are output. ; S23: Gated fusion features based on lightweight channel attention mechanism Perform channel feature enhancement to obtain channel features ; S24: Channel Features The system is divided into multiple spatial sub-regions. Based on the positional relationship of each spatial sub-region relative to the spectral center, a position code characterizing its frequency spatial distribution is generated. A kernel-bias generator is used to dynamically generate parameters for feature modulation using these position codes, and the frequency domain features of each spatial sub-region are weighted and biased to obtain the frequency position modulated output features. ; S25: Determine the channel features and the output characteristics after frequency position modulation The modulation is achieved by adding and fusing the components, then sequentially passing them through a third convolutional layer, an inverse fast Fourier transform shift, and another inverse fast Fourier transform to obtain the modulated frequency domain features. ; S3: The segmentation decoding module decodes the final multi-scale frequency domain enhancement features obtained through multi-stage progressive modeling to restore spatial resolution and generate pixel-level segmentation results. S4: Construct a composite supervised loss function and jointly optimize and train the multi-scale frequency domain enhanced feature extraction network and the segmentation decoding module to perform segmentation operations on the remote sensing image to be processed according to S1~S3.

2. The remote sensing image segmentation method fusing frequency modulation and spatial perception according to claim 1, characterized in that, Before inputting the standardized input image into the multi-scale frequency domain enhancement feature extraction network in step S2, the method further includes: first passing the standardized input image through a convolutional layer to extract initial spatial texture features, and then using the initial spatial texture features as input to the multi-scale frequency domain enhancement feature extraction network.

3. The remote sensing image segmentation method fusing frequency modulation and spatial awareness according to claim 1, characterized in that, The lightweight gating mechanism in S22 includes: S221: The output features of the depth-separable convolutional layer are divided into two sub-feature maps by channel segmentation; S222: The two sub-feature maps are respectively input to two convolutional layer branches with different kernel sizes, and the outputs of the two convolutional layer branches are multiplied element-wise to obtain the gated fusion features.

4. The remote sensing image segmentation method fusing frequency modulation and spatial awareness according to claim 1, characterized in that, The lightweight channel attention mechanism in S23 includes: S231: The gated fusion feature The data is fed into a global average pooling layer for spatial information aggregation to obtain global statistics for each channel. S232: A second convolutional layer processes global statistics to generate weights for each channel; S233: Apply the calculated weights to the gated fusion features The channels are recalibrated to obtain channel characteristics. .

5. The remote sensing image segmentation method fusing frequency modulation and spatial perception according to claim 1, characterized in that, S24 includes the following steps: S241: Channel characteristics Divide into several non-overlapping sub-blocks; S242: Calculate sub-block features Distance between the center of the sub-block and the center of the spectrum Used for frequency position coding: ； in, Represents Euclidean distance. i This indicates the index of the sub-block in the row direction. j This represents the index of the sub-block in the column direction. x ij Indicates the first i Line number j The coordinates of the sub-block at column position on the X-axis, yij represents the first... i Line number j The Y-coordinate of the sub-block at the column position. x c It is the coordinate value of the center point of the spectrum on the X-axis. y c It is the coordinate value of the center point of the spectrum on the Y-axis; S243: Construct kernel-bias generators, each kernel-bias generator comprising a multilayer perceptron, the kernel-bias generator encoding based on position. Dynamically generate the weight of each sub-block and bias ; S244: Sub-block features With the corresponding weights Perform a weighted summation to obtain an intermediate feature, and then apply the bias. After being added to the intermediate feature, it is compared with the original sub-block feature. Performing element-wise multiplication, we obtain the sub-block modulation features: ； S245: After fusing all sub-block modulation features, the output features are obtained by convolutional compression.

6. The remote sensing image segmentation method fusing frequency modulation and spatial awareness according to claim 1, characterized in that, The steps in S2, which involve nonlinear modeling and semantic transformation of the frequency-adaptive semantic enhancement features along the channel dimension using a feedforward neural network, include: The frequency-adaptive semantic enhancement features are expanded in channel dimension through a linear transformation layer. A non-linear activation operation is applied to the expanded features, and then a feature compression layer is used to restore the channel dimension to the original dimension of the frequency-adaptive semantic enhancement features.

7. The remote sensing image segmentation method fusing frequency modulation and spatial perception according to claim 1, characterized in that, In step S4, a composite supervised loss function is constructed based on the pixel-level classification difference, boundary region alignment, and frequency domain consistency between the real label and the segmentation result, and the multi-scale frequency domain enhanced feature extraction network and the segmentation decoding module are jointly optimized and trained.

Images