Remote sensing image semantic segmentation method based on multi-ellipse frequency domain mask separation and space-frequency fusion mechanism

Abstract

This invention specifically relates to a semantic segmentation method for remote sensing images based on a multi-elliptical frequency domain masking separation and spatial-frequency fusion mechanism, belonging to the field of deep learning and image processing. The method includes: extracting multi-scale features of the remote sensing image through a backbone network; designing a PED Block module to achieve the separation and fusion of high- and low-frequency spatial components and high- and low-frequency frequency components; extracting frequency components through a multi-elliptical frequency domain masking separation mechanism combined with prior ground information; employing a spatial-frequency dual-domain interactive attention fusion mechanism to perform cross-attention and gating fusion of spatial and frequency features; gradually restoring details through residual fusion and multi-stage decoding; and finally outputting the semantic segmentation result through a segmentation head. This invention can adaptively divide the frequency domain and fuse spatial-frequency features, significantly improving edge clarity and category discrimination ability in complex scenes, and effectively alleviating the problems of boundary blurring and semantic confusion.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and image processing, specifically to a semantic segmentation method for remote sensing images based on a multi-ellipse frequency domain mask separation and space-frequency fusion mechanism. Background Technology

[0002] As a core task in scenarios such as Earth observation, resource surveys, agricultural production, industrial manufacturing, and medical testing, semantic segmentation of remote sensing images achieves automated identification of surface structures by assigning a semantic category to each pixel in a remote sensing image. However, remote sensing images exhibit significant complexity in terms of spatial structure, texture patterns, and imaging conditions, leading to multiple challenges for existing methods in terms of accuracy, robustness, and generalization ability.

[0003] Zeng Junying (“A Multi-Level Branched Cross-Scale Fusion Semantic Segmentation Network for Remote Sensing Images”, Advances in Laser & Optoelectronics, 2024, 1-20) constructed a multi-level branched network structure, designing a shallow Swin Transformer feature extraction module, spatial branches, semantic branches, and boundary branches. Each branch focuses on extracting feature information at a specific level, and a multi-scale decoding module with a large receptive field is introduced to transmit feature information at different scales. However, spatial domain modeling of spatial domain convolution methods lacks the transmission of spectral structure information, and is prone to problems such as blurred boundaries, adhesion, or breakage when encountering texture features with significant directionality or periodicity, resulting in unclear inter-class distinctions, especially with a high segmentation error rate in complex mixed urban and rural areas. Swin Transformer captures long-range dependencies through a self-attention mechanism, but also lacks the ability to explicitly encode frequency information.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This invention proposes a remote sensing image semantic segmentation method guided by ground feature priors and based on multi-elliptical frequency domain mask separation and spatial-frequency fusion mechanism. This method addresses the problems of edge blurring, category confusion, and insufficient generalization ability caused by the separation of spatial and frequency domain information and the neglect of ground feature prior knowledge in existing remote sensing image semantic segmentation methods. It achieves effective fusion and adaptive enhancement of spatial and frequency dual-domain features to improve segmentation accuracy and robustness in complex scenarios.

[0006] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0007] According to a first aspect of the present invention, a method for semantic segmentation of remote sensing images based on a multi-elliptic frequency domain mask separation and space-frequency fusion mechanism is provided, the method comprising: Input a remote sensing image and extract multi-scale feature maps through a backbone network; the multi-scale features include first-stage feature maps, second-stage feature maps, third-stage feature maps and fourth-stage feature maps; The fourth-stage feature map and the original image are input into the first PED Block module, and the output is... ,Will After channel alignment and bilinear interpolation, the feature map is then concatenated with the third-stage feature map and aligned with the channel to obtain the first decoded feature map. The first decoded feature map is first processed by the PGMAM module to obtain the first land cover prior information. The first decoded feature map, the first prior information, and the original image are then input into the second PED Block module, and the output is... ,Will After channel alignment and bilinear interpolation, the feature map is then concatenated with the second-stage feature map and aligned with the channels to obtain the second decoded feature map. The second decoded feature map is first processed by the PGMAM module to obtain the second land cover prior information. The second decoded feature map, the second prior information, and the original image are then input into the third PED Block module, which outputs... ,Will After channel alignment and bilinear interpolation, the feature map is then concatenated with the first-stage feature map and aligned with the channels to obtain the third decoded feature map. The third decoded feature map is processed through the PGMAM module and then projected. It is then concatenated with the residual of the third decoded feature map and input into the segmentation head to output the semantic segmentation result.

[0008] In some exemplary embodiments, the PED Block module's processing includes: High-frequency spatial components and low-frequency spatial components are extracted by high-frequency and low-frequency convolution branches, respectively. The high-frequency spatial components, low-frequency spatial components, original image, and prior information of ground features are input into the PED Module. The high-frequency and low-frequency components are extracted through a multi-ellipse frequency domain mask separation mechanism. The first PEDBlock module does not have prior information of ground features. The high-frequency spatial component, low-frequency spatial component, high-frequency frequency component, and low-frequency frequency component are input into the spatial-frequency dual-domain interactive attention fusion mechanism module to perform cross-attention fusion and gated fusion, and output the fused high-frequency features and low-frequency features. The fused high-frequency and low-frequency features are then residually fused with the original features to obtain the decoding features for the current stage.

[0009] In some exemplary embodiments, the multi-elliptic frequency domain mask separation mechanism includes: The interpolated original image and the prior ground features are concatenated along the channel dimension, and a convolution operation is performed to obtain the base features; Global average pooling, convolution, activation, and normalization are performed on the base features to obtain the multi-ellipse mask parameters; Multiple elliptical binary masks are generated based on the parameters, and the union of these masks is used to obtain a joint mask. A two-dimensional fast Fourier transform is performed on the base features, and the joint mask is used to separate the high-frequency and low-frequency components. By performing an inverse transform on the divided high-frequency and low-frequency components, different frequency domain components with high and low frequency divisions are obtained.

[0010] In some exemplary embodiments, the space-frequency dual-domain interactive attention fusion mechanism includes: Multi-head self-attention calculations are performed on the high-frequency frequency domain component, the low-frequency frequency domain component, the high-frequency spatial domain component, and the low-frequency spatial component respectively; Cross-attention calculation is performed on the frequency domain components and the spatial domain components to obtain the fused high-frequency enhancement features and low-frequency enhancement features. The fused high-frequency features and low-frequency features are concatenated along the channel dimension, and a fusion weight is generated through a gating mechanism for weighted fusion.

[0011] In some exemplary embodiments, the prior information of ground features includes category distribution priors and semantic prediction information, which are used to adaptively adjust the spectrum partitioning.

[0012] In some exemplary embodiments, the residual fusion adopts an element-wise addition method to fuse the fused features with the backbone network features of the corresponding scale.

[0013] In some exemplary embodiments, the segmentation head includes at least one convolutional layer and an upsampling layer for mapping the final decoded features to the original image resolution and outputting a semantic segmentation map.

[0014] According to a second aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the remote sensing image semantic segmentation method based on the multi-elliptic frequency domain mask separation and space-frequency fusion mechanism described in the first aspect above.

[0015] According to a third aspect of the present invention, a computer program product is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the remote sensing image semantic segmentation method based on the multi-elliptic frequency domain mask separation and space-frequency fusion mechanism described in the first aspect above.

[0016] According to a fourth aspect of the present invention, an electronic device is provided, comprising: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to implement the remote sensing image semantic segmentation method based on multi-elliptic frequency domain mask separation and space-frequency fusion mechanism described in the first aspect by executing the executable instructions.

[0017] The remote sensing image semantic segmentation method based on multi-elliptic frequency domain mask separation and space-frequency fusion mechanism provided by the embodiments of the present invention has the following advantages compared with the prior art: 1. The multi-elliptical frequency domain masking separation mechanism proposed in this invention adaptively adjusts the high- and low-frequency division based on the spectral structure of the input image, significantly enhancing the high-frequency response and structural contour features at the edges of ground features. In complex scenes, it can more accurately distinguish between real boundaries and texture noise, effectively reducing edge blurring, jaggedness, and boundary adhesion, significantly improving the model's ability to recognize continuous boundaries, sharp contours, and directional structures, achieving clearer and more coherent semantic segmentation.

[0018] 2. The spatial-frequency dual-domain attention fusion mechanism proposed in this invention simultaneously models the semantic relationships in the spatial domain and the texture structure in the frequency domain. It achieves complementarity and enhancement of the two types of features through bidirectional attention fusion, and dynamically balances local details and global semantics in a gating manner. This improves the detection capability of targets at extremely small scales, effectively reduces false detections and false negatives, and enhances the regional consistency and overall structure preservation capability of large and medium-scale features, so that block structures such as building outlines present continuous and stable semantic regions.

[0019] 3. The adaptive land cover prior mask module proposed in this invention utilizes prior information on category distribution and semantic prediction to adaptively adjust the spectral division, making the frequency domain representation of different land cover categories more consistent with their inherent spectral characteristics. This significantly improves the discriminative ability between categories, especially among categories with similar spectra and textures but different semantics. Furthermore, for land cover with strong structural characteristics, this module effectively enhances its feature representation, resulting in a more stable and accurate recognition capability for the model.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0022] Figure 1 This is a diagram of the PEDNet network structure of the present invention; Figure 2 A schematic diagram of the MEFDM module; Figure 3 This is a schematic diagram of the CDAFM module; Figure 4 For visualizing the results. Detailed Implementation

[0023] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0024] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] In existing remote sensing image semantic segmentation techniques, spatial domain methods lack the ability to consider both global and local aspects, Transformer and its variants struggle to encode frequency domain structural information, and existing frequency domain methods lack geometric and semantic adaptability and generally ignore prior knowledge of ground features. These problems collectively lead to models exhibiting issues such as blurred boundaries, category confusion, and insufficient generalization ability in complex scenes. Therefore, to address these bottlenecks, this invention proposes a prior-guided multi-elliptical frequency masking and dual-domain fusion mechanism for remote sensing image semantic segmentation (PEDNet).

[0026] One challenge is that fixed-shape frequency domain masks cannot effectively adapt to the anisotropic characteristics of ground features in remotely sensed images when performing spectral separation, especially in complex scenes and irregular structures where the energy distribution and directionality of the spectrum exhibit significant variations and differences. To address this issue, this invention proposes a Multi-Elliptical Frequency Decomposition Module (MEFDM) mechanism. This mechanism uses learnable elliptical masks to achieve flexible frequency partitioning in terms of direction and scale, ensuring that the frequency domain decomposition matches the ground feature structure from a geometric perspective. This effectively alleviates common problems in previous models such as edge blurring, adhesion, and breakage, resulting in clearer and more coherent edge detection.

[0027] In traditional spatial-frequency joint modeling, frequency domain features and spatial features are often processed separately, lacking an effective collaborative modeling mechanism. This results in an inability to fully exploit the complementary information between the two, particularly when dealing with the consistency between details and global semantics. To address this issue, this invention proposes a cross-domain attention-fusion module (CDAFM) to achieve complementary fusion between frequency domain features and spatial semantics. This enables collaborative optimization of global structural understanding and local boundary details, allowing the model to more accurately identify the overall category of structural targets while simultaneously solving the problem of small target loss caused by continuous downsampling in traditional spatial convolution.

[0028] In addition, previous frequency domain separation methods neglected the semantic information of ground features and could not flexibly adjust the frequency domain mask according to the feature differences of different ground feature categories, which easily led to problems such as missegmentation and blurred boundaries. To solve this problem, this invention designs a Prior-Guided Masking-Adaptation Module (PGMAM). This module dynamically adjusts the shape and boundary of the frequency domain mask by introducing prior information of ground feature categories. Especially when dealing with targets with special structures, it can achieve more accurate frequency domain separation and enhancement, thereby improving category discrimination and cross-regional generalization ability, and significantly improving the accurate identification ability of targets with specific structures.

[0029] Furthermore, the high- and low-frequency multi-ellipse mask separation mechanism is explained in detail: 1. Parameterization and Mask Generation As attached Figure 1 Schematic diagram and appendix of PED Module Figure 2 As shown, one of the inputs to MEFDM is the width and height after interpolation transformation, compared with the features of the current layer. Figure 1The original image is then interpolated. The interpolated original image is concatenated with the prior image along the channel dimension, and a convolution operation is performed on the concatenated input to obtain the base features of the frequency domain decomposition.

[0030] in, Indicates interpolation transformation, Indicates the kernel size as The convolution transform function, This indicates a splicing operation based on the channel dimension. These are prior features.

[0031] Features of the base Perform global average pooling and 2 layers Convolution, GELU activation, and Sigmoid normalization yield the normalized parameters used to construct the multielliptic mask:

[0032] in Indicates global average pooling. Represents convolution transformation, This represents the GELU activation function. This represents the Sigmoid activation function.

[0033] Will Transform and copy along the channel as Therefore, the parameters for each ellipse are:

[0034] And inversely normalize to pixel scale according to the input space size:

[0035] For grid coordinates , No. The binary mask for each ellipse is:

[0036] The union of multiple elliptical joint masks is taken, and in Cut off at this point, that is:

[0037] 2. Frequency Domain Separation and Inverse Transform Features of the base Perform a two-dimensional Fast Fourier Transform (FFT) and then center-shift:

[0038] in, Represents the two-dimensional Fast Fourier Transform function. This represents the centered translation function.

[0039] by For the high- and low-frequency channel divider, we can obtain:

[0040] in This is a matrix consisting entirely of 1s. Then, perform a complete inversion operation:

[0041] in This indicates the inverse of the two-dimensional Fourier transform. This represents the inverse of a centralized translation.

[0042] Last use The convolution of the size adjusts both to have the same width, height, and channel values ​​as the output:

[0043] Therefore, the output of this module is: High-Frequency Frequency-domain Component (HFFC), represented as follows: The low-frequency frequency-domain component (LFFC) is represented as... .

[0044] Furthermore, the dual-domain self-attention and gating fusion mechanism is explained in detail: 1. Self-attention in the frequency and spatial domains As attached Figure 1 PED Module and Appendix Figure 2 As shown, first consider the frequency domain branching features. Spatial domain branching characteristics Multi-head self-attention mechanism computation is performed separately. For example, first stretch the feature to a length of Token sequence:

[0045] Soon to be the original The feature map is stretched to a length of The token sequence, the number of channels is changed from the original number through convolution operations. Transform into The self-attention operation for each "feature head" in the multi-head attention mechanism is as follows:

[0046] Among them, parameters For parameters With parameters The number of channels (dimensions). After concatenating the multiple heads, the size is mapped back to the feature map size. Similarly, for features Executing them separately yields the following results:

[0047] The resulting four features This is the input to the CDFAM module. Among them, These represent computations performed using the self-attention mechanism on frequency domain features and spatial domain features, respectively.

[0048] 2. Frequency-space cross-attention High frequency characteristics For example, let

[0049] in, Indicates cross attention. This represents convolutional transformation and nonlinear activation. This indicates that high-frequency features are obtained by splicing channels together. Similarly, get .

[0050] 3. Gating fusion After the above steps This indicates the fused high-frequency characteristics, which combine the original high-frequency domain components with the spatially guided high-frequency domain enhancement components. This represents a fused low-frequency feature that integrates the original low-frequency domain components and the spatially guided low-frequency domain enhancement components. The two cross-domain features are spliced ​​together in the channel dimension and processed through two layers. Convolution and non-linear activation yield pixel-wise and channel-wise gating coefficients:

[0051] The final output features complementary bidirectional fusion as follows:

[0052] Furthermore, a detailed explanation is provided regarding the dynamic adjustment of the frequency domain mask guided by prior knowledge of ground features: 1. Prior injection and channel alignment This mechanism uses prior knowledge of ground features Compared to the original image (The image or feature map of a certain layer) is concatenated along the channel dimension and then convolved to enter the frequency domain decomposition branch:

[0053] Among them, prior Generated from the features of the previous level during the decoding process:

[0054] in For the first Layer fusion features, as shown in the appendix Figure 1 The PEDNet network structure diagram is shown below. . Represents convolution. This represents the Sigmoid activation function.

[0055] 2. Coupling of Prior Knowledge and Masking Due to ellipse parameters Directly by After global average pooling GAP, It is obtained through convolution, GELU activation, and sigmoid normalization; therefore, its gradient chain is:

[0056] Therefore, prior It will explicitly affect the frequency domain mask. ,(through (Deterministic geometric transformation) enables dynamic modulation of frequency band selection based on semantic priors, ultimately affecting the composition of high-frequency and low-frequency features and the representational ability of subsequent branches.

[0057] 3. Explicit injection prior to the terminal Decoding features at the lowest layer Here, the present invention introduces another linear mapping:

[0058] in for Convolution transformation. pass Projected onto After obtaining the same channel dimension, residual fusion is used to obtain the features before terminal semantic prediction:

[0059] Then Input split head Generate final prediction:

[0060] in, The function represents the convolution transformation.

[0061] like Figure 1 As shown, the PED Module on the lower right is part of the PED Block, and the PED Block module is one of the core components of the entire PEDNet network structure. This invention will describe the specific implementation methods of the invention in detail, following a logical order from general to specific.

[0062] Let the original input image be... The input image has a width and height of 512 and 3 channels. This represents the number of input images in each batch during network training and testing, i.e., the batch size. Let... These represent the height and width of the current layer feature map, respectively, and the prior features are denoted as... Parameters in the experiment This represents the number of prior channels, typically set to 1, meaning a single-layer feature map is used to represent prior ground features. The number of multiple ellipses is... Parameters in the experiment The value is 4.

[0063] Taking the commonly used remote sensing image semantic segmentation dataset, Vaihingen dataset, as an example, such as Figure 1 As shown in the network structure diagram above, input a remote sensing image. Size is This indicates that the height and width are both 512, and the number of channels is 3. The backbone network is ConvNeXt-tiny. After preliminary feature extraction from the backbone network, the multi-scale feature maps are obtained and denoted as follows:

[0064] First, the fourth-stage feature map Compared to the original image (Represented by the green dashed line in the diagram) The input is fed into the first PEDBlock module, and the output is denoted as... Channel alignment is achieved through convolution, resolution alignment is achieved through bilinear interpolation, and then the data is combined with the feature map from the third stage. Channel alignment is achieved by concatenating the channels and then performing convolution again. In the diagram, "CA&Ip&Cat&CA" represents a total of 4 steps: channel alignment, bilinear interpolation, feature map concatenation, and channel alignment.

[0065] After the above operations, the decoded feature map is obtained, denoted as . . First, obtain the corresponding prior information through the PGMAM module, denoted as... Thus, the three feature maps are now complete. , Compared to the original image The input is fed into the second PED Block module, and the output is denoted as... Similarly, the decoded feature maps were subsequently obtained. ,a priori and the original image Together, they are fed into the third and final PED Block module, and the output is denoted as... Compared with the first stage feature map After passing through the "CA&Ip&Cat&CA" module, the decoding feature map for this stage is output and denoted as... Then it passes through the PGMAM module and is projected, and then... The residual connection takes the input segmentation head as input and outputs the entire model. The output feature map has the same dimension as the input, i.e., its size is [size missing]. .

[0066] For the PED Block modules represented by the yellow boxes in the overall network structure diagram PEDNet, Figure 1 The lower left sub-graph illustrates its input, output, and execution process. The input to a PED Block includes the feature map, the original image, and prior information. The first PED Block has no prior input, while the second and third PED Blocks both have prior input from the feature map of the previous stage. Original image First, an interpolation operation is performed to transform the size of the feature map, resulting in a feature map with the same width and height as the input feature map, denoted as [the feature map]. The feature map size is denoted as... , where parameters These represent the number of channels, height, and width of the feature map, respectively. Different PEDBlocks correspond to different inputs; the input feature map for the first PED Block is shown below. It is a feature map , corresponding to the input feature map of the second PED Block It is a staged decoding feature map The input feature map corresponding to the third PED Block It is a staged decoding feature map Initial feature extraction is achieved through a single convolutional layer and the GELU activation function. Then, high-frequency convolution branches and low-frequency convolution branches are used to obtain the high-frequency spatial-domain component (HFSC) and the low-frequency spatial-domain component (LFSC), respectively. At this point, the high-frequency spatial-domain component (HFSC), the low-frequency spatial-domain component (LFSC), and the interpolated original image are obtained. and prior As input to the core module PED Module, the output consists of fused high-frequency features and fused low-frequency features, denoted as Fused High-Frequency Feature and Fused Low-Frequency Feature, respectively. These, along with the feature maps that have undergone convolution and activation for preliminary feature extraction, are input to the SENet-based fusion module unit, and then subjected to projection and fusion with the original features. Residual connections are used to obtain the output of the PED Block. The outputs of the three PED Block modules are denoted as follows: , and .

[0067] Figure 1 The sub-graph in the lower right corner corresponds to the core module, PED Module. The original graph has undergone interpolation transformation. and prior As one of the innovations of this invention, the input of the MEFDM module is as shown in the attached figure. Figure 2 As shown, a total of two outputs will be obtained: a High-Frequency Frequency-domain Component (HFFC) and a Low-Frequency Frequency-domain Component (LFFC).

[0068] Thus, the high and low frequency spatial domain components HFSC and LFSC, and the high and low frequency frequency domain components HFFC and LFFC are used as four inputs to CDAFM, one of the innovations of this invention. Figure 3 As shown, the final output consists of two channels: the fused high-frequency feature FusedHigh-Frequency Feature and the fused low-frequency feature FusedLow-Frequency Feature, denoted as FHFF and FLFF, respectively.

[0069] The weighted sum of the cross-entropy loss function and the Dice loss function is calculated based on the final output feature map. Under the constraint of the above total loss function, the model is trained and optimized until convergence.

[0070] The method of the present invention is further verified through simulation below: Based on the above experimental steps, experiments were conducted on remote sensing image datasets such as Vaihingen, Potsdam, and LoveDA. The mean Intersection over Union (mIoU) value for each class was used as an indicator of the network model's performance in semantic segmentation tasks on a given dataset. Numerical and visualization results are shown below. UNetFormer is a method proposed in a 2022 paper published in the top remote sensing journal ISPRS Journal of Photogrammetry and Remote Sensing, while PPMamba is a method proposed in a 2025 paper published in the renowned remote sensing journal IEEE Geoscience and Remote Sensing Letters.

[0071] Visualization results as follows Figure 4 As shown, quantitative analysis reveals that the mIoU value of this invention on the Vaihingen dataset is significantly higher than that of the UNetFormer and PPMamba methods. Visualization also demonstrates that this invention exhibits higher completeness and semantic consistency in identifying building-related targets within the red box compared to the UNetFormer and PPMamba methods. Therefore, both the quantitative and visual results indicate that the proposed method possesses certain advantages and superiority.

[0072] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0073] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0074] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is defined only by the appended claims.

Claims

1. A semantic segmentation method for remote sensing images based on multi-elliptical frequency domain mask separation and space-frequency fusion mechanism, characterized in that, The method includes: Input a remote sensing image and extract multi-scale feature maps through a backbone network; the multi-scale features include first-stage feature maps, second-stage feature maps, third-stage feature maps and fourth-stage feature maps; The fourth-stage feature map and the original image are input into the first PED Block module, and the output is... ,Will After channel alignment and bilinear interpolation, the feature map is then concatenated with the third-stage feature map and aligned with the channel to obtain the first decoded feature map. The first decoded feature map is first processed by the PGMAM module to obtain the first land cover prior information. The first decoded feature map, the first prior information, and the original image are then input into the second PED Block module, and the output is... ,Will After channel alignment and bilinear interpolation, the feature map is then concatenated with the second-stage feature map and aligned with the channels to obtain the second decoded feature map. The second decoded feature map is first processed by the PGMAM module to obtain the second land cover prior information. The second decoded feature map, the second prior information, and the original image are then input into the third PED Block module, which outputs... ,Will After channel alignment and bilinear interpolation, the feature map is then concatenated with the first-stage feature map and aligned with the channels to obtain the third decoded feature map. The third decoded feature map is processed through the PGMAM module and then projected. It is then concatenated with the residual of the third decoded feature map and input into the segmentation head to output the semantic segmentation result.

2. The method according to claim 1, characterized in that, The processing steps of the PED Block module include: High-frequency spatial components and low-frequency spatial components are extracted by high-frequency and low-frequency convolution branches, respectively. The high-frequency spatial components, low-frequency spatial components, original image, and prior information of ground features are input into the PED Module. The high-frequency and low-frequency components are extracted through a multi-ellipse frequency domain mask separation mechanism. The first PED Block module does not contain prior information of ground features. The high-frequency spatial component, low-frequency spatial component, high-frequency frequency component, and low-frequency frequency component are input into the spatial-frequency dual-domain interactive attention fusion mechanism module to perform cross-attention fusion and gated fusion, and output the fused high-frequency features and low-frequency features. The fused high-frequency and low-frequency features are then residually fused with the original features to obtain the decoding features for the current stage.

3. The method according to claim 1, characterized in that, The multi-elliptic frequency domain mask separation mechanism includes: The interpolated original image and the prior ground features are concatenated along the channel dimension, and a convolution operation is performed to obtain the base features; Global average pooling, convolution, activation, and normalization are performed on the base features to obtain the multi-ellipse mask parameters; Multiple elliptical binary masks are generated based on the parameters, and the union of these masks is used to obtain a joint mask. A two-dimensional fast Fourier transform is performed on the base features, and the joint mask is used to separate the high-frequency and low-frequency components. By performing an inverse transform on the divided high-frequency and low-frequency components, different frequency domain components with high and low frequency divisions are obtained.

4. The method according to claim 1, characterized in that, The spatial-frequency dual-domain interactive attention fusion mechanism includes: Multi-head self-attention calculations are performed on the high-frequency frequency domain component, the low-frequency frequency domain component, the high-frequency spatial domain component, and the low-frequency spatial component respectively; Cross-attention calculation is performed on the frequency domain components and the spatial domain components to obtain the fused high-frequency enhancement features and low-frequency enhancement features. The fused high-frequency features and low-frequency features are concatenated along the channel dimension, and a fusion weight is generated through a gating mechanism for weighted fusion.

5. The method according to claim 1, characterized in that, The prior information of ground features includes category distribution priors and semantic prediction information, which are used to adaptively adjust the spectrum allocation.

6. The method according to claim 1, characterized in that, The residual fusion adopts an element-wise addition method to fuse the fused features with the backbone network features of the corresponding scale.

7. The method according to claim 1, characterized in that, The segmentation head includes at least one convolutional layer and an upsampling layer, used to map the final decoded features to the original image resolution and output a semantic segmentation map.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image semantic segmentation method based on the multi-elliptic frequency domain mask separation and space-frequency fusion mechanism as described in any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image semantic segmentation method based on the multi-elliptic frequency domain mask separation and space-frequency fusion mechanism as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the remote sensing image semantic segmentation method based on the multi-elliptic frequency domain mask separation and space-frequency fusion mechanism according to any one of claims 1 to 7 by executing the executable instructions.

Images