Land coverage classification method based on frequency-space dual-domain interaction and uncertainty weighting

Abstract

The invention is suitable for the technical field of remote sensing information, and provides a land coverage classification method based on frequency-space dual-domain interaction and uncertainty weighting, and the method comprises the steps: obtaining an SAR image and an optical remote sensing image of a target region; acquiring spatial domain feature maps of the SAR image and the optical remote sensing image, and acquiring frequency domain feature maps of the SAR image and the optical remote sensing image based on the spatial domain feature maps; performing cross-modal frequency domain guide fusion based on the frequency domain feature maps of the SAR image and the optical remote sensing image to obtain a fusion feature map; inputting the fused feature map into a decoder model for processing to obtain a pixel-level uncertainty distribution map and a complemented optical remote sensing image; and executing reliability weighted classification based on the uncertainty distribution diagram and the complemented optical remote sensing image to obtain a land coverage classification result of the target area. The land cover classification precision can be improved.

Description

Technical Field

[0001] This application belongs to the field of remote sensing information technology, and in particular relates to a land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting. Background Technology

[0002] With the rapid development of Earth observation technology, optical remote sensing images play an irreplaceable role in land cover classification, urban planning, and disaster monitoring. However, optical sensors are passive imaging systems, highly susceptible to interference from atmospheric conditions such as clouds, fog, and haze. Statistics show that approximately 60%-70% of the Earth's surface is covered by clouds year-round, leading to frequent partial or even large-scale data loss in acquired optical remote sensing images. This modal loss phenomenon severely damages the spectral and textural information of ground features, greatly limiting the all-weather application capabilities of remote sensing data.

[0003] To address the above problems, existing technical solutions mainly fall into the following two categories: 1) Traditional interpolation and filtering methods: These methods utilize pixels surrounding the missing region for interpolation or tensor completion algorithms for restoration. These methods are only suitable for small-scale missing areas. For large areas obscured by clouds, the restored images are often blurry and cannot recover the true texture of the ground features.

[0004] 2) Generative methods based on deep learning: In recent years, deep learning methods, represented by Generative Adversarial Networks (GANs) and Convolutional Neural Networks (CNNs), have been widely used for image inpainting. These methods typically utilize auxiliary modal data (such as synthetic aperture radar (SAR)) unaffected by cloud cover as guidance, and reconstruct missing optical remote sensing images by learning the mapping relationship between "SAR-optics".

[0005] Although existing deep learning methods have improved the completion results to some extent, they still have the following three major technical shortcomings for high-precision remote sensing applications: Defect 1: Learning solely in the spatial domain easily leads to "loss of high-frequency details" and "artifacts." Most existing mainstream networks (such as deep learning-based image transformation models) directly calculate the loss function in the spatial domain. Since remote sensing images contain complex texture and edge information (high-frequency components), simple pixel-level optimization tends to produce smooth, blurry results, making it difficult to reconstruct clear road boundaries or building outlines. Furthermore, GAN models are prone to hallucination, generating seemingly realistic but actually non-existent ground features, severely misleading subsequent interpretations.

[0006] Defect 2: Lack of modeling ability for "cognitive uncertainty," resulting in unreliable fault tolerance mechanisms. Most current completion methods are deterministic, or can only model the random uncertainty of data through simple probability distributions. However, they cannot effectively capture the cognitive uncertainty of the model, i.e., the "ignorance" caused by the model's lack of experience with certain complex cloud and fog scenarios. In extreme occlusion situations, this inability to distinguish the source of uncertainty can lead the model to give incorrect high-confidence predictions, severely misleading subsequent classification networks.

[0007] Defect 3: Cross-modal feature fusion lacks physical adaptability and frequency domain selectivity. Current technologies for fusing SAR and optical data often employ simple stitching or addition, ignoring the fundamental differences in their imaging mechanisms and spectral distributions. This crude fusion method cannot adaptively transform SAR structural information to match the spectral characteristics of optical data, and lacks a dynamic assessment mechanism for the degree of damage to different frequency components, resulting in distorted spectral characteristics and poor physical interpretability in the fused image.

[0008] As mentioned above, existing optical remote sensing image modality missing completion methods have two main defects: First, they rely too much on local convolution operations in pixel space, resulting in the loss of high-frequency texture details and blurred edges in the completed image, and are prone to producing artifacts that do not conform to physical facts; Second, they lack an evaluation mechanism for the reliability of the completion results, and directly use low-confidence data containing artifacts for subsequent classification, which seriously misleads the feature extraction network and reduces the accuracy of ground object identification.

[0009] In summary, the current land cover classification accuracy is low due to the poor completion effect of optical remote sensing images. Summary of the Invention

[0010] This application provides a land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting, which can solve the problem of low accuracy in land cover classification.

[0011] This application provides a land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting, including: Acquire multimodal remote sensing data pairs for the target area; the multimodal remote sensing data pairs include SAR images and optical remote sensing images with missing data; Spatial domain feature maps of SAR images and optical remote sensing images are obtained respectively, and the spatial domain feature maps of SAR images and optical remote sensing images are projected to the frequency domain to obtain the frequency domain feature maps of SAR images and optical remote sensing images respectively. Cross-modal frequency domain guided fusion is performed based on the frequency domain feature maps of SAR images and optical remote sensing images to obtain a fused feature map; The fused feature map is input into the trained decoder model for processing to obtain a pixel-level uncertainty distribution map and a completed optical remote sensing image; the uncertainty distribution map is used to characterize the degree of uncertainty of each pixel in the completed optical remote sensing image; Based on the uncertainty distribution map and the completed optical remote sensing image, a reliability-weighted classification is performed to obtain the land cover classification results for the target area.

[0012] Optionally, cross-modal frequency domain guided fusion is performed based on the frequency domain feature maps of SAR images and optical remote sensing images to obtain a fused feature map, including: The frequency domain feature map of the SAR image is decoupled into the amplitude spectrum and the phase spectrum, and the frequency domain feature map of the optical remote sensing image is also decoupled into the amplitude spectrum and the phase spectrum. Using the amplitude spectrum corresponding to the SAR image as a structural prior, the amplitude spectrum corresponding to the optical remote sensing image is guided and fused to obtain the fused amplitude spectrum. The fused amplitude spectrum is recombined with the phase spectrum corresponding to the optical remote sensing image to obtain the recombined frequency domain feature map. The recombined frequency domain feature map is subjected to inverse fast Fourier transform to obtain the fused feature map, which is a spatial feature map containing complete texture information.

[0013] Optionally, the amplitude spectrum corresponding to the SAR image is used as a structural prior to guide the fusion of the amplitude spectrum corresponding to the optical remote sensing image, resulting in a fused amplitude spectrum, including: The fused amplitude spectrum is calculated using the following formula. : ; in, This represents the amplitude spectrum corresponding to the optical remote sensing image. Represents a dynamic spectrum gating function; ; This represents the Sigmoid activation function. This indicates the use of frequency domain channel attention modules for... To process, This indicates a splicing operation. This represents the amplitude spectrum corresponding to the SAR image. Represents element-wise multiplication. Represents a structural adaptive transformation function; ; express Convolutional layer This represents the ReLU activation function.

[0014] Optionally, the fused amplitude spectrum and the phase spectrum corresponding to the optical remote sensing image are recombined to obtain a reconstructed frequency domain feature map, including: The recombined frequency domain feature map is calculated using the following formula. : ; in, Represents the imaginary unit. This represents the phase spectrum corresponding to the optical remote sensing image.

[0015] Optionally, the loss function used during decoder model training is the evidence regularization loss function, the expression of which is: ; in, This represents the value of the evidence regularization loss function. This represents the expected Bayesian risk loss. Indicates the annealing coefficient. , Indicates the current iteration number. This indicates the preset number of annealing steps. This represents the Dirichlet KL divergence regularization term; ; ; This represents the total number of training samples. Indicates the first The true label corresponding to each training sample The decoder model represents the first... The output value of each training sample. Indicates the first The uncertainty intensity parameter corresponds to each training sample, and the uncertainty intensity parameter is a trainable parameter of the decoder model during the training process. This indicates the total number of categories. Represents the gamma function. Represents the double gamma function. Indicates the total amount of evidence. In the Dirichlet distribution, the th Evidence parameters corresponding to each category.

[0016] Optionally, reliability-weighted classification is performed based on the uncertainty distribution map and the completed optical remote sensing image to obtain the land cover classification results for the target area, including: The completed optical remote sensing image is input into a feature extractor for feature extraction to obtain a classification feature map. The reliability weight map is obtained by normalizing the uncertainty distribution map and then inverting it. The classification feature map is multiplied element-wise using the reliability weight map to obtain a weighted feature map. The weighted feature map is input into the Softmax classification head for processing to obtain the land cover classification result of the target area.

[0017] Optionally, the uncertainty distribution map can be normalized and then inverted to obtain a reliability weight map, including: The reliability weight map is calculated using the following formula. : ; in, This represents the Sigmoid activation function. This represents a graph showing the distribution of uncertainty.

[0018] The above-mentioned solution in this application has the following beneficial effects: In the embodiments of this application, spatial domain feature maps of SAR images and optical remote sensing images of the target area are extracted and projected into the frequency domain to obtain frequency domain feature maps. Then, cross-modal frequency domain guided fusion is performed based on the frequency domain feature maps of SAR images and optical remote sensing images. The fused feature map is then input into the trained decoder model for processing to obtain pixel-level uncertainty distribution maps and completed optical remote sensing images. Finally, reliability-weighted classification is performed based on the uncertainty distribution maps and completed optical remote sensing images to obtain the land cover classification results of the target area. Among them, cross-modal frequency domain guided fusion based on frequency domain feature maps can adaptively utilize the amplitude spectrum information of SAR images to accurately repair the damaged frequency components of optical remote sensing images, effectively restoring the geometric structure and high-frequency edge information of land features obscured by clouds, avoiding image blurring and spectral distortion. At the same time, since the uncertainty distribution map can characterize the uncertainty of each pixel in the completed optical remote sensing image, when performing reliability-weighted classification based on the uncertainty distribution map and the completed optical remote sensing image, it can accurately identify unreliable completion areas and automatically ignore the feature interference of these areas, thereby effectively preventing the propagation of misclassification caused by completion artifacts and model overconfidence, and achieving the effect of effectively improving the accuracy of land cover classification.

[0019] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting, provided as an embodiment of this application; Figure 2 A block diagram of the overall network architecture provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the frequency-space dual-domain interaction principle provided in an embodiment of this application. Detailed Implementation

[0022] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0023] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0024] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0026] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0027] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0028] To address the issue of low accuracy in current land cover classification, this application provides a land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting. This method extracts spatial domain feature maps from SAR and optical remote sensing images of the target area and projects them into the frequency domain to obtain frequency domain feature maps. Then, cross-modal frequency domain guided fusion is performed based on the frequency domain feature maps of the SAR and optical remote sensing images. The fused feature map is then input into a trained decoder model for processing to obtain a pixel-level uncertainty distribution map and a completed optical remote sensing image. Finally, reliability weighted classification is performed based on the uncertainty distribution map and the completed optical remote sensing image to obtain the land cover classification result of the target area. Among them, cross-modal frequency domain guided fusion based on frequency domain feature maps can adaptively utilize the amplitude spectrum information of SAR images to accurately repair the damaged frequency components of optical remote sensing images, effectively restoring the geometric structure and high-frequency edge information of land features obscured by clouds, avoiding image blurring and spectral distortion. At the same time, since the uncertainty distribution map can characterize the uncertainty of each pixel in the completed optical remote sensing image, when performing reliability-weighted classification based on the uncertainty distribution map and the completed optical remote sensing image, it can accurately identify unreliable completion areas and automatically ignore the feature interference of these areas, thereby effectively preventing the propagation of misclassification caused by completion artifacts and model overconfidence, and achieving the effect of effectively improving the accuracy of land cover classification.

[0029] The land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting provided in this application will be illustrated below with specific embodiments.

[0030] like Figure 1 As shown in the embodiments of this application, the land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting includes the following steps: Step 11: Acquire a multimodal remote sensing data pair for the target area, which includes SAR images and optical remote sensing images with missing data.

[0031] The target area mentioned above is an area requiring land cover classification. In some embodiments of this application, the SAR image can be acquired by a platform equipped with a synthetic aperture radar sensor, for example, by satellite; the optical remote sensing image can be acquired by equipment such as a multispectral scanner or hyperspectral imager. This optical remote sensing image is obscured by clouds and fog, therefore it contains missing data.

[0032] Understandably, after acquiring SAR images of the target area and optical remote sensing images with missing data, routine preprocessing operations are required to prepare for subsequent steps.

[0033] Step 12: Obtain the spatial domain feature maps of the SAR image and the optical remote sensing image respectively, and project the spatial domain feature maps of the SAR image and the optical remote sensing image to the frequency domain respectively to obtain the frequency domain feature maps of the SAR image and the optical remote sensing image.

[0034] In some embodiments of this application, SAR images can be extracted using a deep neural network including a dual-stream encoder. and optical remote sensing images The spatial domain feature map is obtained, and then it is projected onto the frequency domain to obtain the frequency domain feature map of the SAR image and the frequency domain feature map of the optical remote sensing image.

[0035] Specifically, the implementation of step 12 above includes the following steps 12.1 to 12.2: Step 12.1, transfer the SAR image (like Figure 2 SAR images and optical remote sensing images (like Figure 2 The multi-cloud map in the image is input into two convolutional neural network encoders with the same structure (e.g., the multi-cloud map in the image). Figure 2 The encoders A and B in the image are processed to obtain the spatial domain feature map of the SAR image. Spatial domain feature map of optical remote sensing images .in, , , , , These represent the number of channels, height, and width, respectively.

[0036] Step 12.2, analyze the spatial domain feature maps respectively. and spatial domain feature map Perform a two-dimensional fast Fourier transform on each channel (e.g. Figure 2The Fast Fourier Transform (FFT) maps the spatial domain feature map to the frequency domain, resulting in the frequency domain feature maps of the SAR image and the optical remote sensing image.

[0037] Step 13: Perform cross-modal frequency domain guided fusion based on the frequency domain feature map of the SAR image and the frequency domain feature map of the optical remote sensing image to obtain the fused feature map.

[0038] In some embodiments of this application, the frequency domain feature map can be decoupled into amplitude spectrum and phase spectrum first, and then the amplitude spectrum information of the SAR image can be used to accurately repair the damaged frequency components of the optical remote sensing image.

[0039] Specifically, the implementation of step 13 above includes the following steps 13.1 to 13.4: Step 13.1: Decouple the frequency domain feature map of the SAR image into amplitude spectrum and phase spectrum, and decouple the frequency domain feature map of the optical remote sensing image into amplitude spectrum and phase spectrum.

[0040] In some embodiments of this application, amplitude spectrum is used to characterize the texture and geometric structure information of ground features, and phase spectrum is used to characterize the location and contour information of ground features. The calculation formula is as follows: ; ; In the above formula, Indicates amplitude spectrum, Represents the phase spectrum. and These are the real and imaginary parts of the complex feature (i.e., the frequency domain feature map), respectively.

[0041] It should be noted that, after the above formula is used to calculate the frequency domain feature map of the SAR image, the frequency domain feature map of the SAR image can be decoupled into the amplitude spectrum and the phase spectrum (e.g., Figure 2 The amplitude and phase spectra corresponding to the SAR image branches. After the above formula is used to calculate the frequency domain feature map of the optical remote sensing image, the frequency domain feature map of the optical remote sensing image can be decoupled into amplitude and phase spectra (e.g., Figure 2 (Amplitude and phase spectra corresponding to the branches in the multi-cloud diagram).

[0042] Step 13.2: Using the amplitude spectrum corresponding to the SAR image as a structural prior, guide the fusion of the amplitude spectrum corresponding to the optical remote sensing image to obtain the fused amplitude spectrum.

[0043] In some embodiments of this application, considering the fundamental differences in imaging mechanisms between SAR images and optical remote sensing images, their amplitude spectrum distributions cannot be directly equivalent. Therefore, this embodiment proposes a nonlinear fusion strategy based on a structure-adaptive transformation-dynamic gated injection mechanism. This strategy aims to learn a nonlinear mapping from SAR frequency domain features to optical frequency domain features, and inject structural information in residual form only in the frequency region where the amplitude spectrum of the optical image is damaged through a dynamic spectral gating mechanism. In some embodiments, a cross-modal frequency domain residual gated fusion module can be designed to perform guided fusion. This module specifically calculates the fused amplitude spectrum using the following formula. : ; in, This represents the amplitude spectrum corresponding to the optical remote sensing image. The dimension is , Represents element-wise multiplication. Represents the dynamic spectral gating function, in the above formula It is a dynamic spectral gating term.

[0044] ; This represents the Sigmoid activation function. This indicates the use of frequency domain channel attention modules for... To process, This indicates a splicing operation. This represents the amplitude spectrum corresponding to the SAR image. The dimension is The Sigmoid activation function, also known as the logistic activation function, is used as a non-linear activation function in neural networks, mapping input values ​​to the (0,1) interval.

[0045] It is a complex non-linear attention mechanism used to output a value within a range of The gating tensor between the two modes determines how much SAR information needs to be injected into each frequency point and each channel. It first concatenates the amplitude spectra of the two modes along the channel dimension, and then calculates the gating weights through a frequency channel attention block (FCAB). The FCAB first performs global average pooling (GAP) on the input frequency domain features to obtain a global spectral descriptor, and then captures the dependencies between frequency channels through two fully connected layers (FC). This mathematically simulates an adaptive evaluation of the importance of different frequency components.

[0046] Represents the structural adaptive transformation function. This is a structural adaptation transformation term.

[0047] ; In the above formula, express Convolutional layer This represents the ReLU activation function. Wherein, The two layers Convolutional layers are used to perform channel transformation in the frequency domain. As a learnable micronetwork, its function is to map the amplitude spectrum features of the SAR image to the feature space of the optical image (i.e., ... Figure 2 The amplitude spectrum corresponding to the SAR image guides the amplitude spectrum corresponding to the multi-cloud image, eliminating style differences between modes and making it more suitable for the optical spectral distribution.

[0048] Step 13.3: Recombine the fused amplitude spectrum with the phase spectrum corresponding to the optical remote sensing image to obtain the recombined frequency domain feature map.

[0049] It is worth mentioning that recombinating the fused amplitude spectrum with the phase spectrum corresponding to the optical remote sensing image ensures that the generated image possesses both the clear structure provided by SAR and avoids spatial displacement. Specifically, the recombined frequency domain feature map can be calculated using the following formula. : ; in, Represents the imaginary unit. This represents the phase spectrum corresponding to the optical remote sensing image.

[0050] Step 13.4: Perform inverse fast Fourier transform on the recombined frequency domain feature map to obtain the fused feature map, which is a spatial feature map containing complete texture information.

[0051] That is, in some embodiments of this application, an inverse fast Fourier transform (e.g., ...) can be performed on the reconstructed frequency domain feature map. Figure 2 The inverse transform in the frequency domain maps the recombined frequency domain feature map back to the spatial domain, resulting in a spatial feature map containing complete texture information.

[0052] Step 14: Input the fused feature map into the trained decoder model for processing to obtain a pixel-level uncertainty distribution map and a completed optical remote sensing image. The uncertainty distribution map is used to characterize the degree of uncertainty of each pixel in the completed optical remote sensing image.

[0053] In some embodiments of this application, the decoder model described above can be a UNet decoder (UNet is a convolutional neural network architecture for image segmentation tasks). In some embodiments, the decoder model can be specifically built based on Evidence Theory, where the uncertainty branch of the decoder model no longer outputs a single variance value, but instead outputs a non-negative evidence vector. (For regression completion tasks, it can be viewed as discretizing pixel values ​​into...) The intervals, or the continuous version, are described here in a more general discretized form to illustrate the complexity. In actual regression, they are usually simplified to a single evidence value. ).

[0054] The network output is ensured to be positive by using the Softplus activation function (Softplus is a commonly used activation function in artificial neural networks), denoted as evidence. According to evidence theory, this corresponds to the parameters of the Dirichlet distribution. , The total amount of evidence is At this point, the uncertainty of prediction is modeled as epistemic uncertainty, defined as the reciprocal of the total amount of evidence. : , This indicates the total number of categories.

[0055] The loss function used in the training process of the decoder model described above is the evidence regularization loss function. This loss function aims to minimize the Bayesian risk of prediction errors while penalizing the model for providing erroneous high-confidence evidence through KL divergence (a measure used to quantify the difference between two probability distributions). The expression for this evidence regularization loss function is: ; in, This represents the value of the evidence regularization loss function. This represents the expected Bayesian risk loss. The calculation is based on the expected prediction error under the Dirichlet distribution. Indicates the annealing coefficient. , Indicates the current iteration number. This indicates the preset number of annealing steps, in the initial stage of training. Smaller learning focuses on stabilizing the main task, and training progresses in the later stages. Increase to 1 to strengthen KL regularization constraints. This represents the Dirichlet KL divergence regularization term. This is used to constrain the model's output distribution from deviating too far from a uniform distribution, preventing the model from being blindly confident when there is insufficient evidence.

[0056] ; ; This represents the total number of training samples. Indicates the first The true label corresponding to each training sample The decoder model represents the first... The output value of each training sample. Indicates the first The uncertainty intensity parameter corresponds to each training sample, and the uncertainty intensity parameter is a trainable parameter of the decoder model during the training process. This indicates the total number of categories. Represents the gamma function. Represents the double gamma function. Indicates the total amount of evidence. This represents the th element in the Dirichlet distribution. Evidence parameters corresponding to each category.

[0057] For the prediction error term, the total amount of evidence The larger, The smaller the value, the more confident the model is in its predictions.

[0058] like Figure 2 As shown, the fused feature map output from the inverter is input into the decoder model (e.g., Figure 2 The decoder in the model processes the data, and the decoder model can then output a pixel-level uncertainty distribution map (such as...). Figure 2 Uncertainty map in the image) and the completed optical remote sensing image (e.g. Figure 2 (The complete image in the image).

[0059] Step 15: Perform reliability-weighted classification based on the uncertainty distribution map and the completed optical remote sensing image to obtain the land cover classification results for the target area.

[0060] In some embodiments of this application, step 15 is specifically implemented as follows: steps 15.1 to 15.4: Step 15.1: Input the completed optical remote sensing image into the feature extractor for feature extraction to obtain a classification feature map.

[0061] In some embodiments of this application, such as Figure 2 As shown, conventional feature extractors can be used to process the completed optical remote sensing image (such as...). Figure 2Feature extraction is performed on the complete image in the image to obtain a classification feature map (such as...). Figure 2 (Feature maps in the graph). As a preferred example, the feature extractor can be a graph neural network. Step 15.2: Normalize the uncertainty distribution map and then invert it to obtain the reliability weight map.

[0062] Specifically, the reliability weight map can be calculated using the following formula. : ; in, This represents the Sigmoid activation function. Uncertainty distribution diagram, reliability weight diagram The closer the value is to 1, the more reliable the region is; the closer the value is to 0, the more reliable the region is as an incomplete artifact.

[0063] Step 15.3: Element-wise multiplication of the classification feature map is performed using the reliability weight map to obtain the weighted feature map.

[0064] Specifically, the weighted feature map can be calculated using the following formula. : ; In the above formula, This represents the classification feature map. By using the reliability weight map to perform element-wise multiplication of the classification feature map, we can achieve the following: for the implied regions that the network is "unsure about," their feature responses are automatically set to zero to prevent them from misleading the classifier.

[0065] Step 15.4: Input the weighted feature map into the Softmax classification head for processing to obtain the land cover classification result of the target area.

[0066] In some embodiments of this application, such as Figure 2 As shown, classification feature maps (such as...) can be used to classify feature maps. Figure 2 (Feature map in the image) and uncertainty map are compared. The result of the calculation is input into the Softmax classification header (e.g.) Figure 2 The classification header in the data is processed to obtain the land cover classification results for the target area (e.g., ...). Figure 2 (The classification results in the model). Softmax is a mathematical function commonly used for multi-class classification problems.

[0067] The land cover classification results are used to indicate the type of the target area, such as cultivated land, forest land, grassland, wetland, water body, etc. It is understood that the target area may be of one type or may include multiple types, for example, part of the area may be forest land and another part may be water body.

[0068] It is understood that the learnable networks, learnable models, and other structures involving learnable parameters in the embodiments of this application are all trained before actual application (specifically, conventional deep learning training methods, such as stochastic gradient descent), so that the corresponding learnable parameters are at their optimal values, ensuring the accuracy of land cover classification.

[0069] Based on the above explanation, as Figure 3 As shown, the land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting provided in this application classifies land cover by analyzing the spatial domain feature maps respectively. (like Figure 3 (Optical feature map of clouds) and spatial domain feature map (like Figure 3 The SAR feature map in the image is subjected to Fast Fourier Transform (FFT) to obtain the frequency domain feature map of the SAR image and the frequency domain feature map of the optical remote sensing image. Then, these two are decoupled to obtain the amplitude spectrum corresponding to the cloud optical feature map (e.g., ...). Figure 3 (damaged amplitude map) and phase spectrum (such as) Figure 3 The original phase spectrum), and the amplitude spectrum corresponding to the SAR feature map (e.g. Figure 3 The amplitude map and phase spectrum are then fused through a cross-modal frequency domain residual gated fusion module (i.e., Figure 3 The CFRG module in the image is used to splice the data, apply dynamic spectral gating, and perform structural adaptive transformation to obtain the fused amplitude spectrum (e.g., ...). Figure 3 The amplitude spectrum after fusion is then reconstructed from the original phase spectrum and subjected to inverse Fourier transform to obtain the fused feature map (e.g., the amplitude spectrum after fusion is reconstructed from the original phase spectrum). Figure 3 The fused feature map is processed using a decoder model to obtain a pixel-level uncertainty distribution map and a completed optical remote sensing image. Finally, a reliability-weighted classification is performed based on the uncertainty distribution map and the completed optical remote sensing image to obtain the land cover classification result of the target area.

[0070] As mentioned earlier, cross-modal frequency domain guided fusion based on frequency domain feature maps can adaptively utilize the amplitude spectrum information of SAR images to accurately repair the damaged frequency components of optical remote sensing images, effectively restoring the geometric structure and high-frequency edge information of land features obscured by clouds, avoiding image blurring and spectral distortion. At the same time, since the uncertainty distribution map can characterize the degree of uncertainty of each pixel in the completed optical remote sensing image, when performing reliability-weighted classification based on the uncertainty distribution map and the completed optical remote sensing image, it can accurately identify unreliable completion areas and automatically ignore the feature interference of these areas, thereby effectively preventing the propagation of misclassification caused by completion artifacts and model overconfidence, and achieving the effect of effectively improving the accuracy of land cover classification.

[0071] In summary, the land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting provided in this application has the following advantages: 1) Overcoming the limitations of single spatial domain restoration, significantly improving image texture fidelity and physical consistency: This application introduces a frequency domain processing mechanism, utilizing the decoupling characteristics of amplitude and phase spectra to address the shortcomings of traditional convolutional networks in long-distance dependency modeling. By designing a cross-modal frequency domain residual gating mechanism, it can adaptively utilize the amplitude spectrum information of SAR data to accurately restore the frequency components damaged in optical images, more effectively recovering the geometric structure and high-frequency edge information of cloud-obscured ground features, and avoiding image blurring and spectral distortion.

[0072] 2) A closed-loop fault-tolerant mechanism based on evidence-based reasoning, "completeness-classification," was established to improve the system's robustness: This application introduces evidence-based reasoning theory based on Dirichlet priors, which can effectively quantify the cognitive uncertainty of the model. The risk-avoidance strategy constructed accordingly enables the subsequent classification network to accurately identify unreliable completion regions (high cognitive uncertainty) caused by a lack of evidence and automatically ignore the feature interference in these regions. This effectively prevents the propagation of misclassification caused by completion artifacts and model overconfidence, especially in extreme cloud and fog occlusion scenarios.

[0073] 3) Achieved physical-level deep fusion of cross-modal information: Unlike simple channel stitching, this application is based on the remote sensing imaging mechanism (SAR reflects structure, optics reflects texture) and performs feature interaction at the frequency domain level, making modal fusion more physically interpretable. The generated images are not only visually realistic, but also have stronger spectral consistency.

[0074] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A land cover classification method based on frequency-space dual-domain interaction and uncertainty weighting, characterized in that, include: Acquire multimodal remote sensing data pairs for the target area; the multimodal remote sensing data pairs include SAR images and optical remote sensing images with missing data. Spatial domain feature maps of the SAR image and the optical remote sensing image are obtained respectively, and the spatial domain feature maps of the SAR image and the optical remote sensing image are projected to the frequency domain respectively to obtain the frequency domain feature map of the SAR image and the frequency domain feature map of the optical remote sensing image. Cross-modal frequency domain guided fusion is performed based on the frequency domain feature map of the SAR image and the frequency domain feature map of the optical remote sensing image to obtain a fused feature map; The fused feature map is input into the trained decoder model for processing to obtain a pixel-level uncertainty distribution map and a completed optical remote sensing image; the uncertainty distribution map is used to characterize the degree of uncertainty of each pixel in the completed optical remote sensing image; Based on the uncertainty distribution map and the completed optical remote sensing image, a reliability-weighted classification is performed to obtain the land cover classification result of the target area.

2. The land cover classification method according to claim 1, characterized in that, The frequency domain feature map based on the SAR image and the frequency domain feature map based on the optical remote sensing image are subjected to cross-modal frequency domain guided fusion to obtain a fused feature map, including: The frequency domain feature map of the SAR image is decoupled into amplitude spectrum and phase spectrum, and the frequency domain feature map of the optical remote sensing image is also decoupled into amplitude spectrum and phase spectrum; Using the amplitude spectrum corresponding to the SAR image as a structural prior, the amplitude spectrum corresponding to the optical remote sensing image is guided to be fused to obtain the fused amplitude spectrum. The fused amplitude spectrum is recombined with the phase spectrum corresponding to the optical remote sensing image to obtain the recombined frequency domain feature map; The recombined frequency domain feature map is subjected to inverse fast Fourier transform to obtain a fused feature map, which is a spatial feature map containing complete texture information.

3. The land cover classification method according to claim 2, characterized in that, The step of using the amplitude spectrum corresponding to the SAR image as a structural prior to guide the fusion of the amplitude spectrum corresponding to the optical remote sensing image, to obtain the fused amplitude spectrum, includes: The fused amplitude spectrum is calculated using the following formula. : ； in, This represents the amplitude spectrum corresponding to the optical remote sensing image. Indicates dynamic Spectral gating function; ； This represents the Sigmoid activation function. This indicates the use of frequency domain channel attention modules for... To process, This indicates a splicing operation. This represents the amplitude spectrum corresponding to the SAR image. Represents element-wise multiplication. Represents a structural adaptive transformation function; ； express Convolutional layer This represents the ReLU activation function.

4. The land cover classification method according to claim 3, characterized in that, The step of recombining the fused amplitude spectrum with the phase spectrum corresponding to the optical remote sensing image to obtain the reconstructed frequency domain feature map includes: The recombined frequency domain feature map is calculated using the following formula. : ； in, Represents the imaginary unit. This represents the phase spectrum corresponding to the optical remote sensing image.

5. The land cover classification method according to claim 1, characterized in that, The loss function used in the training process of the decoder model is the evidence regularization loss function, and the expression of the evidence regularization loss function is: ； in, This represents the value of the evidence regularization loss function. This represents the expected Bayesian risk loss. Indicates the annealing coefficient. , Indicates the current iteration number. This indicates the preset number of annealing steps. This represents the Dirichlet KL divergence regularization term; ； ； This represents the total number of training samples. Indicates the first The true label corresponding to each training sample The decoder model represents the first... The output value of each training sample. Indicates the first The uncertainty intensity parameter corresponds to each training sample, and the uncertainty intensity parameter is a trainable parameter of the decoder model during the training process. This indicates the total number of categories. Represents the gamma function. Represents the double gamma function. Indicates the total amount of evidence. In the Dirichlet distribution, the th Evidence parameters corresponding to each category.

6. The land cover classification method according to claim 1, characterized in that, The reliability-weighted classification based on the uncertainty distribution map and the completed optical remote sensing image is used to obtain the land cover classification result of the target area, including: The completed optical remote sensing image is input into a feature extractor for feature extraction to obtain a classification feature map. The reliability weight map is obtained by normalizing the uncertainty distribution map and then inverting it. The classification feature map is multiplied element-wise using the reliability weight map to obtain a weighted feature map. The weighted feature map is input into the Softmax classification head for processing to obtain the land cover classification result of the target area.

7. The land cover classification method according to claim 6, characterized in that, The process of normalizing and inverting the uncertainty distribution map to obtain the reliability weight map includes: The reliability weight map is calculated using the following formula. : ； in, This represents the Sigmoid activation function. This represents a graph showing the distribution of uncertainty.

Images