A method and system for degrading remote sensing image restoration

Abstract

The application discloses a kind of degradation remote sensing image restoration method and system, it is related to image processing technical field, by the input of the degradation remote sensing image obtained to pre-training restoration model, shallow feature of degradation remote sensing image is extracted by the input embedding layer of restoration model;In the multi-level encoder of restoration model, shallow feature is modulated in frequency domain and spatial modulation by double-gated attention mechanism, and deep feature of degradation remote sensing image is extracted gradually;In the multi-level decoder of restoration model, parallel expert processing and adaptive weighted fusion are carried out to deep feature by heterogeneous expert collaborative module, and decoding feature is obtained by gradually reconstructing;Heterogeneous expert collaborative module includes first expert branch for capturing global context dependence and second expert branch for extracting directional edge feature;According to shallow feature and decoding feature, generate restoration result image.The application can realize high-fidelity restoration of composite degraded image in real remote sensing observation scene.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for restoring degraded remote sensing images. Background Technology

[0002] In real-world remote sensing observations, image quality is highly susceptible to interference from various external factors and the inherent characteristics of the sensor itself. These interferences typically manifest as a combination of factors, including atmospheric degradation (such as haze and turbulence), changes in lighting conditions (such as low light), and sensor noise. This complex degradation phenomenon leads to reduced image contrast, blurred details, and a decreased signal-to-noise ratio, severely weakening the information carrying capacity of remote sensing images and consequently affecting the accuracy of downstream tasks such as interpretation, classification, and target detection.

[0003] However, there are internal conflicting objectives in remote sensing restoration tasks. For example, denoising tasks typically require suppressing high-frequency components to eliminate graininess, while deblurring tasks require enhancing high-frequency components to restore edge details. Existing remote sensing image processing schemes usually only focus on processing spatial domain features, making it difficult to effectively decouple complex mixed degradation factors. They are prone to falling into "trivial solutions," resulting in artifacts such as blurred edges or residual noise in the restored image. Summary of the Invention

[0004] The technical problem to be solved by this invention is how to effectively decouple complex mixed degradation factors in the image restoration process, and to provide a method and system for restoring degraded remote sensing images, which can achieve high-fidelity restoration of composite degraded images caused by multiple degradation factors in real remote sensing observation scenarios.

[0005] To address the above technical problems, embodiments of the present invention provide a method for restoring degraded remote sensing images, comprising: The acquired degraded remote sensing image is input into a pre-trained restoration model, and shallow features of the degraded remote sensing image are extracted through the input embedding layer of the restoration model. In the multi-level encoder of the restoration model, the shallow features are frequency-domain modulated and spatial-domain modulated through a dual-gated attention mechanism to extract the deep features of the degraded remote sensing image step by step. In the multi-level decoder of the restoration model, the deep features are processed in parallel by a heterogeneous expert collaboration module and adaptively weighted and fused to reconstruct the decoded features step by step. The heterogeneous expert collaboration module includes a first expert branch for capturing global context dependencies and a second expert branch for extracting directional edge features. Based on the shallow features and the decoded features, a restored image is generated.

[0006] As an improvement to the above scheme, the method for the multi-level encoder of the restoration model to pass skip connection features to the corresponding level multi-level decoder includes: Extract the spatial gradient of the output features of the current encoder, and generate a spatial attention mask based on the spatial gradient; By using discrete wavelet transform, the output features of the current level encoder are mapped to the frequency domain of each sub-band, and the channel importance weights of each sub-band are learned through global pooling and fully connected layers to generate frequency domain channel attention masks. The spatial attention mask and the frequency domain channel attention mask are used to perform dual modulation on the output features of the current level encoder to obtain filtered skip connection features, which are then input into the corresponding level decoder.

[0007] As an improvement to the above scheme, the step of inputting the acquired degraded remote sensing image into a pre-trained restoration model, and extracting shallow features of the degraded remote sensing image through the input embedding layer of the restoration model, includes: Acquire degraded remote sensing images and normalize the pixel values ​​of the degraded remote sensing images; If the size of the degraded remote sensing image is larger than a preset size threshold, the degraded remote sensing image is divided into blocks using a sliding window strategy. The processed degraded remote sensing image is input into a pre-trained restoration model, and shallow features of the degraded remote sensing image are extracted through the input embedding layer of the restoration model.

[0008] As an improvement to the above scheme, the shallow features in the multi-level encoder of the restoration model are frequency-domain modulated and spatial-domain modulated using a dual-gated attention mechanism to extract deep features of the degraded remote sensing image step by step, including: The shallow features are input into the multi-level encoder of the restoration model; For each level of the encoder, the frequency-domain gating unit with dual-gated attention mechanism decomposes the input features of the current level into sub-bands of different frequencies, and adaptively learns the weights of each sub-band to obtain the intermediate features after frequency-domain modulation. By transposing the channel of the dual-gated attention mechanism from the attention module, the intermediate features are aggregated globally in context to obtain the attention output features; The spatial gating unit with dual-gated attention mechanism modulates the attention output features pixel-by-pixel confidence in the spatial domain to obtain the output features of the current level. The output features of the current level are then downsampled and used as the input features of the next level encoder. The output features of the last stage of the multi-level encoder are used as the deep features of the degraded remote sensing image.

[0009] As an improvement to the above scheme, the deep features are processed in parallel by a heterogeneous expert collaboration module and then adaptively weighted and fused to reconstruct the decoded features step by step in the multi-level decoder of the restoration model, including: The deep features are input into the multi-level decoder of the restoration model; For each level of the decoder, the global contextual dependencies of the input features of the current level are captured through the first expert branch of the heterogeneous expert collaboration module, and the global semantic features are output; the first expert branch adopts a dual-gated attention mechanism. The directional edge features of the input features at the current level are extracted through the second expert branch of the heterogeneous expert collaboration module; the second expert branch adopts a stacked structure of multi-scale depthwise separable convolution and a dynamic edge gating mechanism. The adaptive fusion unit of the heterogeneous expert collaboration module dynamically calculates the fusion weights of the first expert branch and the second expert branch based on the statistical characteristics of the input features of the current level. Based on the fusion weights, the global semantic features and the directional edge features are weighted and fused to obtain weighted fused features; The weighted fusion features are residually concatenated with the input features of the current level to obtain the output features of the current level; The output features of the current level are upsampled and then fused with the skip connection features passed from the corresponding level encoder to serve as the input features of the next level decoder. The output features of the last stage of the multi-level decoder are used as the decoding features of the degraded remote sensing image.

[0010] As an improvement to the above scheme, the training method of the restoration model includes: Based on the collected multi-source clear remote sensing images and predefined degradation factors, corresponding degraded images are synthesized to form a training sample set; The degraded image is preprocessed, and the preprocessed degraded image is input into a pre-constructed restoration model to generate a predicted restored image; Calculate the loss function between the predicted restored image and the multi-source clear remote sensing image; Based on the loss function, the parameters of the restored model are updated using the backpropagation algorithm; The iteration continues until the preset stopping condition is met, at which point the trained restored model is obtained.

[0011] As an improvement to the above scheme, the step of synthesizing corresponding degraded images based on acquired multi-source clear remote sensing images and predefined degradation factors to form a training sample set includes: Acquire multi-source clear remote sensing images; the multi-source clear remote sensing images include clear remote sensing images of different land cover types; A physics-driven degradation model is used to apply at least one degradation factor to the multi-source clear remote sensing image to generate a corresponding degraded image; the degradation factors include low light, blur, haze, and noise; when multiple degradation factors are applied, the application order is low light, blur, haze, and noise. A training sample set is obtained based on the multi-source clear remote sensing images and the degraded images.

[0012] As an improvement to the above scheme, the step of employing a physics-driven degradation model to apply at least one degradation factor to the multi-source clear remote sensing image to generate a corresponding degraded image includes: When the applied degradation factor is low light, a power transformation is performed on each pixel value of the image to be degraded to obtain a first degraded image; the power exponent in the power transformation is randomly sampled within a preset range of darkness coefficient values. When the applied degradation factor is blurring, an anisotropic Gaussian kernel is used to perform two-dimensional filtering on the image to be degraded, resulting in a second degraded image. When the applied degradation factor is haze, haze synthesis is performed on the image to be degraded based on the atmospheric scattering model to obtain a third degraded image; When the applied degradation factor is noise, an additive Gaussian noise model is used to add noise to the image to be degraded, resulting in a fourth degraded image.

[0013] As an improvement to the above scheme, the calculation of the loss function between the predicted restored image and the multi-source clear remote sensing image includes: Calculate the pixel-level difference in color space between the predicted restored image and the multi-source clear remote sensing image; Calculate the spectral difference between the predicted restored image and the multi-source sharp remote sensing image in the Fourier domain; Calculate the semantic difference between the predicted restored image and the multi-source sharp remote sensing image in the high-level semantic feature space; Based on preset weight parameters, the pixel-level differences, the spectral differences, and the semantic differences are weighted and fused to obtain a loss function.

[0014] This invention also provides a degraded remote sensing image restoration system, comprising: The shallow feature extraction module is used to input the acquired degraded remote sensing image into a pre-trained restoration model, and extract the shallow features of the degraded remote sensing image through the input embedding layer of the restoration model. The encoding module is used to perform frequency domain modulation and spatial domain modulation on the shallow features in the multi-level encoder of the restoration model through a dual-gated attention mechanism, and extract the deep features of the degraded remote sensing image step by step. The decoding module is used to perform parallel expert processing and adaptive weighted fusion on the deep features in the multi-level decoder of the restoration model through a heterogeneous expert collaboration module, and reconstruct the decoded features step by step; the heterogeneous expert collaboration module includes a first expert branch for capturing global context dependencies and a second expert branch for extracting directional edge features; The restoration result generation module is used to generate a restoration result image based on the shallow features and the decoded features.

[0015] Compared with existing technologies, the present invention discloses a method and system for restoring degraded remote sensing images. This method involves inputting the acquired degraded remote sensing image into a pre-trained restoration model. Shallow features of the degraded remote sensing image are extracted through the input embedding layer of the restoration model. In the multi-level encoder of the restoration model, a dual-gated attention mechanism is used to perform frequency-domain modulation and spatial-domain modulation on the shallow features, progressively extracting deep features of the degraded remote sensing image. In the multi-level decoder of the restoration model, a heterogeneous expert collaboration module performs parallel expert processing and adaptive weighted fusion on the deep features, progressively reconstructing decoded features. The heterogeneous expert collaboration module includes a first expert branch for capturing global context dependencies and a second expert branch for extracting directional edge features. Based on the shallow features and the decoded features, a restored image is generated. Using this invention, high-fidelity restoration of composite degraded images caused by multiple degradation factors in real remote sensing observation scenarios can be achieved. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart of a method for restoring degraded remote sensing images provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the steps of a method for a multi-level encoder to transmit skip connection features to a corresponding level multi-level decoder, as provided in an embodiment of the present invention. Figure 3 This is a schematic flowchart of a training method for a restoration model provided in an embodiment of the present invention; Figure 4 This is a visualization result of a comparative test provided by an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a degraded remote sensing image restoration system provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In the description and claims, it should be understood that the terms "first," "second," etc., used in the description and claims are only for the purpose of distinguishing the description of the same technical features, and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated, nor necessarily the order of description or chronological order. The terms are interchangeable where appropriate. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature.

[0019] This invention provides a method for restoring degraded remote sensing images. Please refer to [link to relevant documentation]. Figure 1 In this embodiment, the degraded remote sensing image restoration method is specifically executed through steps S1 to S4: S1. Input the acquired degraded remote sensing image into the pre-trained restoration model, and extract the shallow features of the degraded remote sensing image through the input embedding layer of the restoration model; S2. In the multi-level encoder of the restoration model, the shallow features are frequency-domain modulated and spatial-domain modulated through a dual-gated attention mechanism to extract the deep features of the degraded remote sensing image step by step. S3. In the multi-level decoder of the restoration model, the deep features are processed in parallel by a heterogeneous expert collaboration module and adaptively weighted and fused to reconstruct the decoded features step by step. The heterogeneous expert collaboration module includes a first expert branch for capturing global context dependencies and a second expert branch for extracting directional edge features. S4. Generate the restored image based on the shallow features and the decoded features.

[0020] During remote sensing imaging, the quality of remote sensing images can degrade due to the combined effects of various factors, such as atmospheric interference (e.g., haze, turbulence), changes in lighting conditions (e.g., low light), sensor noise, and platform movement. If such degraded remote sensing images are directly used as input for subsequent tasks, the accuracy of the task will be greatly affected.

[0021] Current mainstream restoration models for degraded images mostly focus only on processing spatial domain features, ignoring the differences in frequency domain between different degradation types (e.g., noise mainly pollutes high-frequency components, while haze mainly weakens low-frequency contrast). The lack of explicit modeling with frequency domain priors makes it difficult for models to accurately decouple the mixed degradation factors.

[0022] In some preferred embodiments, the restoration model includes an input embedding layer, a multi-level encoder, a multi-level decoder, a skip connection module, and an output layer. The input embedding layer extracts shallow features from the input image through convolution; the multi-level encoder progressively reduces the spatial resolution of the feature map and increases the channel dimension to extract deep features from the input image; the multi-level decoder progressively restores the spatial resolution of the feature map and reduces the channel dimension to reconstruct detailed information from the deep features; the skip connection module introduces the output of the encoder at the corresponding level into the decoder; and the output layer outputs the restored image.

[0023] In the actual processing of remote sensing images, this invention found that different types of degradation factors exhibit different characteristics in the frequency domain and spatial domain. However, the existing single standard self-attention mechanism for degraded image restoration relies entirely on data-driven approaches and lacks explicit modeling of frequency domain priors, making it prone to artifacts in mixed haze and noise scenarios. While simple frequency domain processing methods can remove noise, they severely lose edge sharpness and cannot handle complex blurring in the spatial domain.

[0024] In this embodiment of the invention, the multi-level encoder of the restoration model adopts a dual-gated attention mechanism (WSDA), which simultaneously gates and modulates features from both the frequency and spatial domains to perform differentiated processing for different degradation types, thereby achieving effective decoupling of degradation factors.

[0025] Frequency domain modulation can suppress degradation interference within a specific frequency domain range, while spatial domain modulation can focus attention on real edges and texture regions, thus sharpening details even in a noisy background. The deep features obtained in this way have a large receptive field and rich semantic information.

[0026] Remote sensing restoration tasks inherently involve conflicting objectives. For example, denoising typically requires suppressing high-frequency components to eliminate graininess, while deblurring requires enhancing high-frequency components to restore edge details. Existing models employ a fully parameter-sharing approach, which often struggles to simultaneously handle these two contradictory operations within the same network, easily leading to "mediocre solutions" and resulting in artifacts such as blurred edges or residual noise in the restored image.

[0027] The heterogeneous expert collaboration module of the multi-level decoder provided in this embodiment of the invention sets up two parallel expert branches with complementary functions for decoding. The resulting decoded features can take into account both global semantic preservation and directional edge enhancement, effectively solving the target conflict problem in a single network.

[0028] Among them, the first expert branch in the heterogeneous expert collaboration module is used to capture global contextual dependencies, which can ensure the overall structural correctness and semantic consistency of the reconstructed image; the second expert branch is designed for the rich linear ground features in remote sensing images, and is used to extract directional edge features, which can enhance high-frequency detail information such as edges and textures in the image.

[0029] By combining shallow features and decoded features, the original image details can be supplemented into the decoded features, making up for the spatial details that may be lost during the step-by-step downsampling process of deep networks, so that the restored image has both accurate semantic structure and rich texture details.

[0030] In some preferred embodiments, the output layer of the restoration model generates the restored image by performing a global residual connection between the shallow features and the decoded features, and then mapping the fused features back to the image space through convolution.

[0031] In the above scheme, by introducing a dual-gated self-attention mechanism, the features are simultaneously gated and modulated in both the frequency and spatial domains. This effectively distinguishes and suppresses interference caused by different degradation factors, solving the problem that traditional methods struggle to handle composite degradation and significantly improving the image restoration quality in complex real-world scenarios. Furthermore, by using a heterogeneous expert collaboration module, the decoding basic unit is decoupled into two parallel branches: a global semantic expert and a directional edge refiner. This effectively solves the problem of "mediocre solutions" caused by target conflicts in a single network. This embodiment of the invention can handle multiple unknown mixed degradation types using a single restoration model, without needing to predict specific degradation parameters or cascade multiple dedicated models. It can achieve high-fidelity restoration of composite degradation images caused by multiple degradation factors in real-world remote sensing observation scenarios.

[0032] As a preferred embodiment, please refer to Figure 2 The multi-level encoder of the restoration model passes skip connection features to the corresponding level's multi-level decoder through steps P1-P3: P1. Extract the spatial gradient of the output features of the current encoder, and generate a spatial attention mask based on the spatial gradient; P2. Through discrete wavelet transform, the output features of the current level encoder are mapped to the frequency domain of each sub-band, and the channel importance weights of each sub-band are learned through global pooling and fully connected layers to generate frequency domain channel attention masks. P3. Using the spatial attention mask and the frequency domain channel attention mask, the output features of the current level encoder are double-modulated to obtain the filtered skip connection features, and the skip connection features are input into the corresponding level decoder.

[0033] In classic U-Net-like architectures, skip connections between the encoder and decoder typically involve directly passing the encoder's output features to the decoder. However, in complex degradation scenarios, the low-level features extracted by the encoder often contain severe noise or blurred edges. If these features are not cleaned and directly passed to the decoder, they will severely interfere with the higher-level reconstruction process and compromise the fidelity of the geometric structure.

[0034] In this embodiment of the invention, an edge-frequency domain collaborative skip enhancement is set up on the skip connection path between the encoder and the decoder. This enhancement uses two parallel feature extraction paths to cleanse the output features of the current-level encoder. These feature extraction paths are a spatial domain edge path and a frequency domain statistical path.

[0035] Spatial gradient refers to the rate of change of pixel values ​​in the spatial domain. It reflects the position and intensity of structural information such as edges and textures in the feature map. It is extracted by performing convolution operations on the feature map using gradient operators (such as the Sobel operator).

[0036] The spatial attention mask has the same dimensionality as the spatial size of the input features, and the value at each location represents the importance of that location in the spatial structure. The spatial attention mask is used to weight features pixel-by-pixel to enhance the response of salient structural regions such as edges, while suppressing the response of flat or degenerate interference regions.

[0037] In some preferred embodiments, a set of Sobel operator convolution kernels are initialized for the spatial edge path to explicitly extract the spatial gradient information of the input features, and then a spatial attention mask reflecting the saliency of the edge is generated through a lightweight spatial attention module.

[0038] Discrete wavelet transform can transform features from the spatial domain to the frequency domain, decomposing the input features into low-frequency approximate components and multiple high-frequency detail components. In this embodiment of the invention, the discrete wavelet transform decomposes the encoder's output features into different frequency subbands.

[0039] Different types of degradation factors have different degrees of influence on different frequency subbands. In the embodiments of the present invention, the subbands include low-frequency subbands that characterize the overall outline of the image and high-frequency subbands that characterize details such as edges and textures.

[0040] The frequency domain statistical path uses Discrete Wavelet Transform (DWT) to map features to the frequency domain; then global pooling is used to compress the features of each channel into a scalar to obtain the global statistical information of that channel; next, a fully connected layer is used to perform a nonlinear transformation on the statistical information of each channel obtained by global pooling to learn the importance weights of each channel in each subband, and finally generate the frequency domain channel attention mask.

[0041] The frequency domain channel attention mask has the same dimension as the input features, with each channel corresponding to a weight value. The frequency domain channel attention mask is used to perform weighted modulation on different channels in different frequency sub-bands to suppress channels that are severely affected by degradation interference and enhance information-rich channels.

[0042] The dual modulation process refers to simultaneously modulating features using both spatial attention masks and frequency domain channel attention masks. The spatial attention mask weights features pixel-by-pixel in the spatial dimension, while the frequency domain channel attention mask weights features channel-by-channel in the channel dimension. The two work together to achieve full-dimensional purification of features, resulting in filtered skip connection features.

[0043] In some preferred embodiments, skip connection features Represented as: ; in, This refers to the output features of the current level encoder. Spatial attention mask; For frequency domain channel attention masks; This indicates element-wise multiplication; This refers to the intermediate skip connection feature after double modulation; A learnable residual scaling factor used for modulation In the final skip connection feature The intensity of contribution.

[0044] In the above scheme, spatial edge information and frequency domain statistical characteristics are innovatively integrated during the skip connection process between the encoder and decoder, realizing full-dimensional purification of skip connection features. This can filter out degradation interference in skip connections, avoid the propagation and amplification of noise and blurred edges during the decoding process, significantly improve the geometric structure fidelity of the reconstructed image, and make the restored ground feature edges clearer and more accurate.

[0045] As a preferred implementation, step S1 involves inputting the acquired degraded remote sensing image into a pre-trained restoration model, and extracting shallow features of the degraded remote sensing image through the input embedding layer of the restoration model, including: Acquire degraded remote sensing images and normalize the pixel values ​​of the degraded remote sensing images; If the size of the degraded remote sensing image is larger than a preset size threshold, the degraded remote sensing image is divided into blocks using a sliding window strategy. The processed degraded remote sensing image is input into a pre-trained restoration model, and shallow features of the degraded remote sensing image are extracted through the input embedding layer of the restoration model.

[0046] Degraded remote sensing images from different sources exhibit varying pixel value distributions. For instance, 8-bit images range from 0 to 255 pixel values, while 16-bit images range from 0 to 65535 pixel values. Normalization can map the pixel values ​​of an image to the same numerical range, eliminating dimensional differences.

[0047] It should be noted that the embodiments of the present invention perform pixel value normalization processing during the preprocessing process. Correspondingly, the output normalized image needs to be mapped back to the original pixel value range in order to generate the final restored image.

[0048] The restoration model uses a fixed input size during training. When the size of the degraded remote sensing image exceeds a preset size threshold, it cannot be directly input into the model for processing. In this embodiment of the invention, a sliding window strategy is used to divide the large image into multiple image blocks that match the model's input size. The sliding window moves across the image with a set step size, capturing one image block at a time. Preferably, overlapping areas can be set between adjacent image blocks to avoid boundary effects.

[0049] Corresponding to image block preprocessing, the restoration model needs to stitch and merge the output blocks into a complete image to obtain the restored image.

[0050] It is understood that the preprocessing operations performed on the degraded remote sensing images are consistent with the preprocessing operations during the training of the restoration model. In addition to preprocessing methods such as pixel value normalization and block division, techniques such as rotation and denoising can also be used. The specific preprocessing method chosen does not affect the beneficial effects produced by the embodiments of the present invention.

[0051] In some preferred embodiments, shallow features of degraded remote sensing images are extracted using 3×3 convolutional layers.

[0052] In the above scheme, through normalization processing and sliding window block strategy, the model can handle degraded remote sensing images of different sources, sizes and dynamic ranges, and has good versatility.

[0053] As a preferred implementation, step S2 involves using a dual-gated attention mechanism in the multi-level encoder of the restoration model to perform frequency domain modulation and spatial domain modulation on the shallow features, thereby extracting deep features of the degraded remote sensing image level by level, including: The shallow features are input into the multi-level encoder of the restoration model; For each level of the encoder, the frequency-domain gating unit with dual-gated attention mechanism decomposes the input features of the current level into sub-bands of different frequencies, and adaptively learns the weights of each sub-band to obtain the intermediate features after frequency-domain modulation. By transposing the channel of the dual-gated attention mechanism from the attention module, the intermediate features are aggregated globally in context to obtain the attention output features; The spatial gating unit with dual-gated attention mechanism modulates the attention output features pixel-by-pixel confidence in the spatial domain to obtain the output features of the current level. The output features of the current level are then downsampled and used as the input features of the next level encoder. The output features of the last stage of the multi-level encoder are used as the deep features of the degraded remote sensing image.

[0054] In some preferred embodiments, the multi-level encoder consists of four levels, with each level containing a corresponding number of dual-gated attention mechanism (WSDA) modules to extract deep semantic features step by step, while reducing spatial resolution and increasing channel dimension through downsampling operations.

[0055] Specifically, the multi-level encoder contains [4, 6, 6, 8] customized Transformer blocks from the bottom to the top. Each Transformer block uses the dual-gated self-attention mechanism (WSDA) proposed in this invention as its core computing unit.

[0056] In this embodiment of the invention, the dual-gated self-attention mechanism consists of a serial frequency domain gating unit, a channel transpose self-attention module, and a spatial domain gating unit.

[0057] Different types of degradation factors have different effects on different frequency sub-bands. By using frequency domain gating units to adaptively weight different frequency sub-bands, frequency domain interference caused by specific degradation types can be suppressed in a targeted manner, while retaining or enhancing information-rich frequency components.

[0058] In the frequency domain gating unit, the input features are first decomposed into low-frequency approximate components and three high-frequency detail components using the Haar wavelet basis. Then, 1×1 grouped convolutions are applied in parallel to the four sub-bands to generate learnable gating weights for each sub-band.

[0059] Preferably, the operation of the frequency domain gating unit is represented as follows: ; ; in, The input features for the current level; Represents the discrete wavelet transform; Indicates the inverse discrete wavelet transform; These are intermediate features after frequency domain modulation; This is a frequency-domain gated weight matrix, where each element represents the weight value at the corresponding position in the corresponding frequency sub-band.

[0060] After frequency-domain gated modulation, degradation interference in the features has been initially suppressed, but long-range dependencies between different spatial locations have not yet been established. The channel transpose self-attention module enables features at each location to perceive global context information by calculating the correlation between channels. Through channel transpose self-attention, each channel can aggregate information from all other channels, thereby capturing global context dependencies.

[0061] Different spatial locations have varying degrees of importance for subsequent reconstruction. Prominent structural regions are crucial for the geometric fidelity of the restored image, while excessive attention to severely degraded regions may actually hinder the reconstruction process.

[0062] In the spatial gating unit of this invention embodiment, the convolutional layer is used to predict the reconstructed confidence map of each pixel location, and the attention output is modulated pixel by pixel to achieve adaptive modulation of the spatial dimension, thereby enhancing the response of high-confidence structural regions while suppressing interference from low-confidence degenerate regions.

[0063] Preferably, the operation of the spatial gating unit is represented as follows: ; ; in, For attention output features; Reconstructed confidence maps for each pixel location; This refers to the output features of the current level encoder.

[0064] After processing by the current level encoder, the output features of the current level are downsampled, halving the spatial resolution of the feature map while doubling the channel dimension. This output feature is then used as the input for the next level encoder. Through the progressive processing of multiple encoder levels, the spatial resolution of the features decreases progressively, the receptive field expands progressively, the channel dimension increases progressively, and the semantic information becomes progressively richer.

[0065] In the above scheme, the multi-level encoder extracts features step by step. Each level performs feature purification through frequency domain gating and spatial domain gating, avoiding the accumulation and amplification of degradation interference in the deep network. By using a dual-gated attention mechanism to simultaneously gate and modulate features from both the frequency and spatial domains, it is possible to effectively distinguish and suppress interference caused by different degradation factors, achieving effective decoupling of degradation factors. By capturing global dependencies through channel transposition self-attention module, the feature representation can have rich semantic information.

[0066] As a preferred implementation, step S3 involves using a heterogeneous expert collaboration module to perform parallel expert processing and adaptive weighted fusion on the deep features in the multi-level decoder of the restoration model, and reconstructing the decoded features step by step, including: The deep features are input into the multi-level decoder of the restoration model; For each level of the decoder, the global contextual dependencies of the input features of the current level are captured through the first expert branch of the heterogeneous expert collaboration module, and the global semantic features are output; the first expert branch adopts a dual-gated attention mechanism. The directional edge features of the input features at the current level are extracted through the second expert branch of the heterogeneous expert collaboration module; the second expert branch adopts a stacked structure of multi-scale depthwise separable convolution and a dynamic edge gating mechanism. The adaptive fusion unit of the heterogeneous expert collaboration module dynamically calculates the fusion weights of the first expert branch and the second expert branch based on the statistical characteristics of the input features of the current level. Based on the fusion weights, the global semantic features and the directional edge features are weighted and fused to obtain weighted fused features; The weighted fusion features are residually concatenated with the input features of the current level to obtain the output features of the current level; The output features of the current level are upsampled and then fused with the skip connection features passed from the corresponding level encoder to serve as the input features of the next level decoder. The output features of the last stage of the multi-level decoder are used as the decoding features of the degraded remote sensing image.

[0067] In some preferred embodiments, the multi-level decoder employs a heterogeneous expert collaboration mechanism, consisting of three levels, each level including [2,4,4] Transformer blocks.

[0068] In this embodiment of the invention, each basic computing unit in the multi-level decoder is decoupled into two complementary parallel branches.

[0069] The first expert branch employs the same dual-gated attention mechanism as the encoder, focusing on capturing long-range dependencies and global semantic consistency. It can effectively capture global contextual dependencies and ensure the overall semantic consistency of the reconstructed image.

[0070] For example, in the first expert branch, the input features are first decomposed into sub-bands of different frequencies using a frequency-domain gating unit, and the weights of each sub-band are adaptively learned to suppress frequency-domain interference caused by specific degradation types. The frequency-domain-gated features enter the channel transpose self-attention module, where self-attention weights are calculated along the channel dimension to capture global contextual dependencies, enabling features at each location to perceive information from the entire image. Finally, the attention output is spatially modulated pixel-by-pixel in terms of confidence using a spatial-domain gating unit to enhance the response of high-confidence structural regions and suppress interference from low-confidence degradation regions.

[0071] The second expert branch employs a stacked structure of multi-scale depth-separable convolutions and a dynamic edge gating mechanism. Designed for the rich linear ground features in remote sensing images, it generates a spatially adaptive activation mask by analyzing the local gradient response of the input features, which can extract the directional edge features of the current level input features.

[0072] In the second expert branch, directional features at different scales can be captured by convolution kernels of different sizes. Depth-separable convolution decomposes standard convolution into depthwise convolution and pointwise convolution, which greatly reduces the number of parameters and computation. The dynamic edge gating mechanism generates spatially adaptive activation masks by analyzing the local gradient response of input features and modulates the output of multi-scale convolutions position by position, enabling the network to adaptively enhance the feature response of edge regions while suppressing noise in flat regions.

[0073] The importance of global semantic preservation and directional edge enhancement varies across different types of degradation scenarios. In this embodiment of the invention, the outputs of the two expert branches are adaptively weighted and fused using learnable soft fusion weights, where the fusion weights are dynamically generated from the statistical features of the input features.

[0074] Preferably, the output features of the current level decoder are represented as follows: ; in, The global semantic features output by the first expert branch; The fusion weight for the first expert branch; The directional edge features output by the second expert branch; The fusion weight for the second expert branch; These are the input features of the current level decoder.

[0075] Skip-connect fusion introduces low-level detail information extracted by the encoder into the decoder, compensating for spatial details that may be lost during upsampling. Notably, in this embodiment of the invention, the skip-connect features are purified by edge-frequency domain collaborative skip enhancement, filtering out degenerative interference such as noise and blurred edges, resulting in higher quality fused features.

[0076] Most existing integrated image restoration methods adopt a single network structure with fully shared parameters or a simple hybrid expert architecture. In the above scheme, by decoupling the basic decoding unit into two parallel expert branches, which focus on global semantic preservation and directional edge enhancement respectively, the problem of conflict between the objectives of "denoising" and "deblurring" in a single network is effectively solved, and "mediocre solutions" are avoided.

[0077] As a preferred embodiment, please refer to Figure 3The training of the restoration model is performed through steps T1-T5: T1. Based on the collected multi-source clear remote sensing images and predefined degradation factors, synthesize the corresponding degraded images to form a training sample set; T2. Preprocess the degraded image, and input the preprocessed degraded image into the pre-constructed restoration model to generate a predicted restored image; T3. Calculate the loss function between the predicted restored image and the multi-source clear remote sensing image; T4. Update the parameters of the restored model using the backpropagation algorithm based on the loss function. T5. Continue until the preset iteration stopping condition is met to obtain the trained restored model.

[0078] In this embodiment of the invention, firstly, multiple clear remote sensing images are acquired. Then, based on predefined degradation factors, the clear images are degraded and synthesized to generate corresponding degraded images. The original clear images are then paired with the synthesized degraded images to form a training sample set.

[0079] By constructing a training sample set through synthesis, a large number of pairs of degraded-sharp images can be obtained, providing a data foundation for supervised training of the model.

[0080] By preprocessing the degraded image, the input data can be made to meet the input requirements of the model. The preprocessed degraded image is then input into the pre-built restoration model, which performs forward propagation calculations to complete the mapping from the degraded image to the predicted restored image.

[0081] The initial parameters of the pre-built restoration model are randomized. At this point, the model does not have the ability to map from degraded images to clear images. Through training, the model can learn this mapping relationship from a large number of paired degraded-clear image samples, thereby gradually optimizing the model parameters and finally obtaining a restoration model that can be used for inference.

[0082] In order to quantify the prediction effect of the model under the current parameters and guide the model to optimize in the right direction, in this embodiment of the invention, the predicted restoration model is compared with the original multi-source clear remote sensing image, and the loss function value is calculated.

[0083] A larger loss function value indicates a greater difference between the predicted result and the real, clear image, resulting in poorer model performance; a smaller loss function value indicates a closer prediction result to the real, clear image. Through the loss function, the model can obtain a quantifiable optimization objective.

[0084] After the loss function is calculated, the model needs to adjust its parameters based on the loss value to reduce prediction error. In this embodiment of the invention, the gradient of the loss function with respect to each model parameter is calculated using the backpropagation algorithm, and then the optimizer updates the model parameters according to the gradient direction. Each parameter update adjusts the model in the direction of reducing the loss function value. Through repeated forward propagation, loss calculation, and parameter updates, the model parameters are gradually optimized, and the difference between the predicted restored image and the real clear image gradually decreases.

[0085] In some preferred embodiments, the optimizer is the AdamW optimizer, whose hyperparameters are... Set to 0.9, Set to 0.999; initial learning rate set to The training batch size was set to 16, and the total number of training epochs was set to 200. During training, images were randomly cropped to 128×128 pixels, and horizontal and vertical flipping was used to prevent overfitting.

[0086] When the preset stopping condition is met, the model parameters have been optimized from an initial random state to a state that can effectively restore degraded images to clear images. The trained restoration model can be used for inference restoration of unknown degraded images without additional adjustments.

[0087] Furthermore, as a preferred implementation, step T1 involves synthesizing corresponding degraded images based on the acquired multi-source clear remote sensing images and predefined degradation factors to form a training sample set, which is then executed through steps T11-T13. T11. Acquire multi-source clear remote sensing images; the multi-source clear remote sensing images include clear remote sensing images of different land cover types; T12. Using a physics-driven degradation model, at least one degradation factor is applied to the multi-source clear remote sensing image to generate a corresponding degraded image; the degradation factors include low light, blur, haze, and noise; when multiple degradation factors are applied, the application order is low light, blur, haze, and noise; T13. Based on the multi-source clear remote sensing images and the degraded images, a training sample set is obtained.

[0088] In a preferred embodiment of the present invention, the multi-source clear remote sensing images are derived from real remote sensing image data acquired by Gaofen-multi-mode satellites. Specifically, clear images covering various terrain features are selected from the Gaofen-multi-mode satellite image library, including different land cover types such as farmland, mountains, cities, water bodies, and vegetation-covered areas, to ensure the diversity of land cover in the training data.

[0089] The criteria for image selection include: spatial resolution no lower than the original sensor resolution, no obvious atmospheric degradation or noise interference, clearly distinguishable ground feature edges, and rich texture details.

[0090] Early research primarily relied on filtering-based techniques or variational optimization methods. However, these methods are often based on overly simplified physical assumptions (such as a single degradation model), making it difficult to characterize the nonlinear coupling characteristics of multiple degradation factors (such as the simultaneous presence of haze, noise, blur, and low light) in real remote sensing scenes. Furthermore, these methods often involve cumbersome iterative optimization processes, resulting in low computational efficiency and making it difficult to meet the needs of large-scale remote sensing data processing.

[0091] With the development of deep learning technology, convolutional neural network (CNN)-based methods have achieved significant breakthroughs in image restoration tasks. However, most of these deep learning restoration methods follow a "single-task learning" paradigm, that is, designing a dedicated network model for a specific type of degradation (such as denoising only, dehazing only, or deblurring only). These methods face severe challenges when dealing with real-world remote sensing scenarios: degradation in real-world scenes is often a mixture of multiple unknown types. Using a cascaded approach with multiple single-task models not only leads to a cumbersome processing flow but also causes errors to accumulate at each stage of the cascade process, while significantly increasing computational resource consumption.

[0092] In this embodiment of the invention, degradation factors cover four categories: low light, blur, haze, and noise. It should be noted that for single-factor degradation, the corresponding degradation factor is directly applied to generate a degraded image; for multi-factor composite degradation, the factors are applied in the following order: first, low light degradation is applied, then blur degradation is applied, then haze degradation is applied, and finally noise degradation is applied.

[0093] In the above scheme, clear images from multiple sources cover different land cover types, and diverse degradation parameters generate samples with different degradation degrees and combinations, which can make the training sample set rich in diversity. At the same time, the physical-driven degradation model and the reasonable order of compound degradation application can ensure that the synthetic data are consistent with the degradation characteristics of the real remote sensing observation scene.

[0094] Furthermore, preferably, step T12, employing a physics-driven degradation model, applies at least one degradation factor to the multi-source clear remote sensing image to generate a corresponding degraded image, including: When the applied degradation factor is low light, a power transformation is performed on each pixel value of the image to be degraded to obtain a first degraded image; the power exponent in the power transformation is randomly sampled within a preset range of darkness coefficient values. When the applied degradation factor is blurring, an anisotropic Gaussian kernel is used to perform two-dimensional filtering on the image to be degraded, resulting in a second degraded image. When the applied degradation factor is haze, haze synthesis is performed on the image to be degraded based on the atmospheric scattering model to obtain a third degraded image; When the applied degradation factor is noise, an additive Gaussian noise model is used to add noise to the image to be degraded, resulting in a fourth degraded image.

[0095] This invention employs a physics-driven degradation synthesis strategy. The required degradation model parameters fall into four main categories: low-light degradation parameters, blur degradation parameters, haze degradation parameters, and noise degradation parameters. These parameters are not obtained externally but are generated through random sampling, while adhering to the physical constraints of real remote sensing imaging. The range of these parameters is set based on the statistical characteristics of real remote sensing observation scenarios to ensure consistency between the synthesized data and the actual degradation distribution.

[0096] For example, the saturation coefficient γ for low-light degradation follows a uniform distribution U(1.0, 3.0); the anisotropic Gaussian kernel size for blurring degradation is selected from odd numbers within the range, and the principal axis standard deviation... , The atmospheric light parameters and transmittance parameters of haze degradation are modeled based on the real atmospheric structure; the standard deviation σ of noise degradation ranges from [15,35].

[0097] In this embodiment of the invention, low-light degradation is simulated by power transformation. The power exponent of the power transformation represents the degree of low light; the larger the power exponent, the more severe the image brightness attenuation. By simulating the physical process of light attenuation through power transformation, the synthesized image can conform to the characteristics of low-light imaging.

[0098] In some preferred embodiments, the first degraded image is represented as: ; in, is the darkness coefficient randomly sampled from U(1.0,3.0); x is the pixel position; Let x be the pixel value of the image to be degraded at position x.

[0099] When blur degradation is required, an anisotropic Gaussian kernel is used to perform a two-dimensional filtering operation on the image to be degraded. The anisotropic Gaussian kernel has different standard deviations in different directions, which can simulate an asymmetric blur effect. By convolving the Gaussian kernel with the image to be degraded, the image blurring is achieved, resulting in a second degraded image.

[0100] When haze degradation needs to be applied, haze synthesis is performed on the image to be degraded based on an atmospheric scattering model. The atmospheric scattering model describes the attenuation and scattering process of light propagating in the haze medium, and can be used to simulate the haze effect under different concentrations and distributions.

[0101] In some preferred embodiments, the third degraded image is represented as: ; in, For pixel position; Let x be the pixel value of the image to be degraded at position x; The transmittance map is generated through linear mapping. , This is the upper limit of transmittance. This is the lower limit of transmittance. This is a fog density map based on the actual atmospheric structure; Atmospheric light, , The maximum pixel value. and This is an adjustable parameter.

[0102] When noise degradation needs to be applied, an additive Gaussian noise model is used to add noise to the image to be degraded. The additive Gaussian noise model adds random noise following a Gaussian distribution to each pixel of the image, with the noise intensity determined by the standard deviation of the Gaussian distribution. By adding noise, degradation factors such as sensor electronic noise and quantization errors can be simulated.

[0103] In some preferred embodiments, the fourth degraded image is represented as: ; in, For pixel position; Let x be the pixel value of the image to be degraded at position x; is the standard deviation of random noise sampled from the range [15,35].

[0104] It should be noted that when multiple degradation factors need to be applied, the above degradation operations are performed sequentially according to a preset cascading order: first, low-light degradation is performed to obtain the first degraded image; then, this image is used as the image to be degraded to perform blur degradation or haze degradation; finally, noise degradation is performed. The final degraded image is used as the output.

[0105] In the above scheme, the application of each degradation factor is based on a classical physical model, and the synthesized degradation image conforms to the degradation characteristics of real remote sensing imaging. Furthermore, the embodiments of the present invention cover four common degradation factors: low light, blur, haze, and noise, and support single-factor degradation simulation and composite-factor degradation simulation, which can cover the main degradation sources in real scenes.

[0106] As a preferred embodiment, step T3, calculating the loss function between the predicted restored image and the multi-source clear remote sensing image, includes: Calculate the pixel-level difference in color space between the predicted restored image and the multi-source clear remote sensing image; Calculate the spectral difference between the predicted restored image and the multi-source sharp remote sensing image in the Fourier domain; Calculate the semantic difference between the predicted restored image and the multi-source sharp remote sensing image in the high-level semantic feature space; Based on preset weight parameters, the pixel-level differences, the spectral differences, and the semantic differences are weighted and fused to obtain a loss function.

[0107] The predicted restored image is compared pixel-by-pixel with the original sharp image in the color space. The calculated pixel-level difference reflects the degree of numerical difference between the two images at each pixel location, and is a fundamental indicator for measuring the accuracy of image reconstruction. The pixel-level difference reflects how closely the predicted restored image is numerically similar to the real sharp image, ensuring that the model learns the ability to reconstruct accurately at the pixel level.

[0108] Preferably, the pixel-level difference measure includes the absolute difference and the squared difference in the RGB space.

[0109] Spectral differences reflect the differences in reconstruction quality of an image across different frequency components. For example, low-frequency components correspond to the overall structure and illumination distribution of the image, while high-frequency components correspond to details such as edges and textures. Spectral difference-guided models focus on reconstructing the frequency domain characteristics of an image, which is beneficial for restoring the overall structure, periodic features, and edge details of the image.

[0110] Semantic discrepancies reflect the degree of difference between the predicted restored image and the true sharp image in terms of content perception and semantic understanding. Calculating semantic discrepancies can guide models to generate restored images that are perceptually more realistic and semantically more consistent, rather than simply limiting them to pixel-level matching.

[0111] In some preferred embodiments, the loss function is expressed as: ; in, The difference is at the pixel level; For spectral differences; For semantic differences; , and For example, the preset weight parameters are: , , .

[0112] In the above scheme, the quality of the predicted restored image is comprehensively evaluated from three dimensions: pixel level in color space, spectral level in Fourier domain, and high-level semantic feature space. The three dimensions work together to guide the restoration model to generate high-quality restored images.

[0113] To verify the effectiveness and superiority of the embodiments of the present invention, multiple sets of test embodiments were designed for verification. The following embodiments cover different degradation scenarios, different datasets, and different ablation configurations.

[0114] Test Example 1: To verify the restoration performance of the method of the present invention under a single degradation type (dehazing, deblurring, denoising, and low-light enhancement), a quantitative comparison was made with existing advanced methods.

[0115] We used the RRSHID dataset as the basis for our experiments, which originally contained only haze-degraded images. Following the data preparation process described in this invention, we expanded the dataset to synthesize images containing three types of degradation: blurred, low-light, and noisy. The experiments were conducted on a single NVIDIA A100 GPU, with the network training configuration as follows: batch size 16, learning rate... Training epochs 200, optimizer AdamW ( =0.9, =0.999), loss function weights =1.0、 =0.1、 =0.01.

[0116] Based on this, 10 existing state-of-the-art methods were selected as benchmarks for comparison, including: MoCE-IR, AirNet, Uformer, MPRNet, MIRNetV2, NAFNet, Ada4DIR, DGUNet, AdaIR, and MPMF-Net. All comparison methods were retrained on this experimental dataset to ensure fairness. The method of this invention (denoted as Ours) and the comparison methods were trained under the same training configuration. Three metrics were used for quantitative evaluation: PSNR (Peak signal-to-noise ratio), SSIM (Structure Similarity Index Measure), and LPIPS (Learned Perceptual Image Patch Similarity). The higher the metric value (the lower the LPIPS), the better the restoration quality. The quantitative evaluation results are shown in Tables 1 and 2.

[0117] Table 1 Table 2 As can be seen, in the dehazing task, the method of this invention achieves the best or second-best performance in all three difficulty levels: Dehazing-Thin, Dehazing-Moderate, and Dehazing-Thick. Specifically, in the thin fog scene, the PSNR reaches 24.34dB, SSIM reaches 0.6790, and LPIPS is as low as 0.3842; in the dense fog scene, the PSNR reaches 25.39dB and SSIM reaches 0.7500, significantly outperforming the comparative methods. In the deblurring task, the method of this invention achieves a PSNR of 36.75dB, an SSIM of 0.9305, and LPIPS as low as 0.1303, demonstrating a significant advantage in the deblurring task. In the low-light enhancement task, the PSNR reaches 42.09dB, the SSIM reaches 0.9918, and LPIPS is as low as 0.0108, showing excellent performance in brightness restoration and detail preservation. In the denoising task, the PSNR reached 35.89dB, the SSIM reached 0.9235, and the LPIPS reached 0.1086, effectively suppressing sensor noise while preserving ground texture.

[0118] In addition, the total number of parameters in the network of this invention is 10.53M. While achieving the above-mentioned excellent performance, the number of parameters is significantly lower than that of methods such as Uformer (50.88M) and PromptIR (35.59M), which demonstrates the advantages of lightweight design.

[0119] Test Example 2: For image restoration under complex degradation scenarios, we used the RSMUTD dataset independently constructed in this invention. This dataset contains 10 degradation categories: 4 single-factor degradations (haze H, blur B, low light L, noise N) and 6 complex degradations (L+H, L+B, N+H, N+B, N+L+H, N+L+B). The dataset contains 12,260 image pairs, of which 10,240 were used for training and 2,020 for testing. The experimental hardware and training configuration were consistent with those in Example 1.

[0120] Ten representative state-of-the-art integrated image restoration methods were selected for comparison: AirNet, Uformer, MPRNet, DGUNet, NAFNet, PromptIR, AdaIR, MoCE-IR, TransWeather, and MEASNet. All methods were trained uniformly on the RSMUTD dataset, and their comprehensive performance on 10 types of degradation was evaluated. The evaluation results are shown in Tables 3-6.

[0121] Table 3 Table 4 Table 5 Table 6 As can be seen, the method of this invention achieves optimal (bold) or near-optimal (underlined) performance in the vast majority of degradation categories. In the dehazing task, the PSNR reaches 26.71dB, SSIM reaches 0.9477, and LPIPS is as low as 0.0466; in the deblurring task, the PSNR reaches 29.64dB, SSIM reaches 0.9037, and LPIPS is as low as 0.1014, which is 1.99dB higher than the second-place TransWeather method; in the denoising task, the PSNR reaches 32.33dB, SSIM reaches 0.9039, and LPIPS is as low as 0.0920; in the low-light enhancement task, the PSNR reaches 30.98dB, SSIM reaches 0.9757, and LPIPS is as low as 0.0163.

[0122] In complex degradation scenarios, the advantages of the method of this invention are even more significant. In the L+H (low light + haze) combination, the PSNR reaches 26.28 dB and the SSIM reaches 0.9302; in the L+B (low light + blur) combination, the PSNR reaches 27.63 dB and the SSIM reaches 0.8919; in the N+H (noise + haze) combination, the PSNR reaches 23.76 dB and the SSIM reaches 0.7624; in the N+B (noise + blur) combination, the PSNR reaches 26.93 dB and the SSIM reaches 0.7675; in the N+L+H (noise + low light + haze) triple combination, the PSNR reaches 22.77 dB and the SSIM reaches 0.7077; in the N+L+B (noise + low light + blur) triple combination, the PSNR reaches 25.02 dB and the SSIM reaches 0.7194. The above results demonstrate that the method of the present invention can effectively decouple various degenerate coupling effects and maintain stable high performance in complex scenarios.

[0123] Test Example 3: To verify the generalization performance of the method in real remote sensing scenarios and eliminate the influence of domain differences between synthetic and real data, we constructed the real-world degraded remote sensing image dataset RRS100. This dataset was selected from the RRSD300 dataset and original images from the Gaofen-Multi-mode satellite and contains 100 remote sensing images affected by real atmospheric interference (haze, clouds, uneven illumination, etc.). Since real images lack corresponding clear ground truth, the No-Reference Image Quality Evaluation (NIQE) metric and visualization methods were used for evaluation; a lower NIQE value indicates better image quality.

[0124] We selected five comparative methods for testing on the RRS100 dataset: MoCE-IR, AirNet, MPRNet, Uformer, and PromptIR. All methods used model weights pre-trained on synthetic datasets and directly inferred on real images without additional fine-tuning. Our proposed method also uses pre-trained weights to verify its zero-shot generalization ability. The evaluation results are shown in Table 7.

[0125] Table 7 Table 7 shows a comparison of NIQE (Natural Image Quality Evaluator) metrics and model parameter count. The method of this invention achieves the lowest NIQE value of 4.237, significantly outperforming PromptIR (4.339), Uformer (4.935), and MPRNet (4.486). Furthermore, it also performs well in LIQE (Language-Image Quality Evaluator) and PIQE (Perception-based Image Quality Evaluator) metrics. Regarding parameter count, the method of this invention has 10.53M parameters, far lower than PromptIR (35.59M) and Uformer (50.88M), demonstrating a highly efficient design.

[0126] Please see Figure 4 , Figure 4 Visualization results are presented for testing under various comparative methods and the method of this invention. It can be seen that the method of this invention can effectively remove haze and cloud interference from real remote sensing images, restore the true colors and details of the land surface, and avoid over-enhancement or artifact generation, resulting in a natural and realistic visual effect.

[0127] Test Example 4: The test embodiments of this invention also verify the effectiveness of the core modules (WSDA, DER, EF-Skip) through ablation tests, and quantitatively analyze the contribution of each module to the overall performance.

[0128] Based on the RSMUTD dataset, four ablation configurations were designed: baseline (baseline model, removing the dual-gated WSDA, DER module and dynamic routing, retaining only single-branch processing), +WSDA (embedding the dual-gated self-attention module in the baseline), +DER (embedding DER in the Decoder on the basis of WSDA), and +EF-Skip (complete model, containing all three core modules).

[0129] The module contribution was evaluated from two dimensions: first, the overall performance dimension, which calculated the average PSNR and SSIM on 10 types of degradation tasks; and second, the task decomposition dimension, which analyzed the specific improvement effect of each module on the average PSNR of four types of tasks: dehazing, deblurring, denoising, and low-light enhancement.

[0130] The overall performance ablation results are shown in Table 8. It can be seen that the baseline model achieves an average PSNR of 26.69 dB and SSIM of 0.8473. After embedding the WSDA module, the performance improves to PSNR of 26.89 dB and SSIM of 0.8497, an improvement of 0.20 dB / 0.0024, indicating that the frequency-spatial domain degradation decoupling mechanism can effectively purify feature representations. After embedding the DER module, the performance improves to PSNR of 27.07 dB and SSIM of 0.8504, an improvement of 0.38 dB / 0.0031, indicating that the heterogeneous expert collaboration mechanism can effectively balance multi-task conflicts. The complete model (+EF-Skip) achieves the best performance with PSNR of 27.21 dB and SSIM of 0.8510, an improvement of 0.52 dB / 0.0037 compared to the baseline, indicating that the edge-frequency domain collaborative skip enhancement mechanism can further optimize the cross-stage feature transfer quality.

[0131] Table 8 Table 9 shows the ablation results for the task decomposition dimensions. It can be seen that in dehazing tasks, the +DER configuration achieves the best PSNR of 24.91dB, indicating that the second expert branch plays a crucial role in detail recovery of hazy scenes; in deblurring tasks, the complete configuration achieves the best PSNR of 27.31dB, indicating that feature cleansing of skip connections is essential for blur recovery; in denoising tasks, the complete configuration achieves the best PSNR of 26.16dB, indicating that frequency domain gating has a significant effect on suppressing high-frequency noise; and in low-light enhancement tasks, the complete configuration achieves the best PSNR of 26.54dB, indicating that the expert collaboration mechanism effectively supports illumination recovery.

[0132] Table 9 The degraded remote sensing image restoration method provided in this invention introduces a dual-gated self-attention mechanism, simultaneously gating and modulating features in both the frequency and spatial domains. This effectively distinguishes and suppresses interference caused by different degradation factors, solving the problem of traditional methods struggling to handle complex degradation and significantly improving image restoration quality in complex real-world scenarios. Furthermore, by employing a heterogeneous expert collaboration module, the decoding basic unit is decoupled into two parallel branches: a global semantic expert and a directional edge refiner. This effectively addresses the "mediocre solution" problem caused by target conflicts in a single network. This invention can handle multiple unknown mixed degradation types using a single restoration model, without needing to predict specific degradation parameters or cascade multiple dedicated models. It can achieve high-fidelity restoration of complex degraded images caused by multiple degradation factors in real-world remote sensing observation scenarios.

[0133] This invention provides a system for restoring degraded remote sensing images. Please refer to [link / reference]. Figure 5 The degraded remote sensing image restoration system includes a shallow feature extraction module 11, an encoding module 12, a decoding module 13, and a restoration result generation module 14, wherein: The shallow feature extraction module 11 is used to input the acquired degraded remote sensing image into a pre-trained restoration model and extract the shallow features of the degraded remote sensing image through the input embedding layer of the restoration model. Encoding module 12 is used to perform frequency domain modulation and spatial domain modulation on the shallow features in the multi-level encoder of the restoration model through a dual-gated attention mechanism, and extract the deep features of the degraded remote sensing image step by step. Decoding module 13 is used to perform parallel expert processing and adaptive weighted fusion on the deep features through a heterogeneous expert collaboration module in the multi-level decoder of the restoration model, and reconstruct the decoded features step by step; the heterogeneous expert collaboration module includes a first expert branch for capturing global context dependencies and a second expert branch for extracting directional edge features; The restoration result generation module 14 is used to generate a restoration result image based on the shallow features and the decoding features.

[0134] As a preferred embodiment, the method for the multi-level encoder of the restoration model to pass skip connection features to the corresponding level multi-level decoder includes: Extract the spatial gradient of the output features of the current encoder, and generate a spatial attention mask based on the spatial gradient; By using discrete wavelet transform, the output features of the current level encoder are mapped to the frequency domain of each sub-band, and the channel importance weights of each sub-band are learned through global pooling and fully connected layers to generate frequency domain channel attention masks. The spatial attention mask and the frequency domain channel attention mask are used to perform dual modulation on the output features of the current level encoder to obtain filtered skip connection features, which are then input into the corresponding level decoder.

[0135] In a preferred embodiment, the shallow feature extraction module 11 is specifically used for: Acquire degraded remote sensing images and normalize the pixel values ​​of the degraded remote sensing images; If the size of the degraded remote sensing image is larger than a preset size threshold, the degraded remote sensing image is divided into blocks using a sliding window strategy. The processed degraded remote sensing image is input into a pre-trained restoration model, and shallow features of the degraded remote sensing image are extracted through the input embedding layer of the restoration model.

[0136] In a preferred embodiment, the encoding module 12 is specifically used for: The shallow features are input into the multi-level encoder of the restoration model; For each level of the encoder, the frequency-domain gating unit with dual-gated attention mechanism decomposes the input features of the current level into sub-bands of different frequencies, and adaptively learns the weights of each sub-band to obtain the intermediate features after frequency-domain modulation. By transposing the channel of the dual-gated attention mechanism from the attention module, the intermediate features are aggregated globally in context to obtain the attention output features; The spatial gating unit with dual-gated attention mechanism modulates the attention output features pixel-by-pixel confidence in the spatial domain to obtain the output features of the current level. The output features of the current level are then downsampled and used as the input features of the next level encoder. The output features of the last stage of the multi-level encoder are used as the deep features of the degraded remote sensing image.

[0137] In a preferred embodiment, the decoding module 13 is specifically used for: The deep features are input into the multi-level decoder of the restoration model; For each level of the decoder, the global contextual dependencies of the input features of the current level are captured through the first expert branch of the heterogeneous expert collaboration module, and the global semantic features are output; the first expert branch adopts a dual-gated attention mechanism. The directional edge features of the input features at the current level are extracted through the second expert branch of the heterogeneous expert collaboration module; the second expert branch adopts a stacked structure of multi-scale depthwise separable convolution and a dynamic edge gating mechanism. The adaptive fusion unit of the heterogeneous expert collaboration module dynamically calculates the fusion weights of the first expert branch and the second expert branch based on the statistical characteristics of the input features of the current level. Based on the fusion weights, the global semantic features and the directional edge features are weighted and fused to obtain weighted fused features; The weighted fusion features are residually concatenated with the input features of the current level to obtain the output features of the current level; The output features of the current level are upsampled and then fused with the skip connection features passed from the corresponding level encoder to serve as the input features of the next level decoder. The output features of the last stage of the multi-level decoder are used as the decoding features of the degraded remote sensing image.

[0138] As a preferred embodiment, the training method of the restoration model includes: Based on the collected multi-source clear remote sensing images and predefined degradation factors, corresponding degraded images are synthesized to form a training sample set; The degraded image is preprocessed, and the preprocessed degraded image is input into a pre-constructed restoration model to generate a predicted restored image; Calculate the loss function between the predicted restored image and the multi-source clear remote sensing image; Based on the loss function, the parameters of the restored model are updated using the backpropagation algorithm; The iteration continues until the preset stopping condition is met, at which point the trained restored model is obtained.

[0139] Further, preferably, the step of synthesizing corresponding degraded images based on acquired multi-source clear remote sensing images and predefined degradation factors to form a training sample set includes: Acquire multi-source clear remote sensing images; the multi-source clear remote sensing images include clear remote sensing images of different land cover types; A physics-driven degradation model is used to apply at least one degradation factor to the multi-source clear remote sensing image to generate a corresponding degraded image; the degradation factors include low light, blur, haze, and noise; when multiple degradation factors are applied, the application order is low light, blur, haze, and noise. A training sample set is obtained based on the multi-source clear remote sensing images and the degraded images.

[0140] Furthermore, as a preferred embodiment, the step of employing a physics-driven degradation model to apply at least one degradation factor to the multi-source clear remote sensing image to generate a corresponding degraded image includes: When the applied degradation factor is low light, a power transformation is performed on each pixel value of the image to be degraded to obtain a first degraded image; the power exponent in the power transformation is randomly sampled within a preset range of darkness coefficient values. When the applied degradation factor is blurring, an anisotropic Gaussian kernel is used to perform two-dimensional filtering on the image to be degraded, resulting in a second degraded image. When the applied degradation factor is haze, haze synthesis is performed on the image to be degraded based on the atmospheric scattering model to obtain a third degraded image; When the applied degradation factor is noise, an additive Gaussian noise model is used to add noise to the image to be degraded, resulting in a fourth degraded image.

[0141] Preferably, calculating the loss function between the predicted restored image and the multi-source sharp remote sensing image includes: Calculate the pixel-level difference in color space between the predicted restored image and the multi-source clear remote sensing image; Calculate the spectral difference between the predicted restored image and the multi-source sharp remote sensing image in the Fourier domain; Calculate the semantic difference between the predicted restored image and the multi-source sharp remote sensing image in the high-level semantic feature space; Based on preset weight parameters, the pixel-level differences, the spectral differences, and the semantic differences are weighted and fused to obtain a loss function.

[0142] The degradation remote sensing image restoration system provided in this invention introduces a dual-gated self-attention mechanism, simultaneously gating and modulating features in both the frequency and spatial domains. This effectively distinguishes and suppresses interference caused by different degradation factors, solving the problem of traditional methods struggling to handle complex degradation and significantly improving image restoration quality in complex real-world scenarios. Furthermore, through a heterogeneous expert collaboration module, the decoding basic unit is decoupled into two parallel branches: a global semantic expert and a directional edge refiner. This effectively solves the problem of "mediocre solutions" caused by target conflicts in a single network. This invention can handle multiple unknown mixed degradation types using a single restoration model, without needing to know the specific degradation parameters or cascade multiple dedicated models. It can achieve high-fidelity restoration of complex degradation images caused by multiple degradation factors in real-world remote sensing observation scenarios.

[0143] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0144] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for restoring degraded remote sensing images, characterized in that, include: The acquired degraded remote sensing image is input into a pre-trained restoration model, and shallow features of the degraded remote sensing image are extracted through the input embedding layer of the restoration model. In the multi-level encoder of the restoration model, the shallow features are frequency-domain modulated and spatial-domain modulated through a dual-gated attention mechanism to extract the deep features of the degraded remote sensing image step by step. In the multi-level decoder of the restoration model, the deep features are processed in parallel by a heterogeneous expert collaboration module and adaptively weighted and fused to reconstruct the decoded features step by step. The heterogeneous expert collaboration module includes a first expert branch for capturing global context dependencies and a second expert branch for extracting directional edge features; Based on the shallow features and the decoded features, a restored image is generated; The step of inputting the acquired degraded remote sensing image into a pre-trained restoration model, and extracting shallow features of the degraded remote sensing image through the input embedding layer of the restoration model, includes: Acquire degraded remote sensing images and normalize the pixel values ​​of the degraded remote sensing images; If the size of the degraded remote sensing image is larger than a preset size threshold, the degraded remote sensing image is divided into blocks using a sliding window strategy. The processed degraded remote sensing image is input into a pre-trained restoration model, and shallow features of the degraded remote sensing image are extracted through the input embedding layer of the restoration model. In the multi-level decoder of the restoration model, the deep features are processed in parallel by a heterogeneous expert collaboration module and then adaptively weighted and fused to reconstruct the decoded features step by step, including: The deep features are input into the multi-level decoder of the restoration model; For each level of the decoder, the global contextual dependencies of the input features of the current level are captured through the first expert branch of the heterogeneous expert collaboration module, and the global semantic features are output; the first expert branch adopts a dual-gated attention mechanism. The second expert branch of the heterogeneous expert collaboration module extracts the directional edge features of the input features of the current level; the second expert branch adopts a stacked structure of multi-scale depthwise separable convolution and a dynamic edge gating mechanism. The adaptive fusion unit of the heterogeneous expert collaboration module dynamically calculates the fusion weights of the first expert branch and the second expert branch based on the statistical characteristics of the input features of the current level. Based on the fusion weights, the global semantic features and the directional edge features are weighted and fused to obtain weighted fused features; The weighted fusion features are residually concatenated with the input features of the current level to obtain the output features of the current level; The output features of the current level are upsampled and then fused with the skip connection features passed from the corresponding level encoder to serve as the input features of the next level decoder. The output features of the last stage of the multi-level decoder are used as the decoding features of the degraded remote sensing image.

2. The method for restoring degraded remote sensing images as described in claim 1, characterized in that, The method for the multi-level encoder of the restoration model to pass skip connection features to the corresponding level multi-level decoder includes: Extract the spatial gradient of the output features of the current encoder, and generate a spatial attention mask based on the spatial gradient; By using discrete wavelet transform, the output features of the current level encoder are mapped to the frequency domain of each sub-band, and the channel importance weights of each sub-band are learned through global pooling and fully connected layers to generate frequency domain channel attention masks. The spatial attention mask and the frequency domain channel attention mask are used to perform dual modulation on the output features of the current level encoder to obtain filtered skip connection features, which are then input into the corresponding level decoder.

3. The method for restoring degraded remote sensing images as described in claim 1, characterized in that, The shallow features in the multi-level encoder of the restoration model are frequency-domain modulated and spatial-domain modulated using a dual-gated attention mechanism to extract deep features of the degraded remote sensing image step by step, including: The shallow features are input into the multi-level encoder of the restoration model; For each level of the encoder, the frequency-domain gating unit with dual-gated attention mechanism decomposes the input features of the current level into sub-bands of different frequencies, and adaptively learns the weights of each sub-band to obtain the intermediate features after frequency-domain modulation. By transposing the channel of the dual-gated attention mechanism from the attention module, the intermediate features are aggregated globally in context to obtain the attention output features; The spatial gating unit with dual-gated attention mechanism modulates the attention output features pixel-by-pixel confidence in the spatial domain to obtain the output features of the current level. The output features of the current level are then downsampled and used as the input features of the next level encoder. The output features of the last stage of the multi-level encoder are used as the deep features of the degraded remote sensing image.

4. The method for restoring degraded remote sensing images as described in claim 1, characterized in that, The training method for the restoration model includes: Based on the collected multi-source clear remote sensing images and predefined degradation factors, corresponding degraded images are synthesized to form a training sample set; The degraded image is preprocessed, and the preprocessed degraded image is input into a pre-constructed restoration model to generate a predicted restored image; Calculate the loss function between the predicted restored image and the multi-source clear remote sensing image; Based on the loss function, the parameters of the restored model are updated using the backpropagation algorithm; The iteration continues until the preset stopping condition is met, at which point the trained restored model is obtained.

5. The method for restoring degraded remote sensing images as described in claim 4, characterized in that, The step involves synthesizing corresponding degraded images based on acquired multi-source clear remote sensing images and predefined degradation factors to form a training sample set, including: Acquire multi-source clear remote sensing images; the multi-source clear remote sensing images include clear remote sensing images of different land cover types; A physics-driven degradation model is used to apply at least one degradation factor to the multi-source clear remote sensing image to generate a corresponding degraded image; the degradation factors include low light, blur, haze, and noise; when multiple degradation factors are applied, the application order is low light, blur, haze, and noise. A training sample set is obtained based on the multi-source clear remote sensing images and the degraded images.

6. The method for restoring degraded remote sensing images as described in claim 5, characterized in that, The step of employing a physics-driven degradation model to apply at least one degradation factor to the multi-source clear remote sensing images to generate corresponding degraded images includes: When the applied degradation factor is low light, a power transformation is performed on each pixel value of the image to be degraded to obtain a first degraded image; the power exponent in the power transformation is randomly sampled within a preset range of darkness coefficient values. When the applied degradation factor is blurring, an anisotropic Gaussian kernel is used to perform two-dimensional filtering on the image to be degraded, resulting in a second degraded image. When the applied degradation factor is haze, haze synthesis is performed on the image to be degraded based on the atmospheric scattering model to obtain a third degraded image; When the applied degradation factor is noise, an additive Gaussian noise model is used to add noise to the image to be degraded, resulting in a fourth degraded image.

7. The method for restoring degraded remote sensing images as described in claim 4, characterized in that, The calculation of the loss function between the predicted restored image and the multi-source clear remote sensing image includes: Calculate the pixel-level difference in color space between the predicted restored image and the multi-source clear remote sensing image; Calculate the spectral difference between the predicted restored image and the multi-source sharp remote sensing image in the Fourier domain; Calculate the semantic difference between the predicted restored image and the multi-source sharp remote sensing image in the high-level semantic feature space; Based on preset weight parameters, the pixel-level differences, the spectral differences, and the semantic differences are weighted and fused to obtain a loss function.

8. A degraded remote sensing image restoration system, characterized in that, include: The shallow feature extraction module is used to input the acquired degraded remote sensing image into a pre-trained restoration model, and extract the shallow features of the degraded remote sensing image through the input embedding layer of the restoration model. The encoding module is used to perform frequency domain modulation and spatial domain modulation on the shallow features in the multi-level encoder of the restoration model through a dual-gated attention mechanism, and extract the deep features of the degraded remote sensing image step by step. The decoding module is used to perform parallel expert processing and adaptive weighted fusion on the deep features in the multi-level decoder of the restoration model through a heterogeneous expert collaboration module, and reconstruct the decoded features step by step. The heterogeneous expert collaboration module includes a first expert branch for capturing global context dependencies and a second expert branch for extracting directional edge features; The restoration result generation module is used to generate a restoration result image based on the shallow features and the decoded features; The shallow feature extraction module is specifically used for: Acquire degraded remote sensing images and normalize the pixel values ​​of the degraded remote sensing images; If the size of the degraded remote sensing image is larger than a preset size threshold, the degraded remote sensing image is divided into blocks using a sliding window strategy. The processed degraded remote sensing image is input into a pre-trained restoration model, and shallow features of the degraded remote sensing image are extracted through the input embedding layer of the restoration model. The decoding module is specifically used for: The deep features are input into the multi-level decoder of the restoration model; For each level of the decoder, the global contextual dependencies of the input features of the current level are captured through the first expert branch of the heterogeneous expert collaboration module, and the global semantic features are output. The first expert branch employs a dual-gated attention mechanism; The second expert branch of the heterogeneous expert collaboration module extracts the directional edge features of the input features of the current level; the second expert branch adopts a stacked structure of multi-scale depthwise separable convolution and a dynamic edge gating mechanism. The adaptive fusion unit of the heterogeneous expert collaboration module dynamically calculates the fusion weights of the first expert branch and the second expert branch based on the statistical characteristics of the input features of the current level. Based on the fusion weights, the global semantic features and the directional edge features are weighted and fused to obtain weighted fused features; The weighted fusion features are residually concatenated with the input features of the current level to obtain the output features of the current level; The output features of the current level are upsampled and then fused with the skip connection features passed from the corresponding level encoder to serve as the input features of the next level decoder. The output features of the last stage of the multi-level decoder are used as the decoding features of the degraded remote sensing image.

Images