A dual-domain joint remote sensing image high-fidelity super-resolution diffusion type reconstruction method

Abstract

This invention discloses a dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method. First, remote sensing image data is acquired and preprocessed. Based on a physical consistency degradation model, corresponding low-resolution observation samples are constructed from the high-resolution remote sensing image. Then, spatial domain condition information and frequency domain condition information are extracted from the low-resolution remote sensing image. The fusion ratio of spatial and frequency domain conditions is dynamically adjusted through a scene-adaptive gated fusion module (SGF) to obtain dual-domain joint condition features. Finally, the dual-domain joint condition features are dynamically injected into a diffusion reconstruction network. During the inverse diffusion process, noise is gradually reduced while restoring structural contours and high-frequency textures, outputting a high-resolution remote sensing image. This invention effectively solves the problems of disconnect between spatial structural constraints and frequency domain texture modeling, insufficient spectral constraints in diffusion methods, and the generation of false high-frequency textures in smooth regions in existing remote sensing super-resolution technologies.

Description

Technical Field

[0001] This invention relates to a remote sensing image reconstruction method, specifically a spatial-frequency dual-domain joint conditional diffusion remote sensing image super-resolution reconstruction method, belonging to the field of remote sensing image processing technology. Background Technology

[0002] With the continuous development of Earth observation and spatial information technology, remote sensing imagery has been widely applied in fields such as land resource surveys, ecological environment monitoring, refined urban governance, disaster assessment and emergency response, agricultural yield estimation, and water body change monitoring. These applications often require detailed identification and quantitative analysis of features such as roads, building outlines, riverbanks, farmland strip textures, and vegetation boundaries. These tasks place higher demands on the spatial resolution, edge sharpness, and texture fidelity of remote sensing imagery. Meanwhile, the acquisition of high-resolution remote sensing imagery is constrained by factors such as sensor hardware capabilities, spaceborne platform energy consumption and cost, imaging swath width and coverage efficiency, and orbit and revisit cycle, leading to a difficult trade-off between acquisition cost, timeliness, and coverage. In contrast, medium- and low-resolution remote sensing imagery typically has advantages such as large coverage, high acquisition frequency, and lower cost, but its lack of spatial detail, blurred edges, and missing texture information makes it difficult to meet the needs of refined interpretation and high-precision downstream tasks. Therefore, using image super-resolution (SR) reconstruction technology to recover high-resolution (HR) images from low-resolution (LR) remote sensing images at the algorithmic level has become one of the important research directions in remote sensing image processing. However, the super-resolution task of remote sensing images is different from that of general natural images. The types and distribution of land features in remote sensing images are significantly different from those in general natural images: land features such as roads, rivers, buildings, and forests differ in scale, shape, structural complexity, and texture periodicity. This means that the super-resolution model must not only recover detailed textures but also maintain the integrity of the geometric structure and the continuity of boundaries.

[0003] Existing remote sensing image super-resolution technology has roughly gone through a development process from traditional interpolation methods to deep learning methods, and then to blind super-resolution and generative methods that are closer to real imaging. Early traditional methods mainly used bilinear, bicubic, and other interpolation techniques to magnify images. These methods are essentially smooth interpolation and lack the ability to restore real high-frequency details, usually resulting in blurred edges and missing textures. After entering the deep learning stage, methods based on convolutional neural networks (CNNs) have made significant progress on public datasets by learning the mapping relationship from LR to HR. However, most of these methods take pixel-wise error as the main optimization objective, which tends to tend towards the average solution, resulting in over-smoothing of the results and difficulty in reconstructing discriminative high-frequency textures. More importantly, many CNN-like methods usually adopt idealized degradation assumptions during training (such as fixed blur kernels and simple noise), while the degradation of real remote sensing images is far more complex than ideal assumptions. It often involves multiple factors such as atmospheric scattering, sensor point spread function (PSF), platform jitter, strip noise, compression artifacts, and differences in response across different bands. Even with the same sensor, the degradation mode may differ for different surface scenes. Therefore, models trained under ideal degradation are prone to artifacts, structural distortion, and spectral drift on real data, resulting in insufficient generalization ability.

[0004] To overcome the applicability issues arising from fixed degradation assumptions, blind super-resolution (BSR) has gradually become an important direction in the field of remote sensing. Blind super-resolution methods typically no longer assume that the degradation process is known, but instead use kernel estimation, degradation feature encoding, or learnable degradation priors to enable the network to adapt to different degradation intensities and types. Although blind super-resolution methods are closer to the real imaging process to some extent, existing research mainly focuses on spatial domain blurring and noise variations, paying insufficient attention to issues such as cross-band spectral coupling degradation, inter-band energy ratios, and sensor response differences in remote sensing images. If only spatial sharpness is pursued during reconstruction while ignoring inter-band ratios, it may lead to spectral imbalance and false colors, and even cause significant deviations in the calculation of key remote sensing indices such as NDVI and NDWI, directly affecting the reliability of subsequent quantitative analysis. Therefore, relying solely on spatial domain blind super-resolution modeling is still insufficient to achieve "high-fidelity" reconstruction for real remote sensing applications. To improve global modeling capabilities, Transformer-based super-resolution methods have gradually emerged in recent years. Transformers have an advantage in capturing long-range dependencies due to their self-attention mechanism, which can improve structural consistency to some extent. However, existing Transformer methods still have significant shortcomings in remote sensing super-resolution scenarios: On the one hand, most methods rely on spatial domain attention mechanisms and do not make sufficient use of frequency domain statistical characteristics (such as the specific energy spectrum distribution of ground objects), making it difficult to balance high-frequency recovery and noise suppression; on the other hand, the training process is still often based on the ideal degradation assumption, and even with the introduction of blind super-resolution modules, it is often difficult to cope with the problems of cross-band energy imbalance, spectral coupling degradation, and sensor response differences that are common in real remote sensing imaging; in addition, the computational complexity of the self-attention mechanism is usually high, which makes the model face an efficiency bottleneck when processing high-resolution, large-format remote sensing images and high-magnification super-resolution tasks, limiting its engineering application capabilities.

[0005] To improve perceived quality and texture representation, some existing methods incorporate Generative Adversarial Networks (GANs). While GANs can enhance texture sharpness and visual "clarity," their training process is prone to instability, and in remote sensing scenarios, they are more likely to introduce pseudo-textures and artifacts that do not conform to the true distribution of ground features. Especially in large, smooth areas such as water bodies, skies, shadows, or deserts, the model may generate meaningless high-frequency noise or over-sharpening, disrupting the coherence of structural boundaries and the physical consistency of the image, further affecting the quantitative reliability of downstream tasks (such as exponential calculations or change detection).

[0006] In recent years, diffusion models (DMs), as an important branch of generative learning, have approximated complex data distributions through a Markov chain process of progressive noise addition and inverse denoising. They have demonstrated outstanding performance in training stability and generation quality, and have shown the potential to recover high-frequency details in super-resolution tasks. However, directly applying diffusion models to remote sensing super-resolution still faces key challenges: First, diffusion inference usually requires multiple iterative sampling steps, resulting in high computational costs and slow speed when inferring high-resolution, large-format remote sensing images, making it difficult to meet practical application needs. Second, most current diffusion-based super-resolution methods are still mainly guided by spatial domain conditions, lacking explicit modeling of the frequency domain characteristics of remote sensing images and the specific spectral structure of ground objects, leading to problems such as spectral shift, high-frequency artifacts, or uneven energy distribution in the reconstruction results. Third, insufficient modeling of real remote sensing degradation phenomena (especially cross-band spectral coupling and sensor response differences) makes it difficult to guarantee the spectral consistency of the generated results, thus affecting quantitative remote sensing analysis.

[0007] As mentioned above, a common and critical contradiction in existing technologies lies in the disconnect between spatial structure representation and frequency domain spectral / high-frequency texture modeling. The land cover structure in remote sensing images exhibits clear geometric regularities and spatial continuity, requiring super-resolution reconstruction to maintain boundary orientation, morphological proportions, and structural coherence. Simultaneously, the texture details of different land cover features exhibit specific energy distributions and spectral structures in the frequency domain. For example, urban buildings and roads often contain sharp edges and regular texture components, while farmland strips and vegetation cover areas have periodic or irregular fine texture components. If only spatial domain learning is relied upon, the model often struggles to effectively distinguish between true high-frequency details and sensor noise or aliasing artifacts. Furthermore, without frequency domain enhancement constrained by spatial structure, noise is easily misidentified as recoverable high-frequency signals, introducing "false high frequencies" in smooth areas (such as water bodies and clear skies), resulting in over-sharpening, checkerboard patterns, or ringing artifacts, thereby disrupting geometric consistency and causing spectral statistical shifts. Therefore, how to achieve collaborative modeling of spatial structure priors and frequency domain statistical priors in remote sensing super-resolution reconstruction, while improving detail fidelity and suppressing pseudo-textures and spectral shifts, and taking into account both geometric consistency and spectral consistency, remains a key technical problem that needs to be solved in existing remote sensing super-resolution technologies. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention provides a dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method that integrates spatial and frequency domain conditions. This method can effectively solve the problems of spatial structure constraints and frequency domain texture modeling separation in existing remote sensing super-resolution technologies, insufficient spectral constraints in diffusion methods, and the tendency to generate false high-frequency textures in smooth regions.

[0009] To achieve the above objectives, this dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method specifically includes the following steps:

[0010] Step 1, Remote sensing image data acquisition and preprocessing: Acquire remote sensing image data and perform preprocessing, including pixel value normalization, outlier removal and format conversion; Based on the physical consistency degradation model, construct corresponding low-resolution observation samples from high-resolution remote sensing images;

[0011] Step 2, Construction and adaptive fusion of spatial and frequency domain conditional information: The low-resolution remote sensing image is input into the conditional feature construction module, which extracts spatial and frequency domain conditional information from the low-resolution remote sensing image. The scene adaptive gating fusion module SGF dynamically adjusts the fusion ratio of spatial and frequency domain conditions according to different land cover types to obtain dual-domain joint conditional features.

[0012] Step 3, based on spatial-frequency domain condition-guided diffusion-based super-resolution reconstruction: The dual-domain joint conditional features are dynamically injected into the diffusion reconstruction network based on U-Net-style multi-scale Transformer. During the reverse diffusion process, noise is gradually reduced while restoring structural contours and high-frequency textures, and high-resolution remote sensing images are output.

[0013] Furthermore, the specific process of Step 2 is as follows:

[0014] Step 2-1, Construction of spatial structure condition vector: The spatial structure enhancement module SREM is used to model global geometric dependencies through the Transformer encoder, and a channel attention mechanism is introduced to generate spatial structure condition vector.

[0015] Step 2-2, Construction of the frequency-guided condition vector: The amplitude spectrum and phase spectrum of each scale are obtained by using the frequency-guided enhancement module FREM, and a learnable adaptive high-pass filter is introduced to enhance the high-frequency components. The enhanced amplitude spectrum and phase spectrum are fused to generate the frequency-guided condition vector.

[0016] Steps 2-3: The scene adaptive gating fusion module SGF dynamically adjusts the fusion ratio of spatial domain and frequency domain conditions to obtain dual-domain joint condition features.

[0017] Furthermore, the specific process of Step 2-1 is as follows:

[0018] Step 2-1-1, Global Geometry Modeling:

[0019] The multi-head self-attention mechanism is defined as follows:

[0020]

[0021]

[0022] In the formula: These are query, key, and value matrices, derived from the features of the previous layer. ; Indicates a splicing operation; For multi-head output linear mapping matrix; These are the query linear projection matrix, key linear projection matrix, and value linear projection matrix for each header, respectively. The dimension of the key vector;

[0023] Step 2-1-2, Channel Aggregation and Condition Generation:

[0024] A channel attention mechanism is introduced to apply dynamic modulation to the global feature map, as shown below:

[0025]

[0026]

[0027] In the formula: This represents the feature map after channel attention modulation; Indicates channel attention weights; This represents the Sigmoid activation function; This represents the parameters of the first fully connected layer. Indicates the parameters of the second fully connected layer; Indicates the activation function; Indicates global average pooling; Represents the global feature map; This indicates element-wise multiplication;

[0028] The final spatial structure condition vector is generated through two fully connected layers. , means as follows:

[0029]

[0030] In the formula: Represents the spatial structure condition vector; This represents the global average pooling function; and These represent the weight parameters of the first fully connected layer and the second fully connected layer, respectively. and These represent the bias parameters of the first fully connected layer and the second fully connected layer, respectively.

[0031] Furthermore, the specific process of Step 2-2 is as follows:

[0032] Step 2-2-1: Obtain the amplitude spectrum and phase spectrum by performing multi-scale block Fast Fourier Transform (FFT) on the low-resolution remote sensing image:

[0033] The image is divided into segments at different scales and FFT is performed on each segment, as shown below:

[0034]

[0035] In the formula: This indicates the input low-resolution image; Indicates the FFT block scale; and Represents the pixel coordinates of an image patch in the spatial domain; and Represents frequency domain coordinates; Representing scale Lower frequency coordinates The corresponding complex spectrum; Represents the natural constant; Represents the imaginary unit;

[0036] Obtain the amplitude spectrum at each scale Phase spectrum , means as follows:

[0037]

[0038]

[0039] In the formula: Representing scale Lower frequency coordinates The corresponding amplitude spectrum value; Representing scale Lower frequency coordinates The corresponding phase spectrum value; Represents the modulus operation of complex numbers; Represents the argument operation of complex numbers; The amplitude spectrum characterizes the energy distribution of different frequency components; The phase spectrum carries spatial structural information about its frequency components; Represents joint set operations of different scales;

[0040] Step 2-2-2: Introduce a learnable adaptive high-pass filter to enhance high-frequency components:

[0041] Design a learnable adaptive high-pass filter , to input image As a condition, dynamically generate the spectrum mask. Spectrum mask The amplitude spectrum is affected by the dot product operation. , means as follows:

[0042]

[0043] In the formula: This represents the enhanced amplitude spectrum; Indicates amplitude spectrum; Indicates input image A spectrum mask generated for the condition;

[0044] Introduction After normalization, it is represented as follows:

[0045]

[0046] In the formula: This represents the enhancement amplitude spectrum after normalization. The energy norm of the spectrum; To prevent the stable term from being divided by zero;

[0047] Step 2-2-3, keep the total energy of the spectrum before and after enhancement approximately the same, as shown below:

[0048]

[0049] In the formula: Represents the energy of the enhanced spectrum; The original energy representing the multi-scale spectrum;

[0050] Step 2-2-4: Fuse the enhanced amplitude spectrum and phase spectrum to generate a frequency-guided condition vector.

[0051] Enhanced amplitude spectrum Compared with the original phase spectrum After splicing, the data is fed into a multilayer perceptron (MLP) for encoding, as shown below:

[0052]

[0053] In the formula: Represents frequency domain structural characteristics; This represents the encoding operation of a multilayer perceptron; Indicates the enhanced amplitude spectrum Compared with the original phase spectrum To splice;

[0054] Generate the final frequency-guided condition vector , means as follows:

[0055]

[0056] In the formula: Represents the frequency-guided condition vector; The frequency representation is the overall mapping of the extraction module; This represents the weight parameters of the linear mapping layer; Indicates the bias parameters of the linear mapping layer; This represents the spatial domain structural features extracted from the input low-resolution image.

[0057] Furthermore, the specific processes of Steps 2-3 are as follows:

[0058] The scene adaptive gating module SGF, based on the cross-attention mechanism, is designed to incorporate spatial structural features. As a query, but based on frequency domain structural features Using keys and values, a frequency domain knowledge base is constructed, represented as follows:

[0059]

[0060] In the formula: For learnable query linear projection matrix; The learnable bond linear projection matrix; The linear projection matrix is ​​a learnable value; The dimension of the key vector is represented by its square root. Used as a scaling factor; This represents the cross-attention calculation process; Indicates spatial structural features; Indicates frequency structure characteristics; Indicates matrix transpose; Represents the normalization function;

[0061] A scene-adaptive gating function is introduced to adjust the fusion weights based on the scene texture complexity, as shown below:

[0062]

[0063] In the formula: Indicates the fusion weight. ; Represents the Sigmoid function; Indicates the action on spatial structural features The linear mapping weight parameters; Indicates the effect of frequency-guided condition features The linear mapping weight parameters; Indicates the bias parameter;

[0064] The final fused output is represented as follows:

[0065]

[0066] In the formula: This represents the frequency-guided condition vector after fusion; Indicated by spatial structural features For querying, based on frequency structure features The attention output is obtained by performing cross-attention calculation on the key and value.

[0067] Furthermore, the specific process of Step 3 is as follows:

[0068] Step 3-1, structural characterization encoder in the IRR-DiffSR framework Retraining is required;

[0069] Step 3-2, based on Step 3-1, redesigns the conditional mechanism of the diffusion reconstruction process, replacing the single spatial feature guidance in the baseline model with dual-domain joint conditional features; the diffusion model, through its inverse process, extracts data from random noise... Gradually recover the target image At each time step In this model, the denoising step is performed by the noise prediction function in FTRM-Net. Guided by dynamically injecting dual-domain joint conditional features at each denoising step. ;

[0070] Step 3-3: Utilize a high-frequency guided diffusion sampling mechanism to reconstruct low-resolution images into high-resolution images.

[0071] Furthermore, in Step 3-2, when dynamically injecting dual-domain joint conditional features, FTRM-Net introduces a Fourier branch as a plugin in the MTAB sublayer. First, it performs a Fast Fourier Transform on the input features to obtain the amplitude and phase spectra of the frequency domain features. Then, it learns the mapping function... A frequency domain token is generated, which is then dynamically concatenated with the dual-domain joint conditional features, as shown below:

[0072]

[0073] In the formula: This represents the key features obtained after dynamic splicing; This represents the characteristic of the value obtained after dynamic splicing; Indicates a splicing operation; Represents the spatial structure condition vector; Represents the frequency-guided condition vector; Represents the frequency domain mapping function The generated frequency domain token;

[0074] Based on this, the multi-head transpose attention calculation is represented as follows:

[0075]

[0076] In the formula: This represents the multi-head transpose attention calculation process; Indicates time Input features; This indicates a query for the linear projection matrix; Represents the linear projection matrix of the bond; Represents the linear projection matrix of values; The dimension of the key vector is represented by its square root. Used as a scaling factor; Indicates matrix transpose; Represents the normalization function;

[0077] FTRM-Net introduces a gated feedforward network GFFB into TFB, as shown below:

[0078]

[0079] In the formula: This represents a depthwise separable convolution operation; For example, an activation function (such as GELU); Sigmoid gate; Represents element-wise product;

[0080] The final fusion process is represented as follows:

[0081]

[0082] In the formula: This represents the output features after fusion of the transformed feature blocks (TFB). This represents the output characteristics of the multi-head transposed attention module; This represents the output characteristics of the gated feedforward network.

[0083] Furthermore, the specific process of Step 3-3 is as follows:

[0084] The model at time step The update process is represented as follows:

[0085]

[0086] In the formula: Indicates time step Denoising results of images at different times; Indicates time step Intermediate image of a moment; Indicates time step The corresponding diffusion coefficient; Represents the conditional noise prediction function; Indicates the current diffusion time step; " indicates a conditional constraint relationship;

[0087] Conditional noise prediction function The formal expression is as follows:

[0088]

[0089] In the formula: This is the standard unconditional noise prediction term; This is the spatial structure condition vector mapping function generated by the SREM module, responsible for providing global geometric and low-frequency distribution constraints; This is the spectral enhancement conditional vector mapping function generated by the FREM module, responsible for providing guidance for high-frequency textures and edge details; The weight of spatial conditions in noise prediction; This represents the weight of the frequency condition in noise prediction.

[0090] Compared with existing technologies, this dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method has the following advantages:

[0091] 1. To address the problem that existing remote sensing super-resolution methods rely solely on spatial domain features and struggle to accurately recover high-frequency textures, a frequency domain conditional modeling mechanism is introduced. By performing multi-scale frequency domain transformations on remote sensing images, the energy distribution and spectral structure characteristics of different frequency components are explicitly characterized. Combined with a learnable frequency enhancement strategy, high-frequency details are selectively modeled and constrained, thereby effectively improving texture recovery capabilities and avoiding misidentification of noise or aliasing components as real details.

[0092] 2. To address the issues of "false high frequencies" and over-sharpening in smooth regions (such as water bodies, sky, and deserts), a collaborative constraint mechanism of spatial and frequency domain conditions is proposed. Spatial structural conditions constrain the geometric contours and boundary continuity, while frequency domain statistical conditions limit the distribution of high-frequency energy. This allows the reconstruction process to enhance details while maintaining structural rationality, effectively suppressing the generation of false textures, ringing effects, and checkerboard artifacts.

[0093] 3. Abandoning the traditional approach of simply superimposing or splicing spatial and frequency domain features, we propose an adaptive spatial-frequency domain conditional fusion strategy. By introducing an attention mechanism and gating modulation, we dynamically adjust the contribution ratio of spatial and frequency domain information in the reconstruction process according to different land cover types and texture complexity. This enables the model to adaptively switch between structure-dominated and texture-dominated scenes, thereby achieving a balance between structural consistency and texture realism.

[0094] 4. A framework for super-resolution reconstruction of remote sensing images is constructed based on a conditional diffusion model, continuously injecting fused spatial-frequency domain conditional information into the multi-stage denoising process of the diffusion model. By guiding the generation process in a multi-scale feature space, the diffusion model is always subject to joint constraints of structure and spectrum while gradually denoising. This leverages the advantages of the diffusion model in detail generation while avoiding spectral shift and unstable generation issues in remote sensing scenes.

[0095] 5. By introducing a degradation modeling method that conforms to the characteristics of remote sensing imaging, the model's adaptability to the degradation of real remote sensing images is improved. The proposed method has good robustness and generalization ability under different sensors, different scenes, and different degradation intensities. Compared with existing methods based on convolutional neural networks or generative adversarial networks, this invention can achieve higher stability and consistency while ensuring reconstruction quality, and is more suitable for practical remote sensing applications and engineering deployment. Attached Figure Description

[0096] Figure 1 This is a technical flowchart of the present invention;

[0097] Figure 2 This is a schematic diagram of the overall framework of the present invention;

[0098] Figure 3 This is a schematic diagram of the Transformer fusion block in the diffusion reconstruction network of this invention;

[0099] Figure 4 This is a comparative schematic diagram of the super-resolution reconstruction before and after an embodiment of the present invention;

[0100] Figure 5 This is a schematic diagram comparing the super-resolution reconstruction results of the FreqDiffSR method of this invention with other methods in a typical mountain scene. (a) is a real scene input image without true high-resolution ground truth, (b) is a low-resolution image, (c) is the bicubic interpolation reconstruction result, (d) is the reconstruction result of the ESRGAN method, (e) is the reconstruction result of the SwinIR method, (f) is the reconstruction result of the DRSR method, (g) is the reconstruction result of the TransENet method, (h) is the reconstruction result of the IRR-DiffSR method, and (i) is the reconstruction result of the FreqDiffSR method.

[0101] Figure 6This is a schematic diagram comparing the super-resolution reconstruction results of the FreqDiffSR method of this invention with other methods in a typical farmland scene. (a) is a real scene input image without true high-resolution ground truth, (b) is a low-resolution image, (c) is the bicubic interpolation reconstruction result, (d) is the reconstruction result of the ESRGAN method, (e) is the reconstruction result of the SwinIR method, (f) is the reconstruction result of the DRSR method, (g) is the reconstruction result of the TransENet method, (h) is the reconstruction result of the IRR-DiffSR method, and (i) is the reconstruction result of the FreqDiffSR method.

[0102] Figure 7 This is a schematic diagram comparing the super-resolution reconstruction results of the FreqDiffSR method of this invention with other methods in a typical water scene. (a) is a real scene input image without true high-resolution ground truth, (b) is a low-resolution image, (c) is the bicubic interpolation reconstruction result, (d) is the reconstruction result of the ESRGAN method, (e) is the reconstruction result of the SwinIR method, (f) is the reconstruction result of the DRSR method, (g) is the reconstruction result of the TransENet method, (h) is the reconstruction result of the IRR-DiffSR method, and (i) is the reconstruction result of the FreqDiffSR method.

[0103] Figure 8 This is a schematic diagram comparing the super-resolution reconstruction results of the FreqDiffSR method of this invention with other methods in a typical building scene. (a) is a real scene input image without true high-resolution ground truth, (b) is a low-resolution image, (c) is the bicubic interpolation reconstruction result, (d) is the reconstruction result of the ESRGAN method, (e) is the reconstruction result of the SwinIR method, (f) is the reconstruction result of the DRSR method, (g) is the reconstruction result of the TransENet method, (h) is the reconstruction result of the IRR-DiffSR method, and (i) is the reconstruction result of the FreqDiffSR method. Detailed Implementation

[0104] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0105] like Figure 1 , Figure 2As shown, this dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method is a remote sensing image super-resolution conditional diffusion reconstruction method that integrates spatial domain and frequency domain conditions (defined as: Space-FreqDiffSR). It aims to solve the problems of spatial structure representation and frequency domain spectral band modeling being disconnected in existing remote sensing super-resolution diffusion models, and the difficulty in balancing geometric consistency and high-frequency texture restoration. By strengthening feature guidance in the diffusion process through dual-domain joint conditions, it achieves efficient and high-fidelity super-resolution reconstruction. After data preprocessing, a spatial-frequency dual-branch conditional representation is employed. The spatial branch SREM extracts global geometric and layout information, while the frequency branch FREM explicitly captures and enhances texture details in different frequency bands through multi-scale FFT and learnable high-pass filters. Scene-adaptive gated fusion module SGF performs cross-domain attention alignment and dynamic weighting to obtain scene-matched frequency-enhanced features. Finally, the dual-domain conditional vectors are injected into the diffusion backbone network FTRM-Net based on a U-Net-style multi-scale Transformer, continuously providing structural constraints and texture guidance at each scale of the anti-diffusion process, achieving high-fidelity super-resolution reconstruction of remote sensing images. The specific steps include:

[0106] Step 1, Remote sensing image data acquisition and preprocessing: Acquire remote sensing image data and perform preprocessing, including pixel value normalization, outlier removal and format conversion; Based on the physical consistency degradation model, construct corresponding low-resolution observation samples from high-resolution remote sensing images.

[0107] First, remote sensing image data of the target region or dataset is acquired as the input. High-resolution remote sensing images serve as reference images, used to construct supervised samples during model training or to evaluate reconstruction results; low-resolution remote sensing images serve as the objects to be reconstructed, used for actual super-resolution inference applications. To address the differences in radiometric scale, imaging conditions, and noise levels among remote sensing images from different sources, uniform data standardization processing is performed on the images. This includes pixel value normalization, outlier and invalid value removal, and necessary format conversion operations to eliminate differences in numerical units between data points and ensure the stability of subsequent model training and inference processes.

[0108] During the training phase, to enable the model to better adapt to the complex degradation phenomena in real remote sensing imaging processes, a degradation modeling approach based on physical consistency is adopted, constructing corresponding low-resolution observation samples from high-resolution remote sensing images. This degradation process comprehensively considers factors such as point spread function blurring, downsampling, and noise injection. By simulating the process of spatial resolution degradation and texture detail loss in the real remote sensing imaging chain, the generated low-resolution images maintain consistency with real remote sensing images in terms of statistical characteristics, spatial blurring degree, and high-frequency attenuation features. This effectively reduces the distribution difference between the training phase and actual application scenarios, providing a more reliable data foundation for subsequent model learning.

[0109] Step 2, Construction and adaptive fusion of spatial and frequency domain condition information: The low-resolution remote sensing image is input into the condition feature construction module, which extracts spatial and frequency domain condition information from the low-resolution remote sensing image. The spatial domain condition information is used to constrain the geometric structure and boundary continuity of ground objects, and the frequency domain condition information is used to model multi-scale high-frequency texture features. The scene adaptive gating fusion module SGF dynamically adjusts the fusion ratio of spatial and frequency domain conditions according to different ground object types to obtain dual-domain joint condition features.

[0110] After data preprocessing, the low-resolution remote sensing image is input into the conditional feature construction module, which extracts conditional information for super-resolution reconstruction from two complementary perspectives: the spatial domain and the frequency domain. The spatial domain conditional information focuses on characterizing the geometric structure, edge orientation, and overall layout of features in the remote sensing image, constraining the rationality of the structural morphology during reconstruction and preventing edge distortion or geometric discrepancies. The frequency domain conditional information analyzes the distribution characteristics of the remote sensing image in the spectral space, models different frequency components, and explicitly introduces prior information on texture details and high-frequency energy distribution to guide the reasonable compensation and restoration of high-frequency textures. Specifically:

[0111] Step 2-1, Construction of spatial structure condition vectors:

[0112] In the IRR (Intermediate Representation Refinement) structure, the basic spatial structure representation has been extracted through a high- and low-resolution dual-input (HR–LR) collaborative modeling mechanism. This feature effectively integrates high-resolution images. Geometric priors and low-resolution images The textural information is important. However, IRR mainly focuses on encoding mid-level local features, and still has shortcomings in terms of global geometric consistency and structural stability. To address this, we utilize SREM (Spatial-domain Representation Extraction Module), which is used to capture the overall geometry of an image to prevent structural drift, to further refine IRR features and enable them to generate structurally consistent and geometrically stable spatial structure condition vectors. Building upon this, SREM introduces a Transformer encoder and Channel Attention (CA) mechanism to achieve a closed-loop process from local feature enhancement to global geometric modeling to conditional vector generation, thereby providing learnable structural prior constraints in the diffusion model. Specifically:

[0113] Step 2-1-1, Global Structure Modeling:

[0114] Current convolutional operations have limitations in long-range dependency modeling, making it difficult to capture the global spatial relationships between features in remote sensing imagery. To fully explore the global geometric dependencies of a scene, a multi-layer Transformer encoder is introduced. This encoder uses a self-attention mechanism to model the relationships between distant pixels, enabling the model to understand both local texture and overall composition simultaneously.

[0115] The multi-head self-attention (MHA) mechanism is defined as follows:

[0116]

[0117]

[0118] In the formula: These are query, key, and value matrices, derived from the features of the previous layer. ; Indicates a splicing operation; For multi-head output linear mapping matrix; These are the query linear projection matrix, key linear projection matrix, and value linear projection matrix for each header, respectively. The dimension of the key vector is used for scaling to prevent gradient vanishing.

[0119] Its main function is to calculate a weighted average of each spatial location of the input features with respect to other global locations, thereby capturing long-range dependencies and preserving global geometric information.

[0120] Step 2-1-2, Channel Aggregation & Representation Generation:

[0121] To address the issues of uneven contribution of features from different channels to the final spatial structure conditions, and the potential for redundant information or noise in some channels, a Channel Attention (CA) mechanism is introduced to dynamically modulate the global feature map, as shown below:

[0122]

[0123]

[0124] In the formula: This represents the feature map after channel attention modulation; Indicates channel attention weights; This represents the Sigmoid activation function; This represents the parameters of the first fully connected layer. Indicates the parameters of the second fully connected layer; Indicates the activation function; Indicates global average pooling; Represents the global feature map; This indicates element-wise multiplication.

[0125] Feature map after channel attention modulation The final spatial structure condition vector is further generated through two fully connected layers. , means as follows:

[0126]

[0127] In the formula: Represents the spatial structure condition vector; This represents the global average pooling function; and These represent the weight parameters of the first fully connected layer and the second fully connected layer, respectively. and These represent the bias parameters of the first fully connected layer and the second fully connected layer, respectively.

[0128] Based on this, the generated It not only preserves global geometric information and low-frequency structural features, but also provides stable structural constraints, ensuring that the geometric shape is not distorted during denoising and super-resolution. This enables the SREM module to complete a closed loop from local enhancement to global modeling and then to condition vector generation, allowing the spatial structure condition vector to play a key guiding role in the diffusion model.

[0129] Step 2-2, Construction of the frequency-guided conditional vector:

[0130] Frequency-guided condition vector The design aims to overcome the limitations of spatial domain processing by providing accurate high-frequency detail priors for image reconstruction through frequency domain analysis. Its core principle is that high-frequency details in an image (such as edges and textures) manifest as specific high-frequency components in the frequency domain, while these components are often mixed with noise and difficult to separate in the spatial domain. The Frequency-Guided Enhancement (FREM) module establishes a parallel spatial-frequency dual-branch architecture. The spatial branch provides semantic guidance, while the frequency branch focuses on detail extraction. The two work together to generate high-quality frequency-guided conditions.

[0131] The mathematical expression for the Frequency Boot Enhancement Module (FREM) is as follows:

[0132]

[0133] In the formula: This represents the mapping function for the entire FREM module, which maps to a low-resolution image. As input, through dual-branch collaborative processing, the output is a conditional vector containing rich frequency priors. .

[0134] FREM employs a parallel dual-branch design. Since current frequency domain information requires spatial context for correct interpretation and utilization, frequency domain operations lacking semantic relevance may amplify noise or produce disjointed pseudo-textures. Therefore, the spatial branch acts as a "guide," providing the geometric basis and semantic context for the high-frequency information extracted by the frequency domain branch, ensuring that the subsequently recovered texture is spatially coherent and consistent with the image content. On the other hand, the frequency domain feature branch is the soul of FREM, its design embodying more refined and innovative signal processing concepts.

[0135] The specific process is as follows:

[0136] Step 2-2-1: Obtain the amplitude spectrum and phase spectrum by performing multi-scale block fast Fourier transform (FFT) on the low-resolution remote sensing image:

[0137] Because the global Fourier transform loses the spatial locality of frequency information, a multi-scale block-based Fast Fourier Transform (FFT) is introduced. This method divides the image into different scales (e.g., 16×16 pixel blocks, 32×32 pixel blocks) and performs FFT on each scale separately, as shown below:

[0138]

[0139] In the formula: This indicates the input low-resolution image; Indicates the FFT block scale; and Represents the pixel coordinates of an image patch in the spatial domain; and Represents frequency domain coordinates; Representing scale Lower frequency coordinates The corresponding complex spectrum; Represents the natural constant; It represents the imaginary unit.

[0140] The amplitude spectrum at each scale is obtained based on the Fast Fourier Transform (FFT). Phase spectrum , means as follows:

[0141]

[0142]

[0143] In the formula: Representing scale Lower frequency coordinates The corresponding amplitude spectrum value; Representing scale Lower frequency coordinates The corresponding phase spectrum value; Represents the modulus operation of complex numbers; Represents the argument operation of complex numbers; The amplitude spectrum characterizes the energy distribution of different frequency components; The phase spectrum carries spatial structural information about its frequency components; This represents a joint set operation of different scales.

[0144] This multi-scale analysis can simultaneously capture the frequency features corresponding to fine textures and medium-sized edge structures. The motivation for the multi-scale strategy is that textures in images exist within different scale ranges, and fine fabric textures and medium-sized edge structures correspond to different frequency intervals. Block processing can more accurately locate and capture these scale-related frequency features.

[0145] Step 2-2-2: Introduce a learnable adaptive high-pass filter to enhance high-frequency components:

[0146] After acquiring the spectrum, a crucial issue is how to separate the true high-frequency details beneficial to super-resolution from the complex spectrum, while suppressing noise and artifacts. Since traditional fixed high-pass filters (such as ideal high-pass and Butterworth high-pass filters) lack adaptability, it is necessary to design learnable adaptive high-pass filters. The filter is a parameterized neural network that can adapt the input image... A spectrum mask is dynamically generated based on the given conditions. This mask is applied to the amplitude spectrum through dot product operations. , means as follows:

[0147]

[0148] In the formula: This represents the enhanced amplitude spectrum; Indicates amplitude spectrum; Indicates input image A spectral mask generated for the conditions.

[0149] Essentially, it allows the model to autonomously learn from the data "where" and "to what extent" to enhance high-frequency components. For example, it can suppress high frequencies in smooth areas (such as the sky) to avoid noise amplification, while strongly enhancing high frequencies in textured areas (such as grass) to promote detail recovery.

[0150] At the same time, to avoid excessive concentration of spectral energy after enhancement, an introduction is made After normalization, it is represented as follows:

[0151]

[0152] In the formula: This represents the enhancement amplitude spectrum after normalization. The energy norm of the spectrum; To prevent the stable term from being divided by zero.

[0153] This normalization step can suppress excessive concentration of local energy in the spectrum and prevent gradient explosion or feature amplification during the training phase of the model.

[0154] Step 2-2-3: Keep the total energy of the spectrum approximately the same before and after enhancement:

[0155] After normalization, to further maintain approximately the same total energy in the spectrum before and after enhancement, as shown below:

[0156]

[0157] In the formula: Represents the energy of the enhanced spectrum; The original energy representing the multi-scale spectrum.

[0158] This energy conservation constraint ensures that enhancement operations only redistribute energy at the level of energy distribution without altering the overall energy, thus maintaining visual stability in brightness and contrast.

[0159] From a signal processing perspective, the total energy of the spectrum corresponds to the overall brightness and structural contrast of an image. When high-frequency energy is amplified uncontrollably, the model is prone to generating over-sharpened results or results with ringing artifacts. By combining spectral normalization and energy conservation constraints, the FREM module can enhance high-frequency details while maintaining the naturalness and physical consistency of the image.

[0160] Step 2-2-4: Fuse the enhanced amplitude spectrum and phase spectrum to generate a frequency-guided condition vector.

[0161] Another core dimension of frequency information is the phase spectrum. The phase spectrum carries structural location information about edges and contours in an image, which is crucial for visual perception. Therefore, the enhanced amplitude spectrum... Compared with the original phase spectrum The data is then concatenated. This joint spectral representation is subsequently fed into a multilayer perceptron (MLP) for encoding, as follows:

[0162]

[0163] In the formula: Represents frequency domain structural characteristics; This represents the encoding operation of a multilayer perceptron; Indicates the enhanced amplitude spectrum Compared with the original phase spectrum Then, the parts are assembled.

[0164] Multilayer perceptron (MLP) acts as a nonlinear mapping network to enhance the amplitude spectrum. Phase spectrum containing structural information Deep fusion is performed. Through this joint coding, the frequency domain features are obtained. It can fully preserve the frequency priors required to restore realistic textures.

[0165] Ultimately, the contextual features provided by spatial branches Frequency domain structural features extracted from frequency domain branches The features are concatenated and then fused and reduced in dimensionality through a fully connected layer to generate the final frequency-guided conditional vector. , means as follows:

[0166]

[0167] In the formula: This represents the frequency-guided condition vector, which serves as the frequency-guided condition input for the subsequent diffusion model. It combines structured semantics with learnable high-frequency modulation capabilities, providing robust priors for detail recovery for the generative model. The frequency representation is the overall mapping of the extraction module; This represents the weight parameters of the linear mapping layer; Indicates the bias parameters of the linear mapping layer; This represents the spatial domain structural features extracted from the input low-resolution image.

[0168] Steps 2-3: Design of the Scene Adaptive Gating Module (SGF):

[0169] Considering the significant differences in land cover types and texture complexity in different remote sensing scenarios, it is difficult to adapt to the diverse needs of scenarios by using only a fixed ratio of spatial domain and frequency domain information fusion. Therefore, an adaptive fusion mechanism is introduced, which dynamically adjusts the weight ratio of spatial structure constraints and frequency domain texture guidance in the fusion conditions according to the scene characteristics of the input image. This enables the model to effectively enhance detail recovery in texture-rich regions and suppress unrealistic high-frequency components in smooth regions, thereby achieving an adaptive balance between structural consistency and texture realism.

[0170] During the fusion phase, a scene adaptive gating module (SGF) based on cross-attention is designed to incorporate spatial structural features. As a query, frequency domain structural features Using keys and values, a frequency domain knowledge base is constructed, represented as follows:

[0171]

[0172] In the formula: For learnable query linear projection matrix; The learnable bond linear projection matrix; The linear projection matrix is ​​a learnable value; The dimension of the key vector is represented by its square root. Used as a scaling factor; This represents the cross-attention calculation process; Indicates spatial structural features; Indicates frequency structure characteristics; Indicates matrix transpose; This represents the normalization function used to generate attention weights.

[0173] Through this mapping, the SGF module can retrieve the most relevant high-frequency components for each spatial location from the frequency domain knowledge base, achieving local-global semantic alignment.

[0174] Because different terrain features have different frequency enhancement requirements (e.g., building edges require stronger high-frequency enhancement, while water areas should remain smooth), a scene-aware gate function is introduced to adjust the fusion weights according to the scene texture complexity, as shown below:

[0175]

[0176] In the formula: Indicates the fusion weight. ; Represents the Sigmoid function; Indicates the action on spatial structural features The linear mapping weight parameters; Indicates the effect of frequency-guided condition features The linear mapping weight parameters; This represents the bias parameter.

[0177] The scene-adaptive gating function learns and adaptively adjusts the fusion ratio of spatial and frequency features. When high-texture regions appear in the scene, Higher values ​​enhance frequency domain characteristics; while in smooth or homogeneous regions, The value is low to preserve structural consistency and suppress pseudo-textures.

[0178] The final fused output is represented as follows:

[0179]

[0180] In the formula: This represents the frequency-guided condition vector after fusion; Indicated by spatial structural features For querying, based on frequency structure features The attention output obtained by performing cross-attention calculation on keys and values; This indicates an element-wise multiplication operation.

[0181] Through this gating fusion strategy, the model can achieve a balance between frequency enhancement and spatial consistency. This frequency enhancement condition not only overcomes the limitations of traditional conditional modeling that relies on a single spatial feature, but also achieves a better balance between structural consistency and detail fidelity.

[0182] Step 3, based on spatial-frequency domain conditional guidance for diffusion-based super-resolution reconstruction: Dual-domain conditional guidance features are dynamically injected into a U-Net-based multi-scale Transformer-based diffusion reconstruction network to enhance feature guidance during the diffusion process. During the reverse diffusion process, noise is gradually reduced while restoring structural contours and high-frequency textures, resulting in a high-resolution remote sensing image.

[0183] After obtaining the fused spatial-frequency domain conditional information, a remote sensing image super-resolution reconstruction network with a diffusion model as its core is constructed. The diffusion model models complex image distributions by progressively adding noise to the high-resolution image during the forward process and learning the mapping relationship to gradually recover the image from the noisy state during the inverse process.

[0184] The diffusion reconstruction network employs FTRM-Net (Fourier-Integrated Reconstruction & Modeling Network), which includes an encoder, decoder, and skip connections, for noise prediction and feature reconstruction during the diffusion process. Its core unit is the Transformer Fusion Block (TFB), such as... Figure 3 As shown, each TFB includes a Gated Feed-Forward Block (GFFB) and a Multi-Head Transposed Attention Block (MTAB). The GFFB is responsible for aggregating local features and adaptively adjusting the representation through a gating strategy, while the MTAB utilizes the multi-head transposed attention mechanism to achieve global modeling and conditional fusion. Fourier branches are introduced into the MTAB to process frequency domain features, and joint conditional features from both domains are dynamically injected.

[0185] The inverse diffusion process introduces a noise prediction function that integrates spatial structure constraints and spectral enhancement information at each time step, controlling the balanced fusion of dual-domain conditions through weight parameters. During inverse diffusion, the fused spatial and frequency domain condition information is continuously injected as external priors into multiple time steps of the diffusion model, jointly constraining the direction and amplitude of the denoising process. By introducing dual guidance from structure and spectrum throughout the denoising process, the diffusion model can gradually generate high-resolution images while restoring clear and natural texture details, maintaining the geometric consistency of ground features and the rationality of spectral distribution, thus avoiding problems such as pseudo-high frequencies, ringing effects, or structural drift in remote sensing scenes. Ultimately, the diffusion model outputs remote sensing super-resolution reconstruction results with high spatial resolution and visual realism.

[0186] Specifically:

[0187] Step 3-1: First, the structural representation encoder in the IRR-DiffSR (image reconstruction representation-diffusion model for super-resolution) framework is processed. Retraining was performed to adapt to the remote sensing dataset and scene features used. In terms of model structure, some basic modules (such as convolutional units, normalization layers, and attention modules) were replaced with equivalent modules according to the experimental environment to improve training stability and computational efficiency, but the overall network structure and optimization objectives remained unchanged.

[0188] Step 3-2: Building upon Step 3-1, to inject joint conditional features across the two domains, FTRM-Net introduces a Fourier branch as a plugin in the MTAB sublayer. First, a Fast Fourier Transform is performed on the input features to obtain the amplitude and phase spectra of the frequency domain features. This is achieved by learning the mapping function. A frequency domain token is generated, which is then dynamically concatenated with the dual-domain conditions, as shown below:

[0189]

[0190] In the formula: This represents the key features obtained after dynamic splicing; This represents the characteristic of the value obtained after dynamic splicing; Indicates a splicing operation; Represents the spatial structure condition vector; Represents the frequency-guided condition vector; Represents the frequency domain mapping function The generated frequency domain token.

[0191] Based on this, the multi-head transpose attention calculation can be represented as follows:

[0192]

[0193] In the formula: This represents the multi-head transpose attention calculation process; Indicates time Input features; This indicates a query for the linear projection matrix; Represents the linear projection matrix of the bond; Represents the linear projection matrix of values; The dimension of the key vector is represented by its square root. Used as a scaling factor; Indicates matrix transpose; This represents the normalization function.

[0194] The MTAB module performs cross-domain attention allocation under the constraints of frequency domain information, enabling high-frequency details to be adaptively enhanced in the global context.

[0195] To suppress potential artifacts during frequency enhancement and maintain spatial consistency, FTRM-Net introduces a gated feedforward network GFFB into the TFB, as follows:

[0196]

[0197] In the formula: This represents a depthwise separable convolution operation; For example, an activation function (such as GELU); Sigmoid gate; This represents element-wise product.

[0198] This structure can adaptively adjust the response intensity of the feature channels, highlighting significant textures and suppressing noise components.

[0199] The final fusion process can be represented as follows:

[0200]

[0201] In the formula: This represents the output features after fusion of the transformed feature blocks (TFB). This represents the output characteristics of the multi-head transposed attention module; This represents the output characteristics of the gated feedforward network.

[0202] Finally, FTRM-Net dynamically balances spatial consistency and high-frequency detail recovery, significantly improving the high-frequency modeling capability of the decoding path. To achieve efficient noise prediction, FTRM-Net incorporates depthwise separable convolution (DWConv) and simple channel attention into TFB to reduce the number of parameters and maintain high performance.

[0203] Step 3-3, High-frequency guided high-efficiency diffusion sampling:

[0204] A high-frequency guided diffusion sampling mechanism is used to reconstruct low-resolution images from high-resolution images. The core idea is: at each denoising time step... FTRM-Net is based on spatial structure condition vectors With the spectrum guiding condition vector Together, they guide noise prediction, thereby simultaneously recovering global geometry and local detail textures during the reverse diffusion process. Specifically:

[0205] The model at time step The update process can be represented as follows:

[0206]

[0207] In the formula: Indicates the current diffusion time step; Indicates time step Denoising results of images at different times; Indicates time step Intermediate image of a moment; Indicates time step The corresponding diffusion coefficient; This represents the conditional noise prediction function, used in spatial structure condition vectors. and frequency-guided condition vector It predicts noise components in the current intermediate image under constraints, integrating prior information from both spatial and frequency domains; "" indicates a conditional constraint relationship.

[0208] Compared to the standard diffusion model, the conditional noise prediction function here... In addition to considering the current intermediate image, spatial structure constraints and frequency enhancement information are explicitly introduced, and its formal expression is as follows:

[0209]

[0210] In the formula: This is the standard unconditional noise prediction term; This is the spatial structure condition vector mapping function generated by the SREM module, which is responsible for providing global geometric and low-frequency distribution constraints; This is the spectral enhancement conditional vector mapping function generated by the FREM module, which is responsible for providing guidance for high-frequency textures and edge details; The weight of spatial conditions in noise prediction. The frequency condition is assigned a weight in noise prediction to ensure a balanced fusion of information from both domains.

[0211] To achieve effective injection of dual-domain conditions, FTRM-Net also introduces a cross-attention mechanism within each Transformer Fusion Block (TFB). This allows each denoising time step to not only obtain global geometric and low-frequency constraints but also to explicitly recover texture details, thereby significantly improving the stability of the sampling process and the high fidelity of image reconstruction. In actual reconstruction, and Through multi-layer fusion of TFB, the degree of feature enhancement in different regions is dynamically adjusted. For example, high-frequency structures such as building edges and road textures receive stronger frequency enhancement, while flat water bodies or sky areas mainly rely on spatial structure constraints to avoid noise amplification.

[0212] In practical applications of super-resolution reconstruction of remote sensing images, low-resolution remote sensing images are often affected by the resolution limitations of the imaging system and the actual degradation process, resulting in problems such as blurred edges, loss of high-frequency textures, and inconsistent degradation levels in different land cover areas. To address these typical application scenarios, this dual-domain joint high-fidelity super-resolution diffusion reconstruction method for remote sensing images is employed to process low-resolution remote sensing images. In the implementation process, spatial domain condition information is first constructed based on the low-resolution remote sensing image to constrain the structure and geometry of land cover; simultaneously, frequency domain condition information is constructed to model the distribution characteristics of different frequency components in the image and guide the reasonable restoration of high-frequency details; subsequently, the fused spatial-frequency domain condition information is continuously injected into the reverse diffusion process of the diffusion model to jointly guide the denoising direction, gradually generating a high-resolution remote sensing image. A comparative illustration of the super-resolution reconstruction results of the FreqDiffSR method in typical mountain scenes, typical farmland scenes, typical water body scenes, and typical building scenes with other methods (bicubic interpolation, ESRGAN, SwinIR, DRSR, TransENet, and IRR-DiffSR) is shown below. Figures 4 to 8 As shown.

[0213] The results show that the high-fidelity super-resolution diffusion reconstruction method using dual-domain joint remote sensing images generates stable super-resolution remote sensing images in terms of feature boundary continuity, overall structural consistency, and texture naturalness. In areas with significant structures such as buildings and roads, the reconstruction results can maintain geometric shape and edge sharpness well; in smooth areas such as water bodies and the sky, the introduction of unrealistic high-frequency components can be effectively suppressed, and no obvious pseudo-textures or ringing phenomena appear. Compared with methods that rely solely on spatial domain information or simple texture enhancement, this dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method achieves a better balance between structural constraints and detail restoration.

[0214] Furthermore, the proposed dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method was implemented under different scene types and degradation conditions, and stable reconstruction results were obtained. This indicates that the proposed dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method has good adaptability to real remote sensing image degradation. Even when high-frequency information is severely lacking or texture distribution is complex, it can still achieve reasonable super-resolution reconstruction through the synergistic guidance of spatial-frequency domain conditions.

[0215] As can be seen from the above implementation, this dual-domain joint remote sensing image high-fidelity super-resolution diffusion reconstruction method can effectively solve the problems of difficulty in balancing structure and texture and easy generation of pseudo-high frequencies in the existing technology. It has good stability and practicality and is suitable for a variety of remote sensing image super-resolution application scenarios.

Claims

1. A dual-domain joint remote sensing image high-fidelity super-resolution diffusion type reconstruction method, characterized in that, Specifically, the following steps are included: Step 1, Remote sensing image data acquisition and preprocessing: Acquire remote sensing image data and perform preprocessing, including pixel value normalization, outlier removal and format conversion; Based on the physical consistency degradation model, construct corresponding low-resolution observation samples from high-resolution remote sensing images; Step 2, Construction and adaptive fusion of spatial and frequency domain conditional information: The low-resolution remote sensing image is input into the conditional feature construction module, which extracts spatial and frequency domain conditional information from the low-resolution remote sensing image. The scene adaptive gating fusion module SGF dynamically adjusts the fusion ratio of spatial and frequency domain conditions according to different land cover types to obtain dual-domain joint conditional features. Step 3, based on spatial-frequency domain condition-guided diffusion-based super-resolution reconstruction: The dual-domain joint conditional features are dynamically injected into the diffusion reconstruction network based on U-Net-style multi-scale Transformer. During the reverse diffusion process, noise is gradually reduced while restoring structural contours and high-frequency textures, and high-resolution remote sensing images are output.

2. The high-fidelity super-resolution diffusion reconstruction method for dual-domain joint remote sensing images according to claim 1, characterized in that, Step 2 is performed as follows: Step 2-1, Construction of spatial structure condition vector: The spatial structure enhancement module SREM is used to model global geometric dependencies through the Transformer encoder, and a channel attention mechanism is introduced to generate spatial structure condition vector. Step 2-2, Construction of the frequency-guided condition vector: The amplitude spectrum and phase spectrum of each scale are obtained by using the frequency-guided enhancement module FREM, and a learnable adaptive high-pass filter is introduced to enhance the high-frequency components. The enhanced amplitude spectrum and phase spectrum are fused to generate the frequency-guided condition vector. Steps 2-3: The scene adaptive gating fusion module SGF dynamically adjusts the fusion ratio of spatial domain and frequency domain conditions to obtain dual-domain joint condition features.

3. The high-fidelity super-resolution diffusion reconstruction method for dual-domain joint remote sensing images according to claim 2, characterized in that, Step 2-1 The specific process is as follows: Step 2-1-1, Global Geometry Modeling: The multi-head self-attention mechanism is defined as follows: In the formula: These are query, key, and value matrices, derived from the features of the previous layer. ; Indicates a splicing operation; For multi-head output linear mapping matrix; These are the query linear projection matrix, key linear projection matrix, and value linear projection matrix for each header, respectively. The dimension of the key vector; Step 2-1-2, Channel Aggregation and Condition Generation: A channel attention mechanism is introduced to apply dynamic modulation to the global feature map, as shown below: In the formula: This represents the feature map after channel attention modulation; Indicates channel attention weights; This represents the Sigmoid activation function; This represents the parameters of the first fully connected layer. Indicates the parameters of the second fully connected layer; Indicates the activation function; Indicates global average pooling; Represents the global feature map; This indicates element-wise multiplication; The final spatial structure condition vector is generated through two fully connected layers. , means as follows: In the formula: Represents the spatial structure condition vector; This represents the global average pooling function; and These represent the weight parameters of the first fully connected layer and the second fully connected layer, respectively. and These represent the bias parameters of the first fully connected layer and the second fully connected layer, respectively.

4. The high-fidelity super-resolution diffusion reconstruction method for dual-domain joint remote sensing images according to claim 3, characterized in that, Step 2-2 The specific process is as follows: Step 2-2-1: Obtain the amplitude spectrum and phase spectrum by performing multi-scale block Fast Fourier Transform (FFT) on the low-resolution remote sensing image: The image is divided into segments at different scales and FFT is performed on each segment, as shown below: In the formula: This indicates the input low-resolution image; Indicates the FFT block scale; and Represents the pixel coordinates of an image patch in the spatial domain; and Represents frequency domain coordinates; Representing scale Lower frequency coordinates The corresponding complex spectrum; Represents the natural constant; Represents the imaginary unit; Obtain the amplitude spectrum at each scale Phase spectrum , means as follows: In the formula: Representing scale Lower frequency coordinates The corresponding amplitude spectrum value; Representing scale Lower frequency coordinates The corresponding phase spectrum value; Represents the modulus operation of complex numbers; Represents the argument operation of complex numbers; The amplitude spectrum characterizes the energy distribution of different frequency components; The phase spectrum carries spatial structural information about its frequency components; Represents joint set operations of different scales; Step 2-2-2: Introduce a learnable adaptive high-pass filter to enhance high-frequency components: Design a learnable adaptive high-pass filter , to input image As a condition, dynamically generate the spectrum mask. Spectrum mask The amplitude spectrum is affected by the dot product operation. , means as follows: In the formula: This represents the enhanced amplitude spectrum; Indicates amplitude spectrum; Indicates input image A spectrum mask generated for the condition; Introduction After normalization, it is represented as follows: In the formula: This represents the enhancement amplitude spectrum after normalization. The energy norm of the spectrum; To prevent the stable term from being divided by zero; Step 2-2-3, keep the total energy of the spectrum before and after enhancement approximately the same, as shown below: In the formula: Represents the energy of the enhanced spectrum; The original energy representing the multi-scale spectrum; Step 2-2-4: Fuse the enhanced amplitude spectrum and phase spectrum to generate a frequency-guided condition vector. Enhanced amplitude spectrum Compared with the original phase spectrum After splicing, the data is fed into a multilayer perceptron (MLP) for encoding, as shown below: In the formula: Represents frequency domain structural characteristics; This represents the encoding operation of a multilayer perceptron; Indicates the enhanced amplitude spectrum Compared with the original phase spectrum To splice; Generate the final frequency-guided condition vector , means as follows: In the formula: Represents the frequency-guided condition vector; The frequency representation is the overall mapping of the extraction module; This represents the weight parameters of the linear mapping layer; Indicates the bias parameters of the linear mapping layer; This represents the spatial domain structural features extracted from the input low-resolution image.

5. The high-fidelity super-resolution diffusion reconstruction method for dual-domain joint remote sensing images according to claim 4, characterized in that, The specific process for Steps 2-3 is as follows: The scene adaptive gating module SGF, based on the cross-attention mechanism, is designed to incorporate spatial structural features. As a query, but based on frequency domain structural features Using keys and values, a frequency domain knowledge base is constructed, represented as follows: In the formula: For learnable query linear projection matrix; The learnable bond linear projection matrix; The linear projection matrix is ​​a learnable value; The dimension of the key vector is represented by its square root. Used as a scaling factor; This represents the cross-attention calculation process; Indicates spatial structural features; Indicates frequency structure characteristics; Indicates matrix transpose; This represents the normalization function used to generate attention weights; A scene-adaptive gating function is introduced to adjust the fusion weights based on the scene texture complexity, as shown below: In the formula: Indicates the fusion weight. ; Represents the Sigmoid function; Indicates the action on spatial structural features The linear mapping weight parameters; Indicates the effect of frequency-guided condition features The linear mapping weight parameters; Indicates the bias parameter; The final fused output is represented as follows: In the formula: This represents the frequency-guided condition vector after fusion; Indicated by spatial structural features For querying, based on frequency structure features The attention output is obtained by performing cross-attention calculation on the key and value.

6. The high-fidelity super-resolution diffusion reconstruction method for dual-domain joint remote sensing images according to claim 1, characterized in that, Step 3 involves the following steps: Step 3-1, structural characterization encoder in the IRR-DiffSR framework Retraining is required; Step 3-2, based on Step 3-1, redesigns the conditional mechanism of the diffusion reconstruction process, replacing the single spatial feature guidance in the baseline model with dual-domain joint conditional features; the diffusion model, through its inverse process, extracts data from random noise... Gradually recover the target image At each time step In this model, the denoising step is performed by the noise prediction function in FTRM-Net. Guided by dynamically injecting dual-domain joint conditional features at each denoising step. ; Step 3-3: Utilize a high-frequency guided diffusion sampling mechanism to reconstruct low-resolution images into high-resolution images.

7. The high-fidelity super-resolution diffusion reconstruction method for dual-domain joint remote sensing images according to claim 6, characterized in that, In Step 3-2, when dynamically injecting dual-domain joint conditional features, FTRM-Net introduces a Fourier branch as a plugin in the MTAB sublayer. First, it performs a Fast Fourier Transform on the input features to obtain the amplitude and phase spectra of the frequency domain features. Then, it learns the mapping function... A frequency domain token is generated, which is then dynamically concatenated with the dual-domain joint conditional features, as shown below: In the formula: This represents the key features obtained after dynamic splicing; This represents the characteristic of the value obtained after dynamic splicing; Indicates a splicing operation; Represents the spatial structure condition vector; Represents the frequency-guided condition vector; Represents the frequency domain mapping function The generated frequency domain token; Based on this, the multi-head transpose attention calculation is represented as follows: In the formula: This represents the multi-head transpose attention calculation process; Indicates time Input features; This indicates a query for the linear projection matrix; Represents the linear projection matrix of the bond; Represents the linear projection matrix of values; The dimension of the key vector is represented by its square root. Used as a scaling factor; Indicates matrix transpose; Represents the normalization function; FTRM-Net introduces a gated feedforward network GFFB into TFB, as shown below: In the formula: This represents a depthwise separable convolution operation; For activation functions; Sigmoid gate; Represents element-wise product; The final fusion process is represented as follows: In the formula: This represents the output features after fusion of the transformed feature blocks (TFB). This represents the output characteristics of the multi-head transposed attention module; This represents the output characteristics of the gated feedforward network.

8. The high-fidelity super-resolution diffusion reconstruction method for dual-domain joint remote sensing images according to claim 7, characterized in that, Step 3-3 The specific process is as follows: The model at time step The update process is represented as follows: In the formula: Indicates time step Denoising results of images at different times; Indicates time step Intermediate image of a moment; Indicates time step The corresponding diffusion coefficient; Represents the conditional noise prediction function; Indicates the current diffusion time step; " indicates a conditional constraint relationship; Conditional noise prediction function The formal expression is as follows: In the formula: This is the standard unconditional noise prediction term; This is the spatial structure condition vector mapping function generated by the SREM module, responsible for providing global geometric and low-frequency distribution constraints; This is the spectral enhancement conditional vector mapping function generated by the FREM module, responsible for providing guidance for high-frequency textures and edge details; The weight of spatial conditions in noise prediction; This represents the weight of the frequency condition in noise prediction.

Images