Image super-resolution reconstruction method and device for fusing image restoration and rapid diffusion

Abstract

The invention discloses an image super-resolution reconstruction method and device fusing image restoration and fast diffusion, and the method comprises the steps: constructing a diffusion model, inputting a high-resolution remote sensing image into a forward Markov chain, adding noise, and generating a low-resolution noise graph aligned with a low-resolution remote sensing image; inputting the low-resolution remote sensing image into a reverse Markov chain based on Swindow-UNet for denoising to generate a high-resolution remote sensing image; adopting a fast diffusion mechanism based on high-level feature skipping and multiplexing to accelerate reasoning in the Swinin-UNet; and restoring a new low-resolution remote sensing image by using an image restoration preprocessing module based on a degradation kernel, and inputting the restored low-resolution remote sensing image into the trained reverse Markov chain based on the Swindow-UNet for remote sensing image super-resolution reconstruction. The method can effectively improve the image resolution, gives consideration to the processing efficiency, and can be widely applied to the fields of satellite image analysis, geographic information systems, environment monitoring and the like.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to an image super-resolution reconstruction method and apparatus that integrates image restoration and rapid diffusion. Background Technology

[0002] Remote sensing technology acquires information about the Earth's surface through sensors and is widely used in agriculture, urban planning, resource management, and environmental monitoring. However, due to limitations in equipment and data transmission costs, the spatial resolution of actually acquired remote sensing images is often low, making it difficult to meet practical operational needs such as high-precision surface identification, fine-grained target detection, and quantitative inversion, thus limiting the in-depth application of remote sensing data to some extent. Image super-resolution reconstruction technology aims to recover high-resolution images from low-resolution images through algorithmic models, becoming a key technical means to compensate for hardware deficiencies and improve image usability.

[0003] Traditional super-resolution reconstruction methods are mostly based on interpolation or sparse representations. These methods are computationally simple and highly interpretable, but they have limited ability to recover high-frequency details such as texture and edges, and are prone to blurring and distortion, resulting in only a small improvement in visual quality. In recent years, with the rapid development of deep learning technology, existing super-resolution reconstruction methods mainly rely on deep learning models, with mainstream models including convolutional neural networks (CNNs), generative adversarial networks (GANs), and diffusion models. Among them, CNN-based methods have good feature extraction capabilities, but often suffer from overly smoothed reconstructed images and loss of high-frequency details due to the locality of convolution operations and the limitations of loss functions. While GAN-based methods can generate images with high visual perception quality and rich details, their training process is unstable and prone to problems such as mode collapse, affecting the robustness and generalization ability of the model.

[0004] In contrast, diffusion models, as an emerging generative approach, have shown great potential in image super-resolution reconstruction tasks. Diffusion models, by defining forward denoising and backward denoising processes, progressively restore image details, exhibiting excellent performance in terms of structural consistency and detail realism. Furthermore, their training process is relatively stable, making them promising. However, typical denoising diffusion probabilistic models (DDPM) require hundreds or even thousands of iterations to reconstruct an image, resulting in slow inference speeds and failing to meet the real-time or near-real-time response requirements of large-scale data in remote sensing image processing. While some diffusion model-based super-resolution reconstruction methods (such as SR3) have achieved breakthroughs in image quality, they still suffer from problems such as large model parameter counts, high memory consumption, and significant computational resource consumption. Moreover, these methods still have room for improvement in terms of the visual realism, edge sharpness, and semantic consistency of the generated images.

[0005] Therefore, in order to address the imbalance between efficiency and quality that is common in existing super-resolution reconstruction methods in remote sensing image processing, there is an urgent need to develop a new super-resolution method that can achieve higher quality reconstruction and better perception results while maintaining training stability and lower resource consumption, and is applicable to large-scale remote sensing image processing. Summary of the Invention

[0006] In view of the above, the purpose of this invention is to provide an image super-resolution reconstruction method and apparatus that integrates image restoration and fast diffusion. It combines forward and backward Markov chains to construct the diffusion process, uses Swin-UNet as the noise prediction network, and introduces an image restoration preprocessing module based on a degenerate kernel and a fast diffusion mechanism based on high-level feature skipping and reuse. This can effectively restore remote sensing image details and improve the quality of reconstructed images while reducing the number of inference steps and increasing the inference speed, thereby achieving real-time and efficient remote sensing image super-resolution reconstruction.

[0007] To achieve the above-mentioned objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides an image super-resolution reconstruction method that integrates image restoration and rapid diffusion, comprising the following steps: A diffusion model is constructed. A forward Markov chain is input to a high-resolution remote sensing image to generate a low-resolution noise map that is aligned with the low-resolution remote sensing image. The low-resolution remote sensing image is used as a conditional input to a reverse Markov chain based on Swin-UNet to denoise the low-resolution noise map and generate a reconstructed high-resolution remote sensing image. In the Swin-UNet diffusion inference process, a fast diffusion mechanism based on high-level feature skipping and reuse is adopted to cache the high-level features of the previous decoder. In the next step, the low-level features of the encoder are fused with the cached high-level features to skip the calculation of the high-level features of the next decoder. The diffusion model was trained using a constructed dataset of remote sensing images containing high-resolution and low-resolution remote sensing image pairs. New low-resolution remote sensing images were restored using a degenerate kernel-based image restoration preprocessing module and then input into the Swin-UNet-based inverse Markov chain in the trained diffusion model for super-resolution reconstruction.

[0008] Preferably, the step of inputting the high-resolution remote sensing image into a forward Markov chain to add noise and generate a low-resolution noise map aligned with the low-resolution remote sensing image includes: From the input high-resolution remote sensing image Begin, at each time step According to conditional distribution Noise is gradually added, among which For the noise addition process noise map of the step, To and The corresponding low-resolution remote sensing image, Indicates the distance between them. Preset for controlling different time steps The noise level noise scheduling parameters, It is the identity matrix; when adding the common After the noise step, make the first Low-resolution noise map of the step Approximate to To achieve alignment.

[0009] Preferably, the step of using a low-resolution remote sensing image as a conditional input to a Swin-UNet-based inverse Markov chain to denoise the low-resolution noisy image and generate a reconstructed high-resolution remote sensing image includes: From low-resolution noise map Begin by distributing according to the conditions. Gradually remove noise, among which For the noise reduction process noise map of the step, and These represent the mean and covariance of the noise during each noise addition step; during inference, each step is based on the noise map. Low-resolution remote sensing images Current step Three parameters predict the noise that needs to be removed in the current step, and the low-resolution noise map is gradually reconstructed into a high-resolution remote sensing image through a reverse Markov chain based on Swin-UNet.

[0010] Preferably, the Swin-UNet network adopts an encoder-decoder structure, wherein the encoder performs multi-scale feature extraction and downsampling through alternating multi-level Swin Transformer modules and Patch merging layers, and the decoder performs upsampling through alternating multi-level Swin Transformer modules and Patch expansion layers. Skip connections are introduced between the Swin Transformer modules of each layer in the encoder and decoder, and the last layer of the encoder and the first layer of the decoder are connected through a bottleneck constructed by the multi-level Swin Transformer modules.

[0011] Preferably, the fast diffusion mechanism based on high-level feature skipping and reuse is used to cache the high-level features of the previous decoder. In the next step, the low-level features of the encoder are fused with the cached high-level features to skip the calculation of high-level features in the next decoder, including: In the reverse denoising process The step is to cache the high-level features of the decoder output of the Swin-UNet; In the reverse denoising process Step, the first The high-level features cached in the first step and the second step In the first step, the low-level features output by the Swin-UNet encoder are fused to skip the second step. The high-level feature calculation process of the step decoder enables high-level feature reuse and accelerates the inference process.

[0012] Preferably, the step of restoring the new low-resolution remote sensing image using an image restoration preprocessing module based on a degradation kernel includes: A dual-stream feature extraction network and a frequency-domain Wiener filtering module are used to predict the degradation kernel of low-resolution remote sensing images. The predicted degradation kernel and the low-resolution remote sensing image are then input into a deep Wiener deconvolution network to obtain the restored image, thus completing the preprocessing.

[0013] Preferably, the process of constructing a remote sensing image dataset containing pairs of high-resolution and low-resolution remote sensing images includes: High-resolution remote sensing images captured by satellites are acquired and bicubic interpolation downsampling is performed on them to obtain corresponding low-resolution remote sensing images. Several image pairs containing high-resolution and low-resolution remote sensing images are then constructed into a remote sensing image dataset.

[0014] Secondly, embodiments of the present invention also provide an image super-resolution reconstruction apparatus that integrates image restoration and fast diffusion, implemented using the above-mentioned image super-resolution reconstruction method that integrates image restoration and fast diffusion, including: a diffusion model modeling module, a fast diffusion inference module, and a model training and deployment module; The diffusion modeling module is used to construct a diffusion model. It inputs a high-resolution remote sensing image into a forward Markov chain to add noise and generate a low-resolution noise map that is aligned with the low-resolution remote sensing image. It then uses the low-resolution remote sensing image as a conditional input to a Swin-UNet-based inverse Markov chain to denoise the low-resolution noise map and generate a reconstructed high-resolution remote sensing image. The fast diffusion inference module is used in the Swin-UNet diffusion inference process to adopt a fast diffusion mechanism based on high-level feature skipping and reuse. It is used to cache the high-level features of the previous decoder and fuse the low-level features of the encoder with the cached high-level features in the next step to skip the calculation of the high-level features of the next decoder. The model training and deployment module is used to train the diffusion model using a constructed remote sensing image dataset containing high-resolution and low-resolution remote sensing image pairs. New low-resolution remote sensing images are restored using a degenerate kernel-based image restoration preprocessing module and then input into the trained diffusion model's Swin-UNet-based inverse Markov chain for super-resolution reconstruction.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described image super-resolution reconstruction method of fused image restoration and rapid diffusion when executing the computer program.

[0016] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a computer, implements the above-described method for fused image restoration and rapid diffusion image super-resolution reconstruction.

[0017] Compared with the prior art, the beneficial effects of the present invention include at least the following: The constructed diffusion model forms an ordered noise addition and denoising process, making the training process more stable and avoiding the training collapse phenomenon in methods such as GAN. The Swin-UNet network effectively extracts high-frequency details from remote sensing images, enabling more details to be restored during the denoising process and optimizing the visual effect of the reconstructed image. The image restoration preprocessing module based on the degenerate kernel improves the image generation quality of the diffusion model. Based on the network properties of Swin-UNet, a fast diffusion mechanism based on high-level feature skipping and reuse is designed, which effectively reduces the dependence on computing resources and improves diffusion efficiency and speed. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of the image super-resolution reconstruction method that combines image restoration and rapid diffusion provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the diffusion inference process of the diffusion model provided in this embodiment of the invention; Figure 3 This is a schematic diagram of the structure of the Swin-UNet network provided in an embodiment of the present invention; Figure 4This is a schematic diagram of the patch cutting layer operation provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the algorithm flow of the Swin Transformer module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the image processing flow for the remote sensing image inference process provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a fast diffusion mechanism based on high-level feature skipping and reuse provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the image super-resolution reconstruction device that integrates image restoration and rapid diffusion provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0021] The inventive concept of this invention is as follows: Addressing the problems of poor image generation quality and unstable training in existing deep learning-based super-resolution methods, this invention provides an image super-resolution reconstruction method and apparatus that integrates image restoration and fast diffusion. By constructing a diffusion model containing forward and backward Markov chains, a Swin-UNet network is used for noise prediction during backward denoising. Furthermore, an image restoration preprocessing module based on a degenerate kernel and a fast diffusion mechanism based on high-level feature skipping and reuse are introduced to improve the image generation quality and efficiency of the diffusion model. This enables super-resolution reconstruction of target low-resolution remote sensing images, generating corresponding high-quality, high-resolution remote sensing images with rich high-frequency information.

[0022] like Figure 1 As shown, the embodiment provides an image super-resolution reconstruction method that integrates image restoration and fast diffusion, including the following steps: S1. Construct a diffusion model by inputting a high-resolution remote sensing image into a forward Markov chain to add noise and generate a low-resolution noise map aligned with the low-resolution remote sensing image. Then, use the low-resolution remote sensing image as a conditional input to a Swin-UNet-based inverse Markov chain to denoise the low-resolution noise map and generate a reconstructed high-resolution remote sensing image.

[0023] S1.1, acquire four open-source high-resolution remote sensing image datasets based on satellite imagery: DOTA, AID, UC Merced, and OLI2MSI. Then, perform a 4x downsampling process on the images using bicubic interpolation to obtain corresponding low-resolution remote sensing images, forming high-resolution-low-resolution image pairs. In this embodiment, 12806 images from the DOTA and AID datasets are used for training, and 304 images from the OLI2MSI and UC Merced datasets are used for testing.

[0024] S2.2, as Figure 2 As shown, this invention first constructs a super-resolution diffusion process. Specifically, a Markov chain is constructed, where the starting point and ending point of the forward Markov chain are a high-resolution remote sensing image and a low-resolution remote sensing image, respectively. That is, in the forward Markov chain, noise perturbations are continuously superimposed on the high-resolution remote sensing image, making it gradually approximate the corresponding low-resolution remote sensing image.

[0025] In a forward Markov chain, the high-resolution remote sensing image is input... Begin, at each time step According to conditional distribution Noise is added gradually, for a total of Step noise, among which For the noise addition process noise map of the step, To and The corresponding low-resolution remote sensing image, Indicates the distance between them. Preset for controlling different time steps The noise level noise scheduling parameters, Let be the identity matrix. By integrating the edge probabilities at each step of the noise addition process, the initial distribution can be obtained. Distribution with current steps The relationship, that is ,in This is a preset noise sequence used to control the increase in noise level at each step, and , To and A uniformly scaled identity matrix. Let and This ensures and In other words, the initial distribution of the Markov chain is close to the high-resolution remote sensing image, and the final distribution is close to the low-resolution remote sensing image, thus realizing the gradual transformation from a high-resolution image to a low-resolution image.

[0026] exist In terms of design, due to The main control is the variance of the noise at each step, therefore the noise level is related to... Proportional. In the example, let... ,when hour, .in and Thus making With noise step The increase in noise is roughly linear, which helps to control the smooth increase or decrease of noise, thereby making the diffusion process more stable.

[0027] Compared to the traditional diffusion process that gradually adds noise to an image until it is pure Gaussian white noise, this invention, based on the characteristics of super-resolution reconstruction tasks, directly adds noise to high-resolution remote sensing images until they approximate low-resolution images, greatly reducing the steps required for the diffusion process and improving diffusion efficiency.

[0028] S2.3, in the backward Markov chain, from the low-resolution noise map Begin by distributing according to the conditions. Gradually remove noise, among which For the noise reduction process noise map of the step, and Let be the mean and covariance of the noise in each noise addition step, respectively. The optimization objective of this learnable process is to minimize the noise in the first noise addition step. The first step of adding noise and the denoising process The distance between the predicted noise steps is calculated by minimizing the lower bound of the negative variation. .in Represents distribution With distribution The KL divergence between them. During inference, each step is based on the noise graph. Low-resolution remote sensing images Current step Three parameters predict the noise that needs to be removed in the current step, and the low-resolution noise map is gradually reconstructed into a high-resolution remote sensing image through a reverse Markov chain based on Swin-UNet.

[0029] like Figure 3As shown, the Swin-UNet network adopts an encoder-decoder structure. The encoder achieves multi-scale feature extraction and downsampling through alternating multi-level Swin Transformer modules and Patch merging layers, while the decoder achieves upsampling through alternating multi-level Swin Transformer modules and Patch expansion layers. Skip connections are introduced between the Swin Transformer modules of each layer in the encoder and decoder, and a bottleneck constructed by multi-level Swin Transformer modules is used to connect the last layer of the encoder and the first layer of the decoder. This effectively extracts deep features from remote sensing images and improves training efficiency.

[0030] like Figure 4 As shown, in the encoder, the remote sensing image is segmented into 16 non-overlapping 4×4 patches and then concatenated to convert the remote sensing image input into a sequence input. A linear embedding layer is used to map the feature dimension from 4×4×3=48 to a fixed dimension C. The transformed patches then pass through several Swin Transformer modules and a patch merging layer to generate a hierarchical feature representation. The Swin Transformer modules are mainly used to learn the feature representation, and the patch merging layer is mainly used for 2x downsampling and increasing dimensionality.

[0031] The decoder consists of a Swing Transformer module and patch extension layers. The patch extension layers primarily perform 2x upsampling. The contextual features extracted by the decoder are fused with the multi-scale features extracted by the encoder via skip connections to compensate for spatial information loss caused by downsampling. The final two patch extension layers perform 4x upsampling to obtain the final 4x super-resolution feature map. A linear mapping layer is then used to transform the feature map into the final super-resolution result.

[0032] like Figure 5 As shown, the Swing Transformer module employs two consecutive Swing Transformer blocks. Each Swing Transformer block consists of layer normalization, a multi-head self-attention module, residual connections, and two multilayer perceptron layers with GELU activation functions. In the two consecutive Swing Transformer modules, the first module uses a window-based multi-head self-attention module, and the second module uses a sliding window-based multi-head self-attention module.

[0033] In the bottleneck, two consecutive Swing Transformer modules are used to learn deep feature representations, while the feature dimension and resolution remain unchanged during the process.

[0034] During the denoising process, in the first layer of the Swin-UNet network model and at each decoding stage, the low-resolution remote sensing image and its corresponding noise level label are embedded through isomorphic mapping and added point-by-point with the Transformer feature map to achieve conditional control. In each upsampling stage, the encoded features of cross-layer skip connections are integrated through 3×3 convolutions and combined with decoder features to recover spatial details.

[0035] S2, in the Swin-UNet diffusion inference process, adopts a fast diffusion mechanism based on high-level feature skipping and reuse to cache the high-level features of the previous decoder. In the next step, the low-level features of the encoder are fused with the cached high-level features to skip the calculation of the high-level features of the next decoder.

[0036] like Figure 6 As shown, based on the characteristics of Swin-UNet, this invention designs a fast diffusion mechanism based on high-level feature skipping and reuse, which can skip the calculation of high-level features in Swin-UNet at certain steps. The Swin-UNet network can effectively extract deep features of remote sensing images, better predict noise, and improve image quality, while the fast diffusion mechanism based on high-level feature skipping and reuse can effectively reduce the dependence on computing resources and further improve diffusion efficiency.

[0037] like Figure 7 As shown, in the inverse denoising... The first step involves caching the high-level features (UF1, UF2, UF3, UF4…) output from the Swin-UNet decoder, and then performing this step in the inverse denoising process. Step, the first The high-level features cached in the first step and the first In the first step, the low-level features (DF1', DF2', DF3', DF4'...) output by the Swin-UNet encoder are fused, thus skipping the second step. The high-level feature calculation process of the decoder is repeated every two steps to achieve high-level feature reuse. In this embodiment, the process is repeated every two steps, which reduces the network computation by 25%, thereby accelerating the inference process and improving inference efficiency.

[0038] S3 uses a constructed remote sensing image dataset containing high-resolution and low-resolution remote sensing image pairs to train the diffusion model. New low-resolution remote sensing images are restored using a degenerate kernel-based image restoration preprocessing module and then input into the Swin-UNet-based inverse Markov chain in the trained diffusion model for super-resolution reconstruction.

[0039] S3.1, The diffusion model is trained using the remote sensing image dataset constructed in step S1.1, and the Swin-UNet-based inverse Markov chain in the converged diffusion model is saved for inference on new low-resolution remote sensing images.

[0040] S3.2, as Figure 6 As shown, a dual-stream feature extraction network and a frequency-domain Wiener filtering module are used to predict the degradation kernel of low-resolution remote sensing images. The dual-stream feature extraction network extracts multi-scale features from the degraded image (low-resolution remote sensing image) and a reference image (a high-resolution remote sensing image corresponding to the low-resolution image, or a high-resolution remote sensing image of the same region as the low-resolution image). The frequency-domain Wiener filtering module performs frequency domain operations on the extracted multi-scale features of the degraded image and the reference image to obtain the predicted degradation kernel. The predicted degradation kernel and the low-resolution remote sensing image are input together into a deep Wiener deconvolution network to obtain the restored image, completing the preprocessing. The restored image is then input into a trained Swin-UNet-based inverse Markov chain to progressively recover the high-resolution remote sensing image, thereby significantly improving the image quality of the super-resolution reconstruction of the diffusion model.

[0041] In summary, the image super-resolution reconstruction method that integrates image restoration and rapid diffusion provided by this invention, under the same hardware conditions, accelerates the inference speed by about 20% compared to other diffusion models, reduces the number of inference steps to 15, and improves the image quality evaluation indicators such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the generated super-resolution remote sensing images. The generated images have better visual effects and can perform super-resolution reconstruction of remote sensing images efficiently and at high speed.

[0042] Based on the same inventive concept, such as Figure 8 As shown, this embodiment of the invention also provides an image super-resolution reconstruction device 800 that integrates image restoration and fast diffusion, including: a diffusion model modeling module 810, a fast diffusion inference module 820, and a model training and deployment module 830.

[0043] The diffusion modeling module 810 is used to construct a diffusion model. It inputs a high-resolution remote sensing image into a forward Markov chain to add noise and generate a low-resolution noise map aligned with the low-resolution remote sensing image. It then uses the low-resolution remote sensing image as a conditional input to a Swin-UNet-based inverse Markov chain to denoise the low-resolution noise map and generate a reconstructed high-resolution remote sensing image.

[0044] The fast diffusion inference module 820 is used in the Swin-UNet diffusion inference process to adopt a fast diffusion mechanism based on high-level feature skipping and reuse. It is used to cache the high-level features of the previous decoder and fuse the low-level features of the encoder with the cached high-level features in the next step to skip the calculation of high-level features of the next decoder.

[0045] The model training and deployment module 830 is used to train the diffusion model using a constructed remote sensing image dataset containing high-resolution and low-resolution remote sensing image pairs. New low-resolution remote sensing images are restored using the image restoration preprocessing module based on degradation kernels and then input into the Swin-UNet-based inverse Markov chain in the trained diffusion model for super-resolution reconstruction.

[0046] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described image super-resolution reconstruction method of fused image restoration and rapid diffusion when executing the computer program.

[0047] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the above-described image super-resolution reconstruction method of fused image restoration and rapid diffusion.

[0048] It should be noted that the image super-resolution reconstruction apparatus, electronic device, and computer-readable storage medium provided in the above embodiments, which combine image restoration and rapid diffusion, all belong to the same inventive concept as the image super-resolution reconstruction method that combines image restoration and rapid diffusion. For details of their implementation process, please refer to the embodiments of the image super-resolution reconstruction method that combines image restoration and rapid diffusion, which will not be repeated here.

[0049] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for super-resolution image reconstruction that integrates image restoration and fast diffusion, characterized in that, Includes the following steps: A diffusion model is constructed. A forward Markov chain is input to a high-resolution remote sensing image to generate a low-resolution noise map that is aligned with the low-resolution remote sensing image. The low-resolution remote sensing image is used as a conditional input to a reverse Markov chain based on Swin-UNet to denoise the low-resolution noise map and generate a reconstructed high-resolution remote sensing image. In the Swin-UNet diffusion inference process, a fast diffusion mechanism based on high-level feature skipping and reuse is adopted to cache the high-level features of the previous decoder. In the next step, the low-level features of the encoder are fused with the cached high-level features to skip the calculation of the high-level features of the next decoder. The diffusion model was trained using a constructed dataset of remote sensing images containing high-resolution and low-resolution remote sensing image pairs. New low-resolution remote sensing images were restored using a degenerate kernel-based image restoration preprocessing module and then input into the Swin-UNet-based inverse Markov chain in the trained diffusion model for super-resolution reconstruction.

2. The image super-resolution reconstruction method fusing image restoration and fast diffusion according to claim 1, characterized in that, The step of inputting a high-resolution remote sensing image into a forward Markov chain to add noise and generate a low-resolution noise map aligned with the low-resolution remote sensing image includes: From the input high-resolution remote sensing image Begin, at each time step According to conditional distribution Noise is gradually added, among which For the noise addition process noise map of the step, To and The corresponding low-resolution remote sensing image, Indicates the distance between them. Preset for controlling different time steps The noise level and noise scheduling parameters. It is the identity matrix; when adding the common After the first noise step, make the first Low-resolution noise map of the step Approximate to To achieve alignment.

3. The image super-resolution reconstruction method merging image restoration and rapid diffusion according to claim 1 or 2, characterized in that, The step of using a low-resolution remote sensing image as a conditional input to a Swin-UNet-based inverse Markov chain to denoise the low-resolution noisy image and generate a reconstructed high-resolution remote sensing image includes: From low-resolution noise map Begin by distributing according to the conditions. Gradually remove noise, among which For the noise reduction process noise map of the step, and These represent the mean and covariance of the noise during each noise addition step; during inference, each step is based on the noise map. Low-resolution remote sensing images Current step Three parameters predict the noise that needs to be removed in the current step, and the low-resolution noise map is gradually reconstructed into a high-resolution remote sensing image through a reverse Markov chain based on Swin-UNet.

4. The image super-resolution reconstruction method fusing image restoration and fast diffusion according to claim 1, characterized in that, The Swin-UNet network adopts an encoder-decoder structure, in which the encoder performs multi-scale feature extraction and downsampling through alternating multi-level SwinTransformer modules and Patch merging layers, and the decoder performs upsampling through alternating multi-level SwinTransformer modules and Patch expansion layers. Skip connections are introduced between the SwinTransformer modules of each layer in the encoder and decoder, and the last layer of the encoder and the first layer of the decoder are connected through a bottleneck constructed by the multi-level SwinTransformer modules.

5. The image super-resolution reconstruction method fusing image restoration and fast diffusion according to claim 1, characterized in that, The fast diffusion mechanism based on high-level feature skipping and reuse is used to cache the high-level features of the previous decoder. In the next step, the low-level features of the encoder are fused with the cached high-level features to skip the calculation of high-level features in the next decoder step. This includes: In the reverse denoising process The step is to cache the high-level features of the decoder output of the Swin-UNet; In the reverse denoising process Step, the first The high-level features cached in the first step and the first In the first step, the low-level features output by the Swin-UNet encoder are fused to skip the second step. The high-level feature calculation process of the step decoder enables high-level feature reuse and accelerates the inference process.

6. The image super-resolution reconstruction method fusing image restoration and fast diffusion according to claim 1, characterized in that, The process of restoring new low-resolution remote sensing images using a degradation kernel-based image restoration preprocessing module includes: A dual-stream feature extraction network and a frequency-domain Wiener filtering module are used to predict the degradation kernel of low-resolution remote sensing images. The predicted degradation kernel and the low-resolution remote sensing image are then input into a deep Wiener deconvolution network to obtain the restored image, thus completing the preprocessing.

7. The image super-resolution reconstruction method fusing image restoration and fast diffusion according to claim 1, characterized in that, The process of constructing a remote sensing image dataset containing pairs of high-resolution and low-resolution remote sensing images includes: High-resolution remote sensing images captured by satellites are acquired and bicubic interpolation downsampling is performed on them to obtain corresponding low-resolution remote sensing images. Several image pairs containing high-resolution and low-resolution remote sensing images are then constructed into a remote sensing image dataset.

8. An image super-resolution reconstruction apparatus that integrates image restoration and fast diffusion, implemented using the image super-resolution reconstruction method integrating image restoration and fast diffusion as described in any one of claims 1 to 7, characterized in that, include: The module includes a diffusion modeling module, a fast diffusion inference module, and a model training and deployment module. The diffusion modeling module is used to construct a diffusion model. It inputs a high-resolution remote sensing image into a forward Markov chain to add noise and generate a low-resolution noise map that is aligned with the low-resolution remote sensing image. It then uses the low-resolution remote sensing image as a conditional input to a Swin-UNet-based inverse Markov chain to denoise the low-resolution noise map and generate a reconstructed high-resolution remote sensing image. The fast diffusion inference module is used in the Swin-UNet diffusion inference process to adopt a fast diffusion mechanism based on high-level feature skipping and reuse. It is used to cache the high-level features of the previous decoder and fuse the low-level features of the encoder with the cached high-level features in the next step to skip the calculation of the high-level features of the next decoder. The model training and deployment module is used to train the diffusion model using a constructed remote sensing image dataset containing high-resolution and low-resolution remote sensing image pairs. New low-resolution remote sensing images are restored using a degenerate kernel-based image restoration preprocessing module and then input into the trained diffusion model's Swin-UNet-based inverse Markov chain for super-resolution reconstruction.

9. An electronic device comprising a memory and one or more processors, the memory for storing a computer program, characterized in that, The processor is used to implement the image super-resolution reconstruction method of fusion image restoration and rapid diffusion as described in any one of claims 1 to 7 when executing a computer program.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a computer, it implements the image super-resolution reconstruction method of fusion image restoration and rapid diffusion as described in any one of claims 1 to 7.

Images