Blind image super-resolution reconstruction method and system

Abstract

The invention provides a blind image super-resolution reconstruction method and system, and belongs to the technical field of image reconstruction, and the method comprises the steps: obtaining a damaged low-resolution image; processing the obtained damaged low-resolution image by using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under a maximum posterior probability framework, and is expanded along with the inverse process of the de-noising probability diffusion model. According to the method, the blind degradation setting problem in single image super-resolution is solved, optimization based on the maximum posterior probability is integrated into DM, joint distribution of low-resolution observation, high-resolution data and a degradation kernel is constructed to provide prior information of the data and the kernel, and the blind SR problem is solved by expanding the maximum posterior probability method and a reverse process.

Description

Technical Field

[0001] The present invention relates to the technical field of image reconstruction, and in particular to a blind image super-resolution reconstruction method and system based on degradation modeling and diffusion modeling. Background Art

[0002] Single image super-resolution (SISR) is a long-standing problem in computer vision and image restoration. It has attracted much attention due to its ill-posedness and wide range of practical applications. SISR refers to the generation of a high-resolution (HR) image from a low-resolution (LR) observation y through a degradation model. Its degradation model can be expressed as: where k represents the blur kernel convolved with x, followed by a downsampling operation with a scaling factor s. n is usually assumed to be additive white Gaussian noise (AWGN). In the past few years, SISR methods have been dominated by deep learning methods, most of which mainly deal with predefined degradations (e.g., bicubic downsampling), using a large number of synthetic LR-HR pairs to perform end-to-end supervised training of powerful deep convolutional networks. Despite their success, the performance drops significantly when applied to other unseen degradations. This is particularly evident in real-world images, where the degradations are complex and unavailable.

[0003] To overcome this limitation, recent researchers have paid increasing attention to blind image super-resolution (SR), whose key steps lie in degradation estimation (mainly blur kernel k) and degradation-aware SR reconstruction. One approach is to estimate the underlying k from the LR input y and apply it to a non-blind SR model to generate a HR image. Another approach is to unify kernel estimation and SR reconstruction in an end-to-end framework. However, the results of these methods still suffer from noticeable artifacts and low perceptual quality when dealing with complex degradations.

[0004] Recently, the emerging diffusion model (DM) has been demonstrated to be a powerful generative model for high-fidelity image generation. For example, DALLE-2 utilizes DM hierarchically to generate high-quality image representations based on CLIP embedding. DDFM proposes a DM-based posterior sampling model that integrates likelihood correction into the diffusion sampling iterations to achieve multimodal image fusion. Currently, there are two main paradigms for DM-based methods in single image super-resolution (SISR). One is to train DM from scratch to make it dependent on low-resolution (LR) images. The other more popular approach is to design problem-agnostic models that adopt off-the-shelf pre-trained DM as a generative prior for general restoration. However, it should be noted that both methods usually focus on the non-blind inverse problem, i.e., assume that the image degradation is known and fixed. This is impractical in real-world applications and, therefore, severely limits their generalization ability. Summary of the invention

[0005] The object of the present invention is to provide a blind image super-resolution reconstruction method and system to solve at least one technical problem existing in the above-mentioned background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a blind image super-resolution reconstruction method, comprising:

[0008] Get corrupted low-resolution images;

[0009] The obtained corrupted low-resolution image is processed by using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to obtain prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop for alternating kernel and high-resolution image estimation.

[0010] As a further qualification of the first aspect, given a corrupted low-resolution image y, reconstructing a high-resolution image x from y and a blur kernel k is formulated as a maximum a posteriori probability problem:

[0011] p(k,x|y)=p(k|y)p(x|k,y);

[0012] Here, k and x are solved as follows:

[0013]

[0014] Where logp(y|k,x) represents the log-likelihood of y, logp(k|y) represents the kernel prior captured from y, and logp(x) represents the sharp image prior that is independent of y; 0 and k 0 denote the potential optimal estimates of x and k respectively;

[0015] Decompose the solution of k and x into two sub-problems:

[0016]

[0017] Forming an iterative optimization process:

[0018]

[0019] Then, the potential SR image is used as auxiliary information to estimate k:

[0020]

[0021] As a further limitation of the first aspect, in the denoising probability diffusion model, by training a conditional model, from x T Obtain a series of denoising priors And model logp(y|k,x) in the reverse sampling process; where,

[0022] According to the diffusion posterior sampling algorithm for non-blind inverse problem, then:

[0023]

[0024] in, is x t The denoised samples, Obtained by the analytical likelihood of the Gaussian measurement model:

[0025]

[0026] Therefore, calculating the gradient And back propagate through the network to optimize

[0027] As a further limitation of the first aspect, the modulation condition transformer comprises a kernel estimator K and a kernel modulation transformer backbone, which is trained by ∈ θ (X t ,t,y) and the benchmark true kernel k gt L between the estimated k 1 Loss to achieve:

[0028]

[0029] Among them, f θ (x 0 ,y,∈) represents the total loss, is the noisy target image, and K takes the bicubic interpolated y and scale s as input.

[0030] As a further limitation of the first aspect, the kernel estimator encodes image features through four 5×5 convolutional layers and a Leaky ReLU activation function; and performs global average pooling on the feature map to form an estimation element of the blur kernel:

[0031]

[0032] Among them↑ srepresents bicubic interpolation with scale factor s, k t is the kernel generated at time step t.

[0033] As a further limitation of the first aspect, the kernel modulated transformer backbone includes: a multi-Dconv head transposed attention layer, a Gated-Dconv feedforward network layer, and a modulation based on a blur kernel and a time step; first, an affine transformation is provided for the input feature vector F after layer normalization:

[0034]

[0035] in, represents the transformation function, γ 1 and τ 1 is achieved by combining t and k t The learnable parameters of the two linear layers are used for scaling and offset operations respectively; It is the characteristic after modulation;

[0036] exist Apply multiple Multi-Dconv head transposed attention layers to model global connections, and get

[0037]

[0038] Among them, γ 2 and τ 2 Also from the fusion of t and k t Learned from features;

[0039] The Gated-Dconv feed-forward network layer is used to aggregate local features and mine meaningful information from spatially adjacent pixel locations to generate output features.

[0040] In a second aspect, the present invention provides a blind image super-resolution reconstruction system, comprising:

[0041] An acquisition module, used for acquiring the damaged low-resolution image;

[0042] A processing module is used to process the acquired damaged low-resolution image using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to obtain prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop for alternating kernel and high-resolution image estimation.

[0043] In a third aspect, the present invention provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the blind image super-resolution reconstruction method as described in the first aspect is implemented.

[0044] In a fourth aspect, the present invention provides a computer device comprising a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the blind image super-resolution reconstruction method as described in the first aspect.

[0045] In a fifth aspect, the present invention provides an electronic device, comprising: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the blind image super-resolution reconstruction method as described in the first aspect.

[0046] The beneficial effects of the present invention are as follows: an efficient blind super-resolution (SR) diffusion solution is proposed, and by utilizing a diffusion model (DM), the degradation kernel in SISR is explicitly simulated to achieve high-fidelity image generation; a maximum a posteriori probability (MAP) method is used, which can iteratively and alternately optimize blur kernel estimation and high-resolution (HR) image restoration based on an inverse process; the proposed Modulated Conditional Transformer (MCFormer) is used as a denoising network to provide data and kernel prior information for posterior sampling, and MCFormer is combined with a kernel modulation mechanism to adjust image features according to kernel information, which is conducive to the acquisition of multi-level degradation perception prior features; while greatly reducing the complexity of the model, the performance is significantly improved.

[0047] Additional advantages of the present invention will be more clearly presented in the following description or learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0049] Figure 1 Schematic diagram of the network architecture of the modulation condition transformer according to an embodiment of the present invention.

[0050] Figure 2 It is a schematic diagram of visual comparison of various 4x blind super-resolution methods on an isotropic Gaussian kernel according to an embodiment of the present invention.

[0051] Figure 3 It is a schematic diagram of visual comparison of various 4x blind super-resolution methods on anisotropic Gaussian kernel according to an embodiment of the present invention.

[0052] Figure 4 Schematic diagram of visual results of various 4x blind super-resolution methods on RealSRSet described in embodiments of the present invention.

[0053] Figure 5 It is a visualization diagram of the results under different sampling iteration times described in the embodiment of the present invention.

[0054] Figure 6 It is a schematic diagram of kernel estimation results on an image degraded by anisotropic kernel according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below by the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0056] It should be understood by those skilled in the art that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0057] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with that in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0058] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements and / or groups thereof.

[0059] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. Different embodiments or examples described in this specification and features of different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

[0060] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings, and the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0061] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0062] Diffusion models (DMs) have achieved remarkable success in image super-resolution (SR). However, most models are tailored to solve non-blind inverse problems with fixed known degradation settings, limiting their adaptability to real applications involving complex unknown degradations. In this paper, BlindDiff, a blind SR method based on the DM model, is proposed to solve the blind degradation setting problem in single image super-resolution (SISR). BlindDiff seamlessly integrates maximum a posteriori probability (MAP)-based optimization into DM, constructs a joint distribution of low-resolution (LR) observations, high-resolution (HR) data, and degradation kernels to provide prior information of data and kernels, and solves the blind SR problem by unfolding the MAP method and the reverse process. Different from most DMs, BlindDiff first proposes a modulated conditional transformer (MCFormer) pre-trained with noise and kernel constraints, which further acts as a posterior sampler to provide two kinds of prior information simultaneously. Then, a simple but effective kernel-aware gradient term is inserted between adjacent sampling iterations to guide the diffusion model to learn degradation consistency knowledge. This also enables the joint optimization of the degradation model and the HR image by observing the previous denoised sample. Through a MAP-based back-diffusion process, we show that BlindDiff advocates alternating optimization between blur kernel estimation and HR image restoration in a mutually reinforcing manner. Experiments on both synthetic and real datasets show that BlindDiff achieves state-of-the-art performance while significantly reducing model complexity compared to recent DM-based methods.

[0063] Example 1

[0064] In this embodiment 1, a blind image super-resolution reconstruction system is first provided, including: an acquisition module, used to acquire a damaged low-resolution image; a processing module, used to use a pre-trained diffusion super-resolution model to process the acquired damaged low-resolution image to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to acquire prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop to perform alternating kernel and high-resolution image estimation.

[0065] In this embodiment, the above-mentioned system is used to implement a blind image super-resolution reconstruction method, including: using an acquisition module to acquire a damaged low-resolution image; using a processing module to process the acquired damaged low-resolution image using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to acquire prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop to perform alternating kernel and high-resolution image estimation.

[0066] Here, given a corrupted low-resolution image y, reconstructing a high-resolution image x from y and a blur kernel k is formulated as a maximum a posteriori probability problem:

[0067] p(k,x|y)=p(k|y)p(x|k,y);

[0068] Here, k and x are solved as follows:

[0069]

[0070] where logp(y|k,x) represents the log-likelihood of y, logp(k|y) represents the kernel prior captured from y, and logp(x) represents the sharp image prior that is independent of y; 0 and k 0 denote the potential optimal estimates of x and k respectively;

[0071] Decompose the solution of k and x into two sub-problems:

[0072]

[0073] Forming an iterative optimization process:

[0074]

[0075] Then, the potential SR image is used as auxiliary information to estimate k:

[0076]

[0077] In the denoising probability diffusion model, by training a conditional model, from x T Get a series of denoising priors And model logp(y|k,x) in the reverse sampling process; where,

[0078] According to the diffusion posterior sampling algorithm for non-blind inverse problem, then:

[0079]

[0080] in, is x t The denoised samples, Obtained by the analytical likelihood of the Gaussian measurement model:

[0081]

[0082] Therefore, calculating the gradient And back propagate through the network to optimize

[0083] The modulation condition transformer consists of a kernel estimator K and a kernel modulation transformer backbone, which is trained by ∈ θ (X t ,t,y) and the benchmark true kernel k gt L between the estimated k 1 Loss to achieve:

[0084]

[0085] Among them, f θ (x 0 ,y,∈) represents the total loss, is the noisy target image, and K takes the bicubic interpolated y and scale s as input.

[0086] The kernel estimator encodes image features through four 5×5 convolutional layers and Leaky ReLU activation function; the feature map is globally averaged pooled to form the estimated elements of the blur kernel:

[0087]

[0088] Among them↑ s represents bicubic interpolation with scale factor s, k t is the kernel generated at time step t.

[0089] The kernel modulated transformer backbone includes: multi-Dconv head transposed attention layer, Gated-Dconv feed-forward network layer, and modulation based on blur kernel and time step; first, an affine transformation is provided for the input feature vector F after layer normalization:

[0090]

[0091] in, represents the transformation function, γ 1 and τ 1 is achieved by combining t and k t The learnable parameters of the two linear layers are used for scaling and offset operations respectively; It is the characteristic after modulation;

[0092] exist Apply multiple Multi-Dconv head transposed attention layers to model global connections, and get

[0093]

[0094] Among them, γ 2 and τ 2 Also from the fusion of t and k t Learned from features;

[0095] The Gated-Dconv feed-forward network layer is used to aggregate local features and mine meaningful information from spatially adjacent pixel locations to generate output features.

[0096] Example 2

[0097] In this embodiment 2, an efficient blind super-resolution (SR) diffusion solution (BlindDiff) is proposed. BlindDiff explicitly simulates the degradation kernel in SISR by utilizing the diffusion model (DM), which is expected to achieve high-fidelity generation in practical scenarios. BlindDiff uses the maximum a posteriori probability (MAP) method in the diffusion model, which can iteratively and alternately optimize blur kernel estimation and high-resolution (HR) image restoration based on the inverse process. A modulated conditional transformer (MCFormer) is proposed as a denoising network to provide data and kernel prior information for posterior sampling. In addition, MCFormer combines the kernel modulation mechanism to adjust the image features according to the kernel information, which is conducive to the acquisition of multi-level degradation-aware prior features. Extensive experiments on synthetic and real datasets show that state-of-the-art performance is achieved. In particular, compared with existing DM-based methods, BlindDiff has significantly improved performance while greatly reducing the complexity of the model.

[0098] Diffusion model (DM), also known as denoising probabilistic diffusion model (DDPM), is a type of generative model that includes a diffusion process and an inverse process. Given a clear image x 0 , the diffusion process defines a Markov chain x 0:T , which gradually moves towards x 0 ~q(x 0 ) adds noise and has a fixed variance schedule β 1,...,β T

[0099]

[0100] in Due to the characteristics of Markov chains, we can directly 0 Get x t , through a positive process in

[0101] In the inverse process, there is an oppositely parameterized Markov chain x T:0 , which is a random Gaussian noise Start with and gradually sample from noisy data until you reach x 0

[0102]

[0103] in represents the variance set as a time-dependent constant. Mean μ θ (x t ,t) is the denoising network ∈ parameterized by θ θ (x t ,t) Estimated learnable parameters: In practice, we usually start from x t predict And use x t and the calculated Sample x t-1

[0104]

[0105] The model is trained using a simplified objective function

[0106] The conditional diffusion model aims to diffuse the noise image according to the given condition y. Generate target image x 0 These models train the denoising function ∈ θ (X t ,t,y) to learn the conditional distribution p θ (x t |x t-1 ,y). Once the model is trained, it can be 0 ~p(x|y) gradually generates a series of samples x T:0 :

[0107]

[0108] where ∈ θ (xt ,t,y) provides an estimate of the log density gradient of the data in the Langevin dynamics.

[0109] The BlindDiff proposed in this embodiment aims to integrate the MAP method and DDPM for blind super-resolution (SR). BlindDiff decomposes the blind SR problem into two sub-problems under the MAP framework: kernel estimation and high-resolution (HR) image restoration. The MAP optimizer is further integrated into the DDPM, and the alternate estimation of blur kernel and HR image is realized as the reverse process unfolds.

[0110] In this embodiment, the proposed BlindDiff method is theoretically analyzed, and BlindDiff is introduced in detail as follows.

[0111] Given a low-resolution (LR) image y corrupted by a classical degradation model (described in the background), blind SR involves estimating the blur kernel k and then reconstructing the HR image x from y and k, which can generally be formulated as a MAP problem:

[0112] p(k,x|y)=p(x|y)p(x|k,y) (6)

[0113] Among them, k and x can be solved by the following method:

[0114]

[0115] where logp(y|k,x) represents the log-likelihood of y, logp(k|y) represents the kernel prior captured from y, and logp(x) represents the sharp image prior that is independent of y. 0 and k 0 denote the potential optimal estimates of x and k respectively. By decomposing formula (7) into two sub-problems, we can obtain:

[0116]

[0117] To proceed further, formula (8) can be expanded to form an iterative optimization process:

[0118]

[0119] In this formula, the estimation of k is independent of x, so the whole process can be viewed as a two-step blind SR process, i.e., estimating the kernel from y and then performing non-blind SR based on the kernel. This has been shown to be suboptimal due to the lack of information from x when estimating the kernel. To address this issue, this embodiment follows the common practice in existing end-to-end blind SR methods and uses the latent SR image as auxiliary information to estimate k. Therefore, formula (8a) can be restated as:

[0120]

[0121] In addition, since SISR is a conditional image generation task, we usually model the conditional posterior distribution p(x|y) as a data prior term instead of p(x). This can be achieved by many well-trained generative models. This study adopts DDPMs to achieve this purpose because of their excellent generative ability.

[0122] In the DDPM mode, we can naturally learn from x by training a conditional model. T Obtain a series of denoising priors Now, let us first analyze x 0 The solution. Through the prior And LR condition y, to solve the problem of x, another key point is to model logp(y|k,x) in the reverse sampling process.

[0123] To achieve this, first specify the posterior distribution of this term as a Gaussian distribution: Based on the diffuse a posteriori sampling algorithm [9] for non-blind inverse problems, we modify it to fit the blind setting, expressed as follows:

[0124]

[0125] in, is calculated by formula (4) t Denoised samples. It can be obtained by the analytical likelihood of my Gaussian measurement model:

[0126]

[0127] Therefore, the gradient can be calculated And back propagate through the network to optimize like Figure 2 It should be noted that if we transform formula (12) into the solution of minimizing the energy function: This is the kernel-aware gradient term, which is actually very close to the data fidelity term: This ensures that the solution is consistent with the degradation knowledge.

[0128] Accordingly, it is necessary to provide data prior x t and the kernel prior k t Combining formula (5) and formula (9b), since the estimation of these two components depends on y and the previous sample x t, when training the denoising network, not only the noise-related target ∈ θ (x t ,t,y), and also use kernel-related objectives. To this end, we propose a Modulated Conditional Transformer (MCFormer), which consists of a kernel estimator K and a kernel modulated transformer backbone, which is trained by ∈ θ (X t ,t,y) and the benchmark true kernel k gt L between the estimated k 1 Losses are realized.

[0129]

[0130] where f θ (x 0 ,y,∈) represents the total loss, is the noisy target image, and K takes the bicubic interpolated y and scale s as input.

[0131] Once trained, the model is able to achieve denoised high-resolution image generation and kernel estimation at each backward iteration. Then, by using the gradient term (Formula (12)) connects adjacent iterations. A closed loop can be formed to learn the blur kernel and high-resolution image from each other. As the iterations proceed, BlindDiff achieves alternating optimization of these two sub-problems until they approach their optimal estimates. Therefore, formula (9) can be further rewritten as

[0132]

[0133] In summary, BlindDiff expands the MAP method into DDPMs and decomposes the blind super-resolution problem into two sub-problems that can be solved alternately in the reverse process. The algorithmic pseudocodes of the diffusion and reverse processes in BlindDiff are summarized in Algorithms 1 and 2. It is worth noting that this work is very different from BlindDPS [8], which considers x t and k t are independent and use two independently pre-trained DDPMs to provide kernel and data priors backward in parallel.

[0134] The detailed architecture of the denoising network, namely the Modulated Conditional Transformer (MCFormer), is shown in Figure 1 MCFormer uses a kernel estimator to learn the blur kernel and combines a multi-scale hierarchical backbone network of transformer blocks to model f θ (x 0,y,∈). The network architecture of the proposed Modulated Conditional Transformer (MCFormer) MCFormer consists of a kernel estimator for predicting the blur kernel and a serial transformer block in a multi-scale hierarchical design.

[0135] For the kernel estimator. Given a low-resolution input y, it is first upsampled to the target resolution using bicubic interpolation and compared with the intermediate image x t spliced ​​together. The kernel estimator is placed at the beginning of MCFormer to be modulated in subsequent stages. Specifically, it encodes image features through four 5×5 convolutional layers and Leaky ReLU activation function (LReLU). Then, we perform global average pooling on the feature map to form the estimated elements of the blur kernel, where the dimension of the kernel space is further reduced by principal component analysis (PCA). The entire estimation process is formulated as follows:

[0136]

[0137] Among them↑ s represents bicubic interpolation with scale factor s, k t is the kernel generated at time step t.

[0138] For the kernel modulation transformer block. MCFormer embeds kernel and noise temporal information into the transformer block to learn degradation-aware feature representations. There are three components in the transformer block: 1) Multi-Dconv head transposed attention (MDTA) layer, 2) Gated-Dconv feedforward network (GDFN) layer, and 3) modulation based on blur kernel and time step, as Figure 3 As shown. is the intermediate input tensor, where H, W, and C represent the height, width, and channel dimensions of F, respectively. Time steps t and k t are the other two inputs. It is worth noting that has been preprocessed by a multi-layer perceptron (MLP). Then, another MLP is used to transform k t Projected to the same dimension as t. The first modulation module provides an affine transformation to F after layer normalization:

[0139]

[0140] in, represents the transformation function, γ 1 and τ 1 is achieved by combining t and k t The two linear layers of have learnable parameters, which are used for scaling and offset operations respectively. is the modulated characteristic. Then, We apply MDTA to model global connections, as it has been shown to be an efficient and effective attention mechanism, and obtain Afterwards, another modulation module is used to manipulate the attention features:

[0141]

[0142] Among them, γ 2 and τ 2 Also from the fusion of t and k t Finally, GDFN is used to aggregate local features and mine meaningful information from spatially adjacent pixel locations to generate output features. This feature is expected to be more discriminative and stable in response to degradation.

[0143] In this example, BlindDiff is trained using the Adam optimizer with a mini-batch size of 8 and a HR image patch size of 256×256 for 500K iterations. The initial learning rate is set to 2e-4 and halved every 100K iterations. The proposed MCFormer is a 4-level encoder-decoder. From level 1 to level 4, the number of transformer blocks is [2, 3, 6, 8], the number of channels is [48, 96, 192, 384], and the number of attention heads in MDTA is set to [1, 2, 4, 8]. In the refinement stage, there are 2 transformer blocks with 48 channels and a single-head MDTA.

[0144] BlindDiff is applied to the blind super-resolution task and its 4× SR performance is evaluated on a face dataset FFHQ (256×256) and six natural image datasets, including Set5, BSD100, DIV2K100, and ImageNet-1K (256×256). FFHQ contains 70,000 face images, 1,000 of which are randomly selected for validation and the rest are used for training. For natural image super-resolution, the model is trained on the DIV2K (800) and Flickr2K (2650) datasets, with a total of 3,450 images. It is worth noting that for efficient comparison, in DIV2K100, the resolution of low-resolution images is set to 128×128 so that the corresponding high-resolution image size is 512×512. Learned perceptual patch similarity (LPIPS), Fréchet induction distance (FID), and peak signal-to-noise ratio (PSNR) are used as metrics to measure quantitative performance. In the verification of this embodiment, two degradation cases are concerned: 1. Isotropic Gaussian blur kernel. During the training phase, the blur kernel size is set to 21×21, and the kernel width is uniformly sampled in the range of [0.2,4.0]. During testing, the Gaussian8 kernel set is used, which contains 8 blur kernels in the range of [1.8,3.2]. 2. Anisotropic Gaussian blur kernel. The blur kernel is set to an anisotropic Gaussian kernel of 11×11, with each axis independently randomly distributed in the range of [0.6,5] and randomly rotated in the range of [-π,π]. In order to deviate from the regular Gaussian distribution, a uniform multiplicative noise of up to 25% is applied to each pixel value of the kernel and normalized.

[0145] (1) Evaluation of Isotropic Gaussian Blur Kernel. BlindDiff is quantitatively and qualitatively compared with multiple state-of-the-art methods, including six CNN-based blind super-resolution methods: IKC, DAN, AdaTarget, DCLS, BSRGAN, KDSR, and three diffusion model-based super-resolution methods: DPS, GDP, and DiffPIR. All methods are evaluated on Set5, BSD100, DIV2K100, FFHQ, and ImageNet-1K datasets, which are synthesized by Gaussian8. In the evaluation process, considering the limitations of existing diffusion model methods in handling arbitrary resolutions (such as Set5), a patch-based strategy is adopted in GDP, and adaptive instance normalization is used to obtain more natural super-resolution results. Due to the high computational cost of DPS (out of memory), its performance is only reported on FFHQ and ImageNet-1K. As shown in Table 1, BlindDiff outperforms all methods overall in terms of FID and LPIPS on all datasets, and also significantly exceeds the diffusion model-based methods on all three metrics. Some CNN-based methods, such as DAN and DCLS, achieve higher PSNR than BlindDiff in super-resolution results. This result stems from their reliance on mean squared error (MSE) or mean absolute error (MAE) as loss functions, which are closely related to PSNR but not fully consistent with human visual perception, sometimes resulting in poor visual quality. In contrast, BlindDiff uses a Markov chain to gradually add noise, establishing a clear trajectory from the original data distribution to the simple noise distribution. The probability diffusion process promotes a balance between image diversity and training data fidelity, making the results more perceptually realistic but slightly weaker in pixel perfection. Specifically, despite BSRGAN being trained on paired training data with complex degradation, BlindDiff still outperforms by 0.09 on LPIPS and 36 on FID on ImageNet-1K. DiffPIR shows a significant performance drop when directly applied to blind degraded images. When the GT kernel is present, DPS outperforms DiffPIR, but still underperforms our method in terms of perceptual quality and pixel fidelity. Surprisingly, although GDP adopts a degradation-guided strategy to handle blind degradation, it turns out that it is not suitable for blind super-resolution tasks. Figure 2 The 4x super-resolution results of faces and natural images are shown. Obviously, the super-resolution results of BlindDiff are clearer and reveal more details.

[0146] Table 1 Quantitative comparison of 4x SR on datasets with Gaussian8 kernel.

[0147]

[0148] Evaluation of Anisotropic Gaussian Blur Kernels. BlindDiff was evaluated on the DIV2K100 and ImageNet-1K datasets, where high-resolution images were degraded with random anisotropic Gaussian blur kernels. BlindDiff was compared with 7 CNN methods (DASR, DAN, DCLS, BSRGAN, AdaTarget, DARSR, KDSR) and 3 diffusion model-based methods (DiffPIR, DPS, GD). The quantitative results are shown in Table 2, and it is clear that BlindDiff still outperforms other methods even in this more general and challenging case. Compared with the CNN method DCLS, BlindDiff obtains 38.8% and 21.9% FID improvements on DIV2K100 and ImageNet-1K, respectively. In addition, BlindDiff surpasses the best diffusion model method "GT Kernel+DPS" by 20.85 in terms of FID, while consuming only about 5% of the parameters. In addition, the visual comparison of these methods is shown in Figure 3 As shown in Figure 2, the super-resolution image generated by BlindDiff has more realistic textures, less blur, and clearer boundaries.

[0149] Table 2 Quantitative comparison of 4x SR on datasets with random anisotropic Gaussian kernels

[0150]

[0151]

[0152] Evaluation on real-world images. To verify the effectiveness of BlindDiff in dealing with more complex degradation problems, evaluations are performed on synthetic and real-world datasets. For the synthetic dataset, following the degradation model in BSRGAN, 100 low-resolution (LR) image patches with a resolution of 128×128 are randomly center-cropped from the DIV2K validation set, named DIV2K-real. For the real-world dataset, experiments are conducted on RealSR and RealSRSet, where the LR images in RealSR are also center-cropped to 128×128. BlindDiff is compared with DAN, BSRGAN, DCLS, DASR-real, AdaTarget, KDSR-GAN, DARSR, DiffPIR, and GDP. Since these real images do not have corresponding high-resolution (HR) reference images, three no-reference metrics, namely NIQE, NRQM, and PI, are used for quantitative evaluation, and the results are shown in Table 3. It can be observed that BlindDiff performs better than methods such as DAN, DCLS, and AdaTarget trained for blind Gaussian blur. BSRGAN and KDSR-GAN show advantages in training with high-order complex degradation, especially on the synthetic DIV2K-real dataset. In order to reduce the gap in training samples, BlindDiff-real is constructed, which considers a wider range of degradation factors during training, including downsampling, Gaussian blur, JPEG artifacts, resampling, and noise. The resulting model has significant improvements in all no-reference metric scores and consistently outperforms the comparison methods in most datasets.

[0153] Table 3 Quantitative comparison of 4x SR on real-world datasets

[0154]

[0155] The results of 4 times super resolution are as follows Figure 4 As shown in the figure. It can be observed that the images generated by DAN, DCLS and DiffPIR are often blurred. Although BSRGAN can effectively remove noise and blur, it is more likely to generate super-resolution results with overly smooth textures, resulting in loss of details. In contrast, BlindDiff can restore clearer details and more natural textures. In addition, the super-resolution images reconstructed by BlindDiff-real contain less blur and artifacts.

[0156] In this example, an ablation experiment was performed to verify the effectiveness of each component in BlindDiff. For efficient comparison, all models were pre-trained for 100,000 iterations and tested on Set5.

[0157] (1) Impact of kernel prior. In BlindDiff, MCFormer trained with noise and kernel constraints (Formula 13) is proposed to provide additional kernel prior. In order to study its impact, a baseline model that does not use any of the proposed components is first constructed. As shown in Table 4, when the kernel prior is added but the gradient term based on bicubic interpolation is used (Model A), the model performance decreases due to the difference between the learned kernel and the bicubic interpolation kernel. When combined with the more relevant kernel-aware gradient term (Model B), LPIPS is significantly improved. Continuing to combine it with the kernel modulation operation to form Model C, compared with the baseline model, the results not only prove that the effective use of kernel prior knowledge in the latent space helps better super-resolution reconstruction, but also proves the effectiveness of the modulation operation.

[0158] Table 4 Research on components proposed in BlindDiff

[0159]

[0160] (2) The influence of kernel-aware gradient term. The kernel-aware gradient term is the key to achieving alternating optimization kernels and high-resolution images. First, the comparison between Model A and Model B shows its advantage over bicubic interpolation in processing blind blur kernels. When kernel constraints and modulation are combined in one model (Model D), BlindDiff can fully exploit kernel knowledge to obtain the best LPIPS. To more intuitively illustrate the impact of this component, the kernel-aware gradient term is replaced with the bicubic interpolation-based gradient term in Model D to form Model E. It can be seen that the LPIPS value drops significantly. In a more extreme case, all components are removed and only the bicubic interpolation-based gradient term is added (Model F). The results further demonstrate the negative impact of kernel difference and the necessity of kernel-aware gradient term.

[0161] (3) Influence of kernel-aware gradient term. As shown in Algorithm 2, in the backward process, the kernel-aware gradient is actually calculated and back-propagated with a balanced super-weight λ. Here, the influence of this term is studied by changing the value of λ (from 0 to 10), where λ = 0 means that the model does not consider this gradient back-propagation strategy. The detailed experimental results are listed in Table 5. It can be seen that when λ increases from 0 to 1, the gradient term becomes more important and has a positive impact on better super-resolution image reconstruction. However, when λ is further increased to 10, the model performance decreases due to the over-emphasis on the gradient term. These results verify the effectiveness of the proposed kernel-aware gradient term. In the experiment, λ = 1 is set accordingly.

[0162] Table 5 Impact of kernel-aware gradient terms

[0163]

[0164] (4) Visualization of sampling steps. Since BlindDiff involves alternating optimization of blur kernel and high-resolution image in the posterior sampling process, the intermediate sampling steps are visualized here to study how the number of iterations affects BlindDiff. LPIPS and L1 error are calculated to measure the estimation accuracy of super-resolution image and blur kernel under different numbers of iterations. The results are shown in Figure 2. Figure 5 As shown in the figure, LPIPS and L1 error are reported to measure the estimation accuracy of super-resolution image and blur kernel, respectively. It can be seen that when the number of iterations is greater than 2, BlindDiff remains almost stable and gradually reaches the upper limit. In addition, it is observed that the L1 error between the predicted kernel and the true kernel (GT kernel) also decreases with the iteration, gradually approaching the GT kernel. The visualization of this sampling process verifies that BlindDiff can achieve the alternating optimization of blur kernel estimation and high-resolution image restoration.

[0165] (5) Impact of kernel estimation. To reveal the kernel estimation capability of BlindDiff, the PSNR of low-resolution images is calculated to quantitatively compare the kernel loss. Here, a reference low-resolution image is synthesized using the GT kernel and the high-resolution image, and it is expected that an accurate kernel can reconstruct a similar low-resolution image from the same high-resolution image. Figure 6 In this paper, BlindDiff is compared with existing kernel estimation based blind super-resolution methods including MANet, DIPFKP, KXNet, KULNet and BSRDM. It can be seen that the blur kernel generated by BlindDiff is more consistent with the GT kernel, while other methods produce unreliable results. In addition, equipped with the generated kernel, BlindDiff can reconstruct low-resolution images with the best PSNR performance.

[0166] (6) Extension of blind image deblurring. In order to study the generalization ability of BlindDiff, experiments were conducted on the blind deblurring task and compared with several state-of-the-art deblurring methods (including MPRNet, Restormer and BlindDPS). According to BlindDPS, the motion blur kernel is randomly generated using the code) with a strength of 0.5. All methods are evaluated on the FFHQ dataset, and the quantitative and qualitative results are shown in Table 6. It can be seen that BlindDiff not only achieves the best in terms of pixel perceptual quality (LPIPS / FID) and fidelity (PSNR), but also restores images with more reasonable textures.

[0167] Table 6 Quantitative comparison of blind deblurring on the FFHQ dataset

[0168]

[0169] In this embodiment, the algorithm pseudo codes of the diffusion and reverse processes in BlindDiff are shown in Algorithm 1 and Algorithm 2 as shown in Table 7.

[0170] In Algorithm 1, we input the low-resolution condition y, the scaling factor s, and then repeat the following steps: sampling (x 0 ,y) to obtain data pairs, generate standard normal distribution noise ∈~N(0,I), sample T from uniform distribution, calculate x t The value of x 0 Add noise and estimate blur kernel k t , for θ Perform a gradient descent step to minimize the loss function and perform convergence check.

[0171] In Algorithm 2, we input the low-resolution condition y, the scaling factor s, the sampling condition (y|k,x), and sample the noise x from the standard normal distribution. T Then, for t=T,…,1, the following steps are performed: Generate noise ∈, if t>1, then ∈~N(0,I); otherwise ∈=0, use blur kernel k t calculate Update X t-1 After the loop ends, the final result x is returned. 0 ,k 0

[0172] Table 7

[0173]

[0174] In summary, this embodiment proposes BlindDiff, a super-resolution method based on a generative model, which seamlessly integrates the MAP method into the generative model to solve the blind super-resolution problem. BlindDiff constructs a unique reverse process, which is developed along the MAP method and reversely optimized to achieve alternating optimization of joint blur kernel estimation and high-resolution image restoration. The method of this MAP-driven DDPM method in blind super-resolution is theoretically analyzed. A modulated conditional transformer is proposed, and by introducing a kernel estimation objective and jointly training it with a noise prediction objective, it is able to provide a generative kernel and image prior. A large number of experiments have shown that BlindDiff already has state-of-the-art performance.

[0175] Example 3

[0176] The present embodiment 3 provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the blind image super-resolution reconstruction method as described above is implemented. The method includes:

[0177] Get corrupted low-resolution images;

[0178] The obtained corrupted low-resolution image is processed by using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to obtain prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop for alternating kernel and high-resolution image estimation.

[0179] Example 4

[0180] This embodiment 4 provides a computer device, including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, the processor calls the program instructions to execute the blind image super-resolution reconstruction method as described above, the method comprising:

[0181] Get corrupted low-resolution images;

[0182] The obtained corrupted low-resolution image is processed by using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to obtain prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop for alternating kernel and high-resolution image estimation.

[0183] Example 5

[0184] This embodiment 5 provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes instructions for implementing the blind image super-resolution reconstruction method as described above, the method comprising:

[0185] Get corrupted low-resolution images;

[0186] The obtained corrupted low-resolution image is processed by using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to obtain prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop for alternating kernel and high-resolution image estimation.

[0187] In summary, embodiments of the present invention explore the potential of diffusion models (DM) in blind super-resolution (SR) and propose BlindDiff, a new diffusion super-resolution method that is flexible and robust to various degradations. The method is inspired by the recent denoising diffusion probability model (DDPM) for non-blind inverse problems and is significantly modified to extend it to the blind setting in SISR. Specifically, BlindDiff constructs the blind SR problem under the maximum a posteriori probability (MAP) framework and unfolds with the inverse process of DDPM. Following the MAP optimization paradigm, the key issue now boils down to obtaining prior information of HR clean data and degradation kernels through DDPM. In this case, it is inappropriate to directly apply conventional pre-trained DDPMs to provide generative prior information because they lack modeling of degradation. Therefore, the denoising network is reconfigured and a modulated conditional transformer (MCFormer) trained with noise and kernel constraints is proposed to ensure that prior information of data and kernels is obtained during each posterior sampling process. Then, the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term with respect to the LR condition and two priors is introduced to guide the diffusion model to learn degradation consistency knowledge. Therefore, the denoised image and blur kernel can be jointly optimized by the gradient backpropagated from the intermediate state. In addition, such a gradient term can bridge two adjacent iterations to form a closed loop, performing alternating kernel and HR image estimation until the iterative process approaches its optimal approximation. In addition, MCFormer injects modulation between the extracted kernel knowledge and image features to exploit their interdependence, so that multi-level degradation-aware features can be learned for better generation. The intrinsic combination based on MAP solution makes BlindDiff highly intuitive physical meaning and consistent with the classical degradation model, achieving excellent adaptation to various complex degradations in practical applications.

[0188] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0192] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative work on the basis of the technical solution disclosed in the present invention should be included in the scope of protection of the present invention.

Claims

1. A blind image super-resolution reconstruction method, characterized in that: include: Get corrupted low-resolution images; The obtained corrupted low-resolution image is processed by using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to obtain prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop for alternating kernel and high-resolution image estimation.

2. The blind image super-resolution reconstruction method according to claim 1, characterized in that: Given a corrupted low-resolution image y, reconstructing a high-resolution image x from y and a blur kernel k is formulated as a maximum a posteriori probability problem: p(k,x|y)=p(k|y)p(x|k,y); Here, k and x are solved as follows: Where logp(y|k,x) represents the log-likelihood of y, logp(k|y) represents the kernel prior captured from y, and logp(x) represents the sharp image prior that is independent of y; x0 and k0 represent the potential optimal estimates of x and k, respectively; Decompose the solution of k and x into two sub-problems: Forming an iterative optimization process: Then, the potential SR image is used as auxiliary information to estimate k:

3. The blind image super-resolution reconstruction method according to claim 2, characterized in that: In the denoising probability diffusion model, by training a conditional model, from x T Get a series of denoising priors And model logp(y|k,x) in the reverse sampling process; where, According to the diffusion posterior sampling algorithm for non-blind inverse problem, then: in, is x t The denoised samples, Obtained by the analytical likelihood of the Gaussian measurement model: Therefore, calculating the gradient And back propagate through the network to optimize 4. The blind image super-resolution reconstruction method according to claim 3, characterized in that: The modulation condition transformer consists of a kernel estimator K and a kernel modulation transformer backbone, which is trained by ∈ θ (X t ,t,y) and the benchmark true kernel k gt It is implemented with the L1 loss between the estimated k: Among them, f θ (x0,y,∈) represents the total loss, is the noisy target image, and K takes the bicubic interpolated y and scale s as input.

5. The blind image super-resolution reconstruction method according to claim 4, characterized in that: The kernel estimator encodes image features through four 5×5 convolutional layers and Leaky ReLU activation function; the feature map is globally averaged pooled to form the estimated elements of the blur kernel: Among them↑ s represents bicubic interpolation with scale factor s, k t is the kernel generated at time step t.

6. The blind image super-resolution reconstruction method according to claim 5, characterized in that: The kernel modulated transformer backbone includes: multi-Dconv head transposed attention layer, Gated-Dconv feed-forward network layer, and modulation based on blur kernel and time step; first, an affine transformation is provided for the input feature vector F after layer normalization: in, represents the transformation function, γ1 and τ1 are obtained by fusing t and k t The learnable parameters of the two linear layers are used for scaling and offset operations respectively; It is the characteristic after modulation; exist Apply multiple Multi-Dconv head transposed attention layers to model global connections, and get Among them, γ2 and τ2 are also derived from the fusion of t and k t Learned from features; The Gated-Dconv feed-forward network layer is used to aggregate local features and mine meaningful information from spatially adjacent pixel locations to generate output features.

7. A blind image super-resolution reconstruction system, characterized in that: include: An acquisition module, used for acquiring the damaged low-resolution image; A processing module is used to process the acquired damaged low-resolution image using a pre-trained diffusion super-resolution model to obtain a super-resolution image; wherein the diffusion super-resolution model constructs a blind super-resolution problem under the maximum a posteriori probability framework and unfolds with the inverse process of the denoising probabilistic diffusion model; wherein a modulated conditional transformer trained with noise and kernel constraints is used to obtain prior information of high-resolution pure data and degradation kernels; the inverse diffusion process is regarded as a deep network, and a kernel-aware gradient term relative to the low-resolution condition and two priors is introduced to guide the denoising probabilistic diffusion model to learn degradation consistency knowledge; wherein the kernel-aware gradient term bridges two adjacent iterations to form a closed loop for alternating kernel and high-resolution image estimation.

8. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by the processor, the blind image super-resolution reconstruction method according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the blind image super-resolution reconstruction method according to any one of claims 1 to 6.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the blind image super-resolution reconstruction method as described in any one of claims 1 to 6.

Images