An ultra-resolution method based on efficient diffusion bridge and initial value correction strategy

Abstract

The application belongs to the technical field of image super-resolution, and discloses a super-resolution method based on an efficient diffusion bridge and an initial value correction strategy. The method introduces a combination of a non-Markov chain fast sampling and an initial value correction strategy based on a diffusion bridge model, and is used for efficiently reconstructing a high-quality high-resolution image from a low-resolution image. The application can efficiently and high-quality realize an image super-resolution task, and has a good application prospect.

Description

Technical Field

[0001] This invention relates to the field of image super-resolution technology, and more specifically to a super-resolution method based on an efficient diffusion bridge and an initial value correction strategy. Background Technology

[0002] Super-resolution (SR) technology aims to recover high-resolution (HR) images from low-resolution (LR) images and is an important research direction in the field of computer vision. Its core objective is to reconstruct lost high-frequency details through algorithms, thereby improving image clarity and visual quality. SR technology is widely used in video streaming (such as ultra-high-definition video enhancement), medical image analysis (such as CT / MRI image reconstruction), photography (image magnification), remote sensing image processing, and security monitoring.

[0003] With the development of deep learning, scholars have widely adopted deep models to achieve super-resolution tasks. Deep learning models are mainly divided into regression-based methods and generative methods. Regression-based methods (such as Regression) achieve super-resolution through pixel-level optimization, but the generated images are often too smooth, lacking realism and detail. Generative models can generate more realistic images, but they also have limitations. For example, FSRGAN training is unstable and may introduce artifacts.

[0004] In recent years, diffusion models have become a research hotspot in super-resolution tasks due to their powerful generative capabilities. For example, SR3 generates high-quality images with stronger perceptual effects through a progressive denoising process. Furthermore, researchers have proposed the diffusion bridge model (DDBM), which directly models the mapping relationship from low-resolution to high-resolution by introducing an end-to-end bridge structure that uses low-resolution images as target points. However, existing diffusion bridge models are still limited by the sampling efficiency problem of Markov chain structures, and biases in initial value estimation during the sampling process can lead to image quality degradation.

[0005] In summary, the main problems currently facing super-resolution technology include:

[0006] 1. Non-diffusion model methods have limitations in super-resolution performance and other aspects;

[0007] 2. Traditional diffusion bridge models involve numerous sampling steps and have high computational costs;

[0008] 3. The diffusion bridge model suffers from image quality degradation due to initial value estimation bias during rapid sampling.

[0009] Therefore, it is necessary to propose a super-resolution method based on an efficient diffusion bridge and an initial value correction strategy to solve the above problems. Summary of the Invention

[0010] The purpose of this invention is to solve the problems of limitations in super-resolution and other aspects of non-diffusion model methods in the prior art, such as low sampling efficiency of diffusion models, large initial value estimation deviation, and insufficient image detail recovery.

[0011] To achieve the above objectives, the present invention specifically adopts the following technical solution:

[0012] A super-resolution method based on an efficient diffusion bridge and initial value correction strategy includes the following steps:

[0013] S1. Based on the diffusion bridge framework, an initial value correction strategy is adopted. The model training requires multiple iterations. A time step is randomly sampled, and a noisy state variable for that time step is generated based on the estimated value of the previous cycle and the low-resolution image. This state variable is input into the conditional Unet network, and the corrected initial value estimate is output.

[0014] S2. For the initial value estimation in each round of S1, the mean squared error loss is used to supervise the initial value estimation, output the loss value, and update the conditional Unet network parameters by backpropagating the value through gradient. Repeat the above process until the training converges.

[0015] S3. For the model framework trained in the previous step, fast sampling using a non-Markov chain is adopted to break through the continuity constraint of the traditional diffusion model. First, the desired key time step is selected, and each time step is evenly spaced. Then, at each time step, the initial value is estimated through the conditional Unet network, and the initial value is converted into a noise estimate through a reparameterization strategy. Based on the estimate and the endpoint value at time T, the state variable at the next time step is calculated. The above skip sampling process is repeated iteratively to gradually optimize the initial value and finally output a high-quality super-resolution result.

[0016] Furthermore, in step S1, the specific implementation of the initial value correction strategy includes:

[0017] S11. Assume the current training round number is the [number]. Round, total sampling steps are From uniform distribution Random sampling time step ;

[0018] S12. Add noise forward to the high-resolution image to obtain... Degenerate state variables at time 1 ;

[0019] S13, Input state variables Apply the conditional Unet network and output the initial value estimate for this round. ;

[0020] S14. When entering the next round of training, the estimated value... Add noise Time. Acquisition. State variables at time 1 ;

[0021] S15, Input state variables Apply the conditional Unet network and output the initial value estimate for this round. ;

[0022] Step S2 includes step S16, applying the mean square loss function to... The process involves supervision, calculating the loss value, updating the conditional Unet network parameters via gradient backpropagation, and iterating through steps S11 to S16 until the model training converges.

[0023] Furthermore, the mean squared loss error mentioned in step S2 is defined as follows:

[0024]

[0025] in, For true high-definition images, For the denoising network in the first... The estimated value of the cycle, This indicates the current batch size.

[0026] Furthermore, in step S3, the fast sampling of the non-Markov chain can overcome the sampling time burden caused by the continuous sampling limitation of the traditional bridge model. Step S3 includes the following sub-steps:

[0027] S31. Specify the desired number of actual time steps. ;

[0028] S32. Add noise to the current estimated sample up to the time step. ;

[0029] S33. Output the estimated initial value through the conditional Unet network. ;

[0030] S34. Obtain noise estimates through reparameterization. ;

[0031] S35, Based on noise estimation and Obtain the state variables from the previous time step. ;

[0032] S36. Repeat the above process until the last step, and output the final prediction. .

[0033] Further, in step S34, the reparameterization formula is as follows:

[0034]

[0035] in, express The state variable at time t, This represents the high-resolution image estimate predicted at the previous time step. , These are the coefficient terms.

[0036] Further, in step S35, the state variables at the previous specified time are obtained. The expression is as follows:

[0037]

[0038] in, For the variance coefficient term, This is a random noise term that follows a Gaussian distribution.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] This invention introduces a non-Markov chain fast sampling and initial value correction strategy, which can improve image quality while reducing sampling steps, and achieve efficient and high-quality super-resolution reconstruction. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of a super-resolution method based on an efficient diffusion bridge and an initial value correction strategy. Detailed Implementation

[0042] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0043] An initial value correction strategy is used to enhance the supervision of the previous round of initial value estimation when training the diffusion bridge model. The goal of this strategy is to broaden the scope of supervision for the loss function and improve the model's learning performance. Based on this strategy, noise is added to the previous round of estimation and network prediction, so that the HR image estimation of the conditional Unet network includes information from the previous round of estimation, thereby indirectly supervising the results of the two rounds of prediction and increasing the scope of supervision.

[0044] The Diffusion Bridge Model is a probabilistic generation model based on the diffusion process, used for the generation or transformation of high-dimensional data. Its core principle is to map the noise distribution to the target distribution through forward and backward diffusion processes, emphasizing the "bridging" between the two distributions. It is suitable for conditional generation or data interpolation tasks.

[0045] A fast sampling strategy is employed to optimize the inference efficiency of the diffusion bridge model. This strategy aims to reduce the inference time overhead caused by continuous sampling in the bridge model. By implementing this strategy, continuous interaction between the state variables and HQ image estimates at a specified number of steps is achieved, along with skip sampling. This ensures that multi-step sampling expands the model's capacity while reducing the number of sampling steps, thus optimizing the balance between model performance and efficiency.

[0046] Based on the above-mentioned inventive concept, this invention proposes a super-resolution method based on an efficient diffusion bridge and an initial value correction strategy, as shown in the appendix. Figure 1 As shown, the specific steps include:

[0047] S1. Based on the DDBM diffusion bridge framework, an initial value correction strategy is adopted. Model training requires multiple iterations. The model obtains initial value estimates from the model in the previous training iteration. Then, a time step is randomly sampled, and a noisy state variable for that time step is generated based on the estimate from the previous cycle. This state variable is input into the conditional Unet network, and the corrected initial value estimate is output.

[0048] S2. For each round of initial value estimation in S1, mean squared error loss is used to supervise the initial value estimation, output the loss value, and update the conditional Unet network parameters by backpropagating the value through gradient. Repeat the above process until the model training converges.

[0049] S3. For the model framework trained in S2, a non-Markov chain fast sampling strategy is adopted to overcome the continuity constraint of traditional diffusion models. First, the desired key time steps are selected, with each time step spaced evenly. Then, at each time step, the initial value is estimated through a conditional Unet network, and this estimate is converted into a noise estimate through a reparameterization strategy. Based on the estimate and the endpoint value at time T, the state variables at the next time step are calculated. The above skip sampling process is repeated iteratively to gradually optimize the initial value, and finally output a high-quality super-resolution result.

[0050] In step S1, the initial value correction strategy is implemented as follows: assuming the current training round number is the [number]th [round number]... The total number of rounds is [number] steps. First, from a uniform distribution Randomly sample a certain time step Then, forward noise is added to the high-resolution image to obtain... State variables at time steps Input state variables To conditional Unet network, output initial value estimate When entering the next round (the... During (round) training, the estimated value from the previous round Add noise Time step, acquire State variables at time 1 Then, input the state variables. Apply the conditional Unet network and output the initial value estimate for this round. During training, the mean squared loss function is used to... Supervised training is performed, the loss value is calculated, and the conditional UNet network parameters are updated via gradient backpropagation. This process is repeated until the model training converges. The mean squared loss function is defined as follows:

[0051] (1)

[0052] in, For true high-definition images, For the denoising network in the first... The estimated value for each cycle. This indicates the current batch size.

[0053] In step S2 above, fast sampling of the non-Markov chain can overcome the sampling time burden caused by the continuous sampling limitation of the traditional bridge model. Specify the desired actual number of time steps. To implement a fast sampling process for non-Markov chains, firstly, noise is added to the current estimated sample up to time step [number missing]. Subsequently, the initial value estimate is predicted by the conditional Unet network. Next, noise estimates are obtained through reparameterization. The reparameterization formula is shown below:

[0054] (2)

[0055] in, express The state variable at time t, This represents the high-resolution image predicted at the previous time step. , These are the coefficient terms. Next, based on the noise estimate... and Obtain the state variables from the previous time step. The corresponding formula is as follows:

[0056] (3)

[0057] in, This is the variance coefficient term. This represents a random noise term that follows a Gaussian distribution. Repeat the above process until the final step, outputting the final prediction. .

[0058] 1. Experimental setup

[0059] 1.1. Dataset

[0060] To verify the effectiveness of the invention in image super-resolution tasks, the FFHQ face dataset was used as the training set. Then, the invention was evaluated on the CelebA-HQ face dataset. The upsampling ratio was set to 8 (…). Upsampling of low-resolution images High-resolution image).

[0061] 1.2. Measurement Standards

[0062] To comprehensively evaluate the performance of this invention, the following metrics were used: Peak Signal-to-Noise Ratio (PSNR) was used to measure reconstruction accuracy; Structural Similarity Index (SSIM) was used to evaluate structural fidelity and perceived quality; and Consistency was used to calculate the mean square error between the low-resolution image and the downsampled super-resolution image. Among these, higher PSNR and SSIM are better, while lower Consistency is better.

[0063] 1.3. Baseline Model

[0064] To evaluate the performance of this invention, it was compared with state-of-the-art models. The comparison was made with FSRGAN, Regression, SR3, IDM, and DDBM on the CelebA-HQ face dataset.

[0065] 2. Experimental Results

[0066] The experimental results of this invention were compared with those of other benchmark models. Table 1 below shows the comparison of different models on the DIV2K-valid dataset and the same evaluation metrics: PSNR, SSIM, and LPIPS. The best results in each column are highlighted in bold, and NFE indicates the number of evaluation iterations.

[0067] Table 1. Quantitative comparison of different models on the DIV2K-valid dataset.

[0068] Method NFE PSNR↑ SSIM↑ Consistency fsrgan 1 23.01 0.62 33.80 Regression 1 23.96 0.69 2.71 SR3 2000 23.04 0.65 2.68 DDBM 100 24.26 0.70 2.07 Ours 10 24.53 0.74 1.95

[0069] Based on the comparison of the above experimental results, the following observations can be drawn:

[0070] Compared to state-of-the-art methods, this invention offers significant performance improvements. On face datasets, DDBM performs exceptionally well, achieving state-of-the-art (SOTA) results in PSNR, SSIM, and Consistency. It outperforms suboptimal models by 0.27, 0.04, and 0.12 respectively, demonstrating the effectiveness of this invention.

[0071] This invention achieves a good balance between sampling efficiency and performance. Compared with other diffusion methods such as SR3 and DDBM, this invention uses less than 10% of the evaluation iterations to outperform in all three metrics. Even with more evaluation iterations than non-diffusion models, PSNR, SSIM, and Consistency are significantly improved by more than 0.57, 0.05, and 0.83, respectively. The above analysis demonstrates that this invention achieves superior super-resolution performance with fewer evaluation iterations.

[0072] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. The scope of patent protection of the present invention shall be determined by the claims. Similarly, any equivalent structural changes made based on the content of the present invention's specification shall also be included within the scope of protection of the present invention.

Claims

1. A super-resolution method based on efficient diffusion bridge and initial value correction strategy, characterized in that: Comprise the following steps: S1, based on the diffusion bridge framework, using initial value correction strategy, model training needs multiple iterations, randomly sample a time step, and based on the estimated value and low resolution image of the last period, generate the state variable of the time step with noise, input the state variable into the conditional Unet network, and output the corrected initial value estimate; S2, for each round of initial value estimation in S1, the initial value estimation is supervised by the mean square error loss, the loss value is output, and the value is updated by gradient back propagation to update the conditional Unet network parameters, repeat the above process until the training converges; S3, for the model framework trained in the last step, non-Markov chain fast sampling is used, first select the expected key time step, each time step is uniformly spaced, then estimate the initial value through the conditional Unet network at each time, and convert the initial value to a noise estimate value through the reparameterization strategy, based on the estimated value and the endpoint value at T time, calculate the state variable at the next sampling time, repeat the above jump sampling process, gradually optimize the initial value, and finally output high-quality super-resolution results.

2. The super-resolution method based on efficient diffusion bridge and initial value correction strategy according to claim 1, characterized in that: In step S1, the specific implementation of the initial value correction strategy includes: S11. Assume the current training round number is the [number]. Round, total sampling steps are From uniform distribution Random sampling time step ; S12, forward noise adding to the high-resolution image to obtain the degradation state of the moment ; S13, input state variable to the conditional Unet network and output the initial value estimate for this round ; S14, when entering the next round of training, to the estimated value noise to the moment, get the state variable at the moment ; S15, input state variable to the conditional Unet network and output the initial value estimate for this round ; The step S2 includes step S16, using mean square loss function to supervise The loss value is calculated, and the conditional Unet network parameters are updated through gradient back propagation. The steps S11 to S16 are repeated until the model training converges.

3. The super-resolution method based on efficient diffusion bridge and initial value correction strategy according to claim 1, characterized in that: The mean square error loss defined in step S2 is as follows: wherein, is a real high-definition image, is a denoising network at the first period of estimation value, denotes the current batch size.

4. The super-resolution method based on efficient diffusion bridge and initial value correction strategy according to claim 1, characterized in that: In step S3, the following sub-steps are included: S31, specify the desired actual time step number ; S32, forward noise adding to the current estimation sample to the time step ; S33, outputting the estimated initial value by the conditional Unet network ; S34, obtaining a noise estimate by reparameterization ; S35, based on the noise estimate and obtain the state variable of the previous time ; S36, repeat the above process until the last step, output the final prediction .

5. The super-resolution method based on efficient diffusion bridge and initial value correction strategy according to claim 4, characterized in that: In step S34, the reparameterization formula is as follows: wherein denotes state variable at time instant denotes the high resolution image estimate predicted at the previous time instant , are coefficient terms, respectively.

6. The super-resolution method based on efficient diffusion bridge and initial value correction strategy according to claim 5, characterized in that: In step S35, the state variable at the immediately preceding specified time point is obtained The expression of the state variable at the immediately preceding specified time point is as follows: wherein, is a variance coefficient term, is a random noise term subject to a Gaussian distribution.

Images