Diffusion Super-Resolution Training With Rectified HR Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current super-resolution (SR) methods face challenges in generating high-quality images due to training-sampling discrepancies and estimation errors, particularly in the use of ground truth HR images, leading to blurred and deficient details in predicted images.
Innovation Solution
The DREAM technology employs diffusion rectification and estimation-adaptation, using an estimated HR image instead of a ground truth HR image as the seed, and introduces estimation error to the training process, aligning training and sampling phases by smoothly shifting focus between standard diffusion and rectification based on estimation error trends.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If ground truth HR image is used as seed to forward noise addition process during training, then training process can account for estimation error, but sampling process involves estimation error because ground truth image is not available during sampling
Solution Approach 1:
The patent inverts the conventional approach by using the model's own estimated HR image (rather than ground truth) as the seed for forward noise addition during training. This reversal aligns the training process with the sampling process, both of which then operate with estimated images, eliminating the training-sampling discrepancy.
Solution Approach 2:
The patent implements a feedback mechanism where the model's own predictions are fed back into the training process. The estimated HR image generated by the model during sampling is used as the seed for forward diffusion, and this same estimated image guides the training process, creating a self-consistent loop that reduces estimation errors.
2Measurement precision
If ground truth HR image is used during training, then training data is accurate, but model fails to generalize to sampling where ground truth is unavailable
Solution Approach 1:
The patent performs preliminary action by generating the estimated HR image through the model's own prediction before the forward diffusion process begins. This estimated image, rather than ground truth, is prepared in advance as the seed, ensuring the model learns to operate under the same constraints it will face during sampling.
Solution Approach 2:
The patent changes the critical parameter used as input seed from ground truth HR image to model-generated estimated HR image. This parameter change transforms the training objective from learning to match ground truth to learning to maintain consistency with its own predictions, thereby improving generalization to sampling scenarios.
3Device complexity
If standard diffusion training is used, then training is simple, but model produces blurred images with deficient high frequency content
Solution Approach 1:
The patent introduces an intermediary element: the model's own estimated HR image serves as a mediator between the ground truth and the forward diffusion process. This intermediary guides the training to produce images that are consistent with the model's own predictions, thereby preserving high frequency content and sharpness that standard diffusion training loses.
Data Source
AI summary
The technology described herein provides an improved training framework for a diffusion model used for a super resolution (SR) task. In particular, the technology provides diffusion rectification to correct a training-sampling discrepancy inherent in current training methods. The technology also provides estimation-adaptation. The diffusion rectification portion of the technology uses an estimated HR image, rather than a ground truth HR image as the seed to the forward process. This improves model performance issues caused by a training-sampling discrepancy. The training-sampling discrepancy occurs because the training and sampling processes do not use the same data. The estimation adaption strategy injects ground truth to the plurality of noisy images to reduce the training-estimation error in the images. In an aspect, a different amount of ground truth is injected into training images based on the training image's location in the Markov chain.