Two-Stage Diffusion Upscaling for High-Resolution Image Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional diffusion models face inefficiencies in generating high-resolution images due to increased processing time and memory costs, and upsampling noisy intermediate images introduces noise-related inaccuracies.
Innovation Solution
Generate a noiseless prediction at a low resolution using a first diffusion model, upsample it, and then use a second diffusion model to create a high-resolution output, avoiding noise correction steps and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If one diffusion model generates high-resolution output images directly, then image quality is improved, but processing time and memory costs increase significantly
Solution Approach 1:
The patent divides the image generation process into two separate diffusion models: a first diffusion model that generates low-resolution images, and a second diffusion model that generates high-resolution images from upsampled versions of the low-resolution images. This segmentation allows each model to specialize in a specific resolution range, reducing the computational burden on any single model while maintaining high final image quality.
Solution Approach 2:
The patent introduces a resolution dimension to the generation process by training the second diffusion model to conditionally generate high-resolution images based on upsampled low-resolution inputs. This dimensional approach allows the system to efficiently progress from low to high resolution through a structured two-stage process rather than attempting single-step high-resolution generation.
2Manufacturing precision
If one diffusion model generates high-resolution output images directly, then image quality is improved, but memory costs increase significantly
Solution Approach 1:
The patent segments the high-resolution generation task across two specialized models. The first model handles low-resolution generation with minimal memory requirements, while the second model handles high-resolution generation from upsampled inputs. This segmentation prevents any single model from requiring excessive memory to handle the full high-resolution generation task alone.
Solution Approach 2:
The patent performs preliminary low-resolution image generation and upsampling before the high-resolution generation stage. This preliminary action prepares the input data in advance, allowing the second diffusion model to focus computational resources on refining details rather than generating the entire image from scratch at high resolution, thereby reducing peak memory requirements.
3Length of moving object
If conventional methods upsample noisy intermediate images, then high-resolution output is achieved, but noise-related inaccuracies are introduced
Solution Approach 1:
The patent extracts and removes noise from the intermediate image before upscaling by using the first diffusion model to generate a denoised low-resolution image. This extracted clean representation serves as the input for the second model, preventing noise amplification during the upscaling process and maintaining image accuracy throughout the resolution transition.
Solution Approach 2:
The patent performs noise removal as a preliminary action before the upscaling and high-resolution generation steps. By cleaning the image data in advance at low resolution, the system avoids carrying noise through subsequent processing stages where it would be amplified, ensuring higher fidelity in the final high-resolution output.
Data Source
AI summary
Methods, non-transitory computer readable media, apparatuses, and systems for high-resolution image generation using diffusion models include obtaining a prompt and generating, using a first diffusion model, a predicted denoised image at a first resolution based on the prompt. The predicted denoised image is generated at a first intermediate diffusion step of the first diffusion model. The predicted denoised image is upsampled to obtain an upsampled denoised image at a second resolution that is higher than the first resolution. A second diffusion model then generates an output image at the second resolution based on the prompt and the upsampled denoised image.