Two-Stage Image Generation Training for High-Resolution Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image generation models struggle to produce high-resolution images that align with user expectations, particularly in super-resolution tasks, due to issues with model selection, signal-to-noise ratio, and sampling strategy, leading to suboptimal performance.
Innovation Solution
A two-stage training process is employed, starting with a low-resolution image generation model that is fine-tuned using second training data and then further refined with a reward model to enhance high-resolution image generation capabilities, incorporating diffusion models and graphics memory optimization techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a single-stage high-resolution image generation model is trained directly, then the model can generate high-resolution images, but the performance and alignment with user expectations are suboptimal
Solution Approach 1:
The patent divides the image generation task into two separate models: a low-resolution image generation model and a high-resolution image generation model. The low-resolution model first generates an initial image, which then serves as input for the high-resolution model to produce the final high-quality image. This segmentation allows each model to specialize in its respective resolution range, improving overall generation quality and user alignment.
Solution Approach 2:
The patent implements a two-stage training process where the low-resolution model is trained first to establish a foundation, and then the high-resolution model is trained using the low-resolution model's outputs as additional training data. This preliminary action enables the high-resolution model to learn from both low and high-resolution examples, enhancing its ability to generate images that meet user expectations.
2Device complexity
If the first image generation model is directly used for high-resolution generation, then the model structure is simple, but the generation performance is suboptimal
Solution Approach 1:
The patent segments the image generation capability into two distinct models with different resolution specializations. Rather than making one complex model handle all resolutions, it creates a simple low-resolution model and a simple high-resolution model that work sequentially. This segmentation maintains individual model simplicity while achieving superior overall generation quality through their collaborative workflow.
3Ease of manufacture
If traditional training methods are used for high-resolution generation, then the training process is straightforward, but the super-resolution performance is insufficient
Solution Approach 1:
The patent implements a preliminary training stage where the low-resolution model is trained first using standard methods. Then, in a second training stage, the low-resolution model's generated images are combined with high-resolution ground truth images to train the high-resolution model. This two-stage preliminary action allows the system to build upon established training results while achieving superior super-resolution performance.
Solution Approach 2:
The low-resolution model serves as an intermediary in the training process. It generates intermediate images that are then used as additional training data for the high-resolution model. This intermediary role allows the high-resolution model to learn from both directly observed high-resolution images and synthesized low-resolution-to-high-resolution transformations, improving super-resolution capability.
Data Source
AI summary
The embodiments of the disclosure provide a method, apparatus, device, and storage medium for image generation. The method includes obtaining a trained first image generation model, the first image generation model being configured to generate an image having a first resolution. A second image generation model is obtained by training the first image generation model using second training data, the second training data includes an image having a second resolution, the second image generation model is configured to generate an image having the second resolution, and the second resolution is higher than the first resolution.The second image generation model is trained with a second reward model.