Identity-Preserving Image Generation With Four-Stage Denoising
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image generation methods struggle to maintain consistency between the identity document features of a target object and the semantic information described by the text, leading to suboptimal image generation effects due to poor editing and ID feature retention capabilities.
Innovation Solution
An image generation method involving a four-stage process using denoising networks with specific attention modules at each stage, where the first stage focuses on text features without ID constraints, the second stage integrates text and ID features, the third stage allows free generation without ID constraints, and the fourth stage corrects the target object's location based on ID features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single denoising network is used for image generation, then the generation process is simple, but the ID feature retention and editing capability are poor
Solution Approach 1:
The patent divides the denoising network into four distinct denoising networks (first, second, third, and fourth denoising networks) that operate at different time steps. Each network has specific configurations for attention modules, allowing independent optimization of different generation phases. This segmentation enables the system to maintain simple individual network structures while achieving high ID feature retention through coordinated multi-network operation.
Solution Approach 2:
The patent dynamically adjusts the configuration of attention modules across different denoising networks based on time steps. The first and third denoising networks use cross attention modules for text-guided generation, while the second and fourth use self attention modules for ID feature preservation. This dynamic adaptation allows the system to optimize for different objectives at different stages without requiring a permanently complex unified structure.
2Manufacturing precision
If ID feature is constrained throughout the entire generation process, then ID retention is improved, but the editing capability and text alignment deteriorate
Solution Approach 1:
The patent implements periodic switching between different attention module types at specific time steps. Cross attention modules are activated during first and third time steps for text-aligned generation, while self attention modules are activated during second and fourth time steps for ID feature preservation. This periodic switching enables the system to alternately optimize for text alignment and ID retention, achieving both editing capability and ID feature retention without constant constraint.
Solution Approach 2:
The patent performs preliminary ID feature extraction and stores it before the generation process begins. This pre-extracted ID feature is then selectively applied during specific time steps (second and fourth) when self attention modules are active, allowing the system to prepare ID constraints in advance and apply them only when needed, rather than continuously constraining the generation process.
3Loss of information
If cross attention modules are used at all time steps, then text alignment is improved, but the ID feature retention and generation quality deteriorate
Solution Approach 1:
The patent segments the attention module usage across different denoising networks and time steps. Cross attention modules are selectively applied in the first and third denoising networks for text-aligned generation, while self attention modules are used in the second and fourth denoising networks for ID feature preservation and generation quality optimization. This segmentation prevents continuous cross attention usage while maintaining text alignment during appropriate phases.
Solution Approach 2:
The patent dynamically switches between cross attention and self attention module types based on the current time step and generation phase. The system transitions from cross attention (text-guided) to self attention (ID-guided) and back again, allowing text semantic alignment to be optimized during text-focused phases while preventing degradation of generation quality during ID-focused phases.
Data Source
AI summary
An image generation method, device, and medium are disclosed. The method includes: obtaining to-be-processed noise, a text feature of a target text, an identity document (ID) feature of a target object in a first image, a time step sequence including first time steps, second time steps, third time steps and fourth time steps; processing these first time steps, the to-be-processed noise, and the text feature using a first denoising network to obtain a first result; processing these second time steps, the first result, the text feature, and the ID feature using a second denoising network to obtain a second result; processing these third time steps, the second result, and the text feature using a third denoising network to obtain a third result; and processing these fourth time steps, the third result, the text feature, and the ID feature using a fourth denoising network to obtain a second image.