Neural Image Stylization With Likeness and Pose Preservation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image generation neural networks struggle to adapt the style of a given image while preserving the likeness and appearance of the agent depicted in the image, often modifying the likeness when changing the style or failing to consistently apply the target style.
Innovation Solution
Conditioning an image generation neural network with both a likeness embedding and a pose input to maintain the appearance of the agent, using a denoising neural network to iteratively update the image representation while preserving the agent's likeness and adapting the style as specified by a style prompt.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing image generation neural networks adapt the style of a given image, then the target style is applied to the output image, but the likeness of the agent depicted in the image is modified
Solution Approach 1:
The patent introduces an intermediary component (identity preservation module or reference image mechanism) that mediates between the style adaptation process and the original agent appearance. This intermediary ensures that while style transformations are applied, the core identity characteristics of the agent are maintained and transferred back to the stylized output, thus preventing likeness modification.
Solution Approach 2:
The patent applies different quality requirements to different parts of the image: the background and stylistic elements are transformed to match the target style, while the agent's facial features and identity-critical regions are preserved with high fidelity. This local differentiation allows style adaptation without compromising agent likeness.
2Measurement precision
If existing image generation neural networks maintain consistent likeness across input and output images, then the appearance of the agent is preserved, but the target style cannot be consistently applied
Solution Approach 1:
The patent segments the image processing into distinct components: one handling style transfer and another handling identity preservation. By separating these functions and then integrating them through the intermediary mechanism, the system achieves both consistent likeness preservation and effective style adaptation simultaneously.
Solution Approach 2:
The patent creates a composite output that combines elements from both the original image (agent likeness) and the target style image. This composite approach allows the final output to contain both the preserved agent appearance and the applied target style, resolving the contradiction between maintaining likeness and adapting style.
3Measurement precision
If the system generates discriminative information to preserve agent appearance, then the likeness is maintained, but privacy and security risks increase
Solution Approach 1:
The patent extracts only the essential identity-critical features of the agent that are necessary for maintaining likeness, rather than preserving or generating complete discriminative information about the agent's appearance. This extraction approach maintains sufficient likeness for recognition purposes while minimizing privacy risks by removing unnecessary identifying details.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating inputs using denoising neural networks. One of the methods includes receiving an input comprising an original image of a first agent; obtaining a style prompt representing a target style of a stylized image generated from the original image; generating, from the original image, a pose input that represents a pose of the first agent in the original image; generating, from the original image, a likeness embedding that represents a likeness of the first agent in the original image; and processing the style prompt, the pose input, and the likeness embedding using an image generation neural network to generate the stylized image that depicts the first agent in the target style.