Distortion-Based Rendering for High-Resolution Digital Human ROIs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI models struggle to generate high-resolution images of small regions, such as the mouth, in digital humans due to sparse data and dominance of other facial features, leading to anomalous artifacts and poor rendering of these regions, especially in life-sized digital humans.
Innovation Solution
The method involves distorting control images to enhance regions of interest (ROIs) like the mouth by reconfiguring and augmenting pixels, using a distortion function, and training a generative AI model to generate distorted images. A reverse distortion function then generates high-resolution undistorted images, reallocating memory to enhance ROI data without increasing overall memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a generative AI model is trained on standard control images, then the overall image generation is efficient, but the resolution and quality of small regions of interest (such as the mouth) are poor due to sparse data and dominance of other facial features
Solution Approach 1:
The patent segments the control image into multiple regions of interest (ROIs) such as the mouth, eyes, and other facial features. Each ROI is independently distorted and augmented to enhance its data density. This segmentation allows the generative AI model to focus computational resources on specific small regions that require high resolution, rather than treating the entire image uniformly, thereby improving ROI rendering without proportionally increasing overall complexity.
Solution Approach 2:
The patent applies local quality enhancement by selectively distorting and augmenting only the regions of interest rather than the entire control image. The distortion function locally transforms coordinates within each ROI to expand its representation in the latent space. This approach concentrates computational effort where it is most needed (in small ROIs) while maintaining efficiency in other regions, resolving the contradiction between precision and complexity.
2Manufacturing precision
If the control image is distorted to enhance ROI data density, then high-resolution rendering of small regions is achieved, but the overall image structure and proportions are altered
Solution Approach 1:
The patent applies preliminary distortion and augmentation to the control image before feeding it to the generative AI model. By pre-distorting the control image to enhance ROI data density, the model learns the enhanced representations during training. The reverse distortion is then applied during inference to restore proper proportions. This preliminary action allows the system to benefit from enhanced ROI data density while ensuring that final output images maintain correct anatomical proportions.
Solution Approach 2:
The patent employs an inversion strategy by applying a reverse distortion function to the generated images. During training, the control image is distorted to enhance ROI representation. During inference, after the generative model produces an image from the distorted latent representation, the reverse distortion function restores the original image proportions and structure. This inversion approach allows the system to temporarily alter proportions for training purposes while ensuring final outputs maintain correct anatomy.
3Manufacturing precision
If a reverse distortion function is applied to generated images, then high-resolution undistorted images are produced, but additional computational processing is required
Solution Approach 1:
The patent performs preliminary distortion of the control image during the training phase, allowing the generative AI model to learn the distorted representations. The reverse distortion operation is then applied only during the inference phase to restore proper proportions. By shifting the heavy distortion and augmentation work to the training phase (preliminary action), the system achieves high-resolution output without significantly impacting inference speed, as the reverse distortion is a relatively lightweight post-processing step compared to full image generation.
Data Source
AI summary
Synthesizing high-resolution input for rendering a digital human includes generating, with a generative artificial intelligence (AI) model, a distorted image of the digital human by enhancing a region of interest (ROI) within the distorted image relative to other regions of the distorted image. The generative AI model is previously trained against a distorted control image generated using a distortion function to distort a control image used to guide image generation by the generative AI model. The distorted control image is generated by reconfiguring and augmenting pixels of the control image. An undistorted image of the digital human is generated using a reverse distortion function to reverse distortion of the distorted image.