High-Resolution Avatar Mesh Generation via Position Map
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current avatar rendering systems face challenges in reproducing detailed features, such as a protruding tongue, due to low-resolution mesh geometry and missing geometry information, which results in inaccurate rendered images.
Innovation Solution
A method is introduced to generate a high-resolution mesh using a machine-learning model that creates a position map with three-dimensional coordinates, allowing for the generation of a dense mesh with sufficient geometry information, and a warp field to realign textures, ensuring accurate and detailed rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a primary mesh is generated from input images, then the mesh can be created with available geometry information, but the mesh resolution is low and lacks detailed facial features
Solution Approach 1:
A position map is introduced as an intermediary data structure between the input images and the final high-resolution mesh. The position map contains dense 3D coordinate information that serves as a bridge to generate the detailed mesh geometry, enabling high-resolution output without directly processing the limited input image data
Solution Approach 2:
The patent transitions from 2D input images to a 3D position map with dense coordinate information, then to a high-resolution 3D mesh. This dimensional transformation and densification of coordinate data enables recovery of fine geometric details that were not explicitly present in the original 2D images
2Loss of information
If textures from different views are used, then appearance information can be captured, but the textures cannot compensate for missing geometry details
Solution Approach 1:
The patent separates the geometry generation and texture mapping processes into distinct stages. First, the high-resolution mesh geometry is generated from the position map, ensuring accurate 3D structure. Then, textures are mapped onto this pre-established geometry, allowing each process to optimize for its specific purpose without compromising the other
3Manufacturing precision
If a dense mesh is generated from a position map, then high-resolution geometry can be achieved, but computational complexity increases
Solution Approach 1:
The position map is generated in advance as a pre-processed data structure containing all necessary 3D coordinate information. This preliminary computation consolidates the complex geometry generation task before mesh creation, allowing the actual mesh generation process to proceed more efficiently with the pre-organized position data
Data Source
AI summary
In one embodiment, a system may access an input image of an object captured by cameras, and the input image depicts appearance information associated with an object. The system may generate a first mesh of the object based on features identified from the input image of the object. The system may generate, by processing the first mesh using a machine-learning model, a position map that defines a contour of the object. Each pixel in the position map corresponds to a three-dimensional coordinate. The system may further generate a second mesh based on the position map, wherein the second mesh has a higher resolution than the first mesh. The system may render an output image of the object based on the second mesh. The system disclosed in the present application can render a dense mesh which has a higher resolution to provide details which cannot be compensated by texture information.