3D User Model Texture Generation via Weighted Image Reprojection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating textures for 3D user models struggle to capture fine details such as wrinkles, beauty spots, and precise makeup, resulting in a less realistic rendering.
Innovation Solution
The method involves reprojecting one or more images of the user onto a 3D model, merging the reprojected images using a map of merging weights, and removing unwanted shadows and highlights to generate a more realistic texture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a neural network is used to predict texture weights from input images, then the texture generation process is automated, but the fine details such as wrinkles, beauty spots, and precise makeup are lost
Solution Approach 1:
The patent segments the texture synthesis process into two distinct stages: a coarse texture generation stage using neural networks for automation, and a fine detail recovery stage using reprojected images for precision. This segmentation allows each stage to specialize - the neural network handles overall texture synthesis while the reprojected images preserve and inject fine details like wrinkles and makeup, resolving the contradiction between automation and detail fidelity
Solution Approach 2:
The patent merges the outputs of the neural network-based texture synthesis with the reprojected images from multiple input images. By combining these two sources - the automated coarse texture and the detail-rich reprojected images - the system achieves both automation and fine detail preservation simultaneously, as the merged result contains both the overall texture structure and the fine details
2Manufacturing precision
If multiple images are reprojected and merged with weighting, then fine details are captured, but the processing complexity increases
Solution Approach 1:
The patent applies local quality by using spatially varying merging weights that depend on the viewing angle and surface orientation. Different regions of the 3D model are blended from different input images with different weights, allowing fine details to be captured from optimal viewpoints while maintaining processing efficiency through localized rather than global processing of all images
3Stability of the object's composition
If reprojected images are merged using angle-based weights, then viewing angle consistency is improved, but computational requirements increase
Solution Approach 1:
The patent performs preliminary action by pre-computing the merging weights based on viewing angles and surface normals before the actual texture synthesis. This preliminary calculation of angle-based weights allows the system to establish viewing angle consistency upfront, and then use these pre-computed weights during the merging process without requiring intensive real-time computations, thereby reducing computational energy requirements
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A texture is generated for a 3D model of a user by reprojecting one or more images of the user onto the 3D model. When two or more images are reprojected, a map of merging weights is defined for each of the at least two reprojected images, and the at least two reprojected images are merged by applying the map of merging weights defined for each of the at least two reprojected images. In a first embodiment, the merging weight assigned for a given reprojected image is predefined depending on the position of the point on the 3D model. In a second embodiment, a given reprojected image corresponds to a given view of the user from a given angle of view, and, for a point of the 3D model, the merging weight assigned for the given reprojected image is obtained as an increasing function of the alignment between a perpendicular to the 3D model at said point and the given angle of view.