Personalized 3D Face Model Generation from Multiple 2D Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D face modeling methods, such as those using 3D depth cameras and scanners, face challenges in accurately representing face pose, illumination, and expression, especially when dealing with multiple viewpoints, leading to less precise and realistic models compared to 2D modeling.
Innovation Solution
A method that generates a personalized 3D face model by extracting feature points from multiple 2D images, deforming a generic 3D face model to match these points, and refining the shape based on texture patterns to minimize differences across various views, using a combination of hardware and software components like processors and memory to execute computer-readable instructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a 3D depth camera or 3D scanner is used to model a 3D face, then 3D face modeling can be performed, but the accuracy in representing face pose, illumination, and expression is insufficient compared to 2D modeling
Solution Approach 1:
The patent creates a virtual 3D face model that copies and reconstructs the actual face geometry and appearance from multiple 2D images. By projecting texture maps onto a 3D mesh structure, the system generates a realistic digital replica that preserves facial features, pose, and expression details more accurately than direct 3D scanning methods
Solution Approach 2:
The patent transitions from 2D image data to 3D representation by constructing a three-dimensional mesh model from multiple two-dimensional images taken from different viewpoints. This dimensional transformation enables accurate representation of face pose and expression while maintaining the rich texture and color information from the original 2D photographs
2Manufacturing precision
If multiple 2D images are used to generate a personalized 3D face model, then realism and precision are improved, but the complexity of the modeling process increases
Solution Approach 1:
The patent divides the complex modeling process into distinct segments: feature point extraction from multiple images, 3D mesh construction from extracted points, texture map generation from image regions, and final model assembly. This segmentation allows each step to be processed independently and simplifies the overall workflow while maintaining high precision
Solution Approach 2:
The patent performs preliminary feature point extraction and landmark detection on multiple 2D images before constructing the final 3D model. By pre-processing the images to identify key facial features and structural points, the system simplifies subsequent 3D reconstruction steps and improves modeling efficiency
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A method of generating a three-dimensional (3D) face model includes extracting feature points of a face from input images comprising a first face image and a second face image; deforming a generic 3D face model to a personalized 3D face model based on the feature points; projecting the personalized 3D face model to each of the first face image and the second face image; and refining the personalized 3D face model based on a difference in texture patterns between the first face image to which the personalized 3D face model is projected and the second face image to which the personalized 3D face model is projected.