3D Face Reconstruction Using Thin Plate Spline Mesh Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for 3D face reconstruction from 2D images, such as 3D Morphable Models and Generic Elastic Models, require manual initialization and are inefficient due to their reliance on loop subdivision for mesh refinement, leading to non-homogeneous vertex distributions and accuracy issues.
Innovation Solution
A novel refinement technique using interpolating splines to create a dense 2D correspondence map parameterized by a thin plate spline framework, which does not move initial fiducial points and results in a more homogeneous vertex distribution, combined with linear data-completion algorithms like WGHA and WK-SVD for accurate depth recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If loop subdivision is used for mesh refinement, then the mesh is densified, but the vertex distribution becomes non-homogeneous and initial fiducial points are moved
Solution Approach 1:
The patent creates a copy of the initial triangular mesh and uses this copy for densification operations. The original mesh with its fiducial points remains unchanged, while the copied mesh is subdivided to create additional vertices. This allows mesh densification without moving or altering the positions of initial fiducial points, resolving the contradiction between increasing mesh density and maintaining vertex distribution homogeneity.
2Measurement precision
If 3D Morphable Models are used for reconstruction, then accurate 3D face models can be obtained, but manual initialization and iterative fitting are required making it slow
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing correspondence relationships between 2D image points and 3D model points in a lookup table. During actual reconstruction, the system directly queries this pre-computed correspondence data rather than performing iterative optimization. This preliminary preparation eliminates the need for slow iterative fitting while maintaining reconstruction accuracy, enabling real-time performance.
3Measurement precision
If manual initialization is used for 3DMM fitting, then accurate parameter fitting can be achieved, but the process becomes tedious and requires user oversight
Solution Approach 1:
The patent implements self-service by automatically establishing correspondence between 2D and 3D points using the pre-computed lookup table, eliminating the need for manual initialization. The system autonomously queries the correspondence data and performs reconstruction without user intervention or oversight. This automated approach maintains fitting accuracy while dramatically simplifying operation and removing the tedious manual initialization step.
Data Source
AI summary
A system and method for providing a fast, efficient and accurate method for recovery of depth information from a single image of a face is disclosed. The technique uses a novel thin-plate spline-based dense-correspondence method to align the face, and the representation incorporates a weighted framework, interpreting the depth recovery problem as a weighted data problem.