Face Tracking Module Visibility-Based Vertex Exclusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional face tracking systems for virtual and augmented reality experience accuracy and latency issues due to incorrect association of face model vertices with depth images, leading to unsatisfying user interactions.
Innovation Solution
A face tracking module that estimates the visibility of face model mesh vertices from a depth camera's point of view, excluding non-visible vertices to improve pose estimation accuracy by fitting a generative face model mesh to a depth map, using identity and expression coefficients to non-rigidly deform the mesh and minimizing an energy function based on visible vertex distances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all face model vertices are associated with depth images, then the face model can be fully fitted, but measurement precision deteriorates due to incorrect associations of non-visible vertices
Solution Approach 1:
The patent extracts and excludes non-visible vertices from the face model that cannot be properly associated with depth image data. By identifying and removing these problematic vertices (those that are occluded or not visible from the camera viewpoint), the system eliminates sources of incorrect association while retaining only the visible vertices that can be accurately matched, thereby improving pose estimation precision without excessive complexity
Solution Approach 2:
The patent segments the face model vertices into visible and non-visible groups based on their visibility from the depth camera viewpoint. This segmentation allows the system to process only the relevant visible vertices for pose estimation, avoiding the computational burden and errors associated with attempting to associate all vertices including those that are occluded
2Reliability
If conventional face tracking is used, then face tracking functionality is provided, but latency increases resulting in unsatisfying user experience
Solution Approach 1:
The patent extracts and removes non-visible vertices from the processing pipeline, reducing the number of vertices that need to be associated and processed. This extraction of unnecessary computational elements directly reduces processing time and latency while maintaining tracking reliability, as the system focuses only on the visible vertices that actually contribute to accurate pose estimation
Solution Approach 2:
The patent applies partial action by processing only the subset of visible vertices rather than all vertices. This partial processing approach is sufficient to achieve accurate pose estimation without the excessive computation required to process every vertex, thereby reducing latency while maintaining tracking functionality and reliability
Data Source
AI summary
An electronic device estimates a pose of a face by fitting a generative face model mesh to a depth map based on vertices of the face model mesh that are estimated to be visible from the point of view of a depth camera. A face tracking module of the electronic device receives a depth image of a face from a depth camera and generates a depth map of the face based on the depth image. The face tracking module identifies a pose of the face by fitting a face model mesh to the pixels of a depth map that correspond to the vertices of the face model mesh that are estimated to be visible from the point of view of the depth camera.