3D Face Tracking Using Local Feature Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing real-time 3D face tracking techniques face challenges in achieving high tracking accuracy while maintaining low memory costs, often being restricted by the use of RGB-Depth (RGB-D) cameras and limited application scenarios.
Innovation Solution
The implementation of local feature regression for real-time 3D face tracking using a general RGB video camera, which involves learning a small but effective random forest to perform local feature regression, improving tracking accuracy and reducing memory costs, enabling broader application scenarios such as game studios and mobile phone applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing real-time 3D face tracking techniques are used, then tracking speed is maintained, but tracking accuracy is insufficient and memory costs are high
Solution Approach 1:
The patent segments the face image into multiple local regions (e.g., forehead, eyes, nose, mouth, chin) and performs regression independently on each region using separate regression trees. This segmentation allows the system to focus computational resources on tracking accuracy for each facial component while using compact region-specific models that reduce overall memory requirements compared to a single comprehensive model.
Solution Approach 2:
The patent applies local quality by using region-specific regression trees tailored to different facial areas. Each regression tree is trained on features relevant to its specific facial region, allowing optimized accuracy for each area. This local specialization enables the system to achieve high overall tracking accuracy while keeping each individual model compact, thereby reducing total memory cost.
2Adaptability or versatility
If RGB-D camera is used for inputting face images, then certain tracking capabilities are achieved, but device complexity and application restrictions increase
Solution Approach 1:
The patent achieves universality by designing a regression-based 3D face tracking system that works with standard RGB cameras rather than requiring specialized RGB-D cameras. The system extracts 3D facial information from 2D RGB images through learned regression models, making the technology universally applicable across different device types including mobile phones, webcams, and consumer cameras, thereby greatly expanding adaptability while reducing device complexity requirements.
Data Source
AI summary
The present disclosure provides method, apparatus and system for 3-dimension (3D) face tracking. The method for 3D face tracking may comprise: obtaining a 2-dimension (2D) face image; performing a local feature regression on the 2D face image to determine 3D face representation parameters corresponding to the 2D face image; and generating a 3D facial mesh and corresponding 2D facial landmarks based on the determined 3D face representation parameters. The present disclosure may improve tracking accuracy and reduce memory cost, and accordingly may be effectively applied in broader application scenarios.