Facial Features Tracker Using 35 Orientation Classes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional constrained local models (CLM) for facial feature tracking have limitations in handling wide head rotations, leading to occluded landmark points and increased complexity, especially when dealing with large databases for training, which is inefficient and costly.
Innovation Solution
A facial features tracker system that utilizes a comprehensive set of 3D head orientation classes, each with a respective 3D deformable model, allowing for real-time tracking and animation of facial features by selecting the appropriate head orientation class based on yaw and pitch angles, and switching between classes seamlessly, reducing the need for extensive database construction and improving runtime efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional CLM schemes use limited head rotation classes to reduce training complexity, then training cost and device complexity are reduced, but tracking accuracy deteriorates for wide head rotations and landmark points become occluded
Solution Approach 1:
The patent divides the continuous space of head rotations into a discrete set of 35 orientation classes, each representing a specific yaw and pitch angle range. This segmentation allows the system to handle wide head rotations by selecting the appropriate pre-trained model class, avoiding the need for a single overly complex model while maintaining tracking accuracy across diverse poses.
Solution Approach 2:
The patent pre-trains separate CLM models for each of the 35 head orientation classes before runtime. This preliminary action allows the system to have ready-to-use specialized models for different head poses, eliminating the need for complex runtime adjustments and enabling accurate tracking without requiring an enormous training database for all possible rotations.
2Adaptability or versatility
If the number of head rotation classes is increased to cover wide rotations, then tracking coverage and adaptability are improved, but training work and database construction become enormous and cost ineffective
Solution Approach 1:
The patent systematically varies the head orientation parameters (yaw and pitch angles) to define 35 distinct classes that comprehensively cover wide head rotations. By organizing models according to these parameter changes, the system achieves high adaptability while keeping training manageable, as each class is trained on a focused subset of data rather than requiring an enormous database for all possible rotations.
3Measurement precision
If conventional schemes use larger databases and more machine learning to handle wide rotations, then tracking accuracy is improved, but runtime speed and simplicity deteriorate
Solution Approach 1:
The patent performs model selection and parameter initialization in advance by organizing pre-trained models into 35 orientation classes. At runtime, the system quickly determines the appropriate class based on detected head orientation and activates the corresponding pre-trained model, avoiding the computational burden of training or selecting from large databases during runtime, thus maintaining both accuracy and speed.
Solution Approach 2:
The patent implements dynamic switching between different head orientation classes based on real-time detection of yaw and pitch angles. This dynamic adaptation allows the system to maintain high tracking accuracy across varying head poses while keeping runtime operations simple and efficient, as the model selection is based on straightforward angular measurements rather than complex computations.
Data Source
AI summary
Tracking units for facial features with advanced training for natural rendering of human faces in real-time are provided. An example device receives a video stream, and upon detecting a visual face, selects a 3D model from a comprehensive set of head orientation classes. The device determines modifications to the selected 3D model to describe the face, then projects a 2D model of tracking points of facial features based on the 3D model, and controls, actuates, or animates hardware based on the facial features tracking points. The device can switch among an example comprehensive set of 35 different head orientation classes for each video frame, based on suggestions computed from a previous video frame or from yaw and pitch angles of the visual head orientation. Each class of the comprehensive set is trained separately based on a respective collection of automatically marked images for that head orientation class.