3D Avatar Morphing via Facial Tracking Control Points
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional 3D model morphing techniques require the inconvenient and non-intuitive application of numerous color-dot tags on the subject's face for facial expression tracking, limiting their practicality and user experience.
Innovation Solution
A method of 3D model morphing driven by facial tracking that eliminates the need for color-dot tags by defining and picking up facial feature and boundary control points from a 3D model, allowing real-time deformation and morphing without the need for physical markers, and incorporating features like pupil movement detection and hair flowing effect simulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If color-dot tags are used for facial expression tracking, then measurement precision is improved, but ease of operation deteriorates due to the need to paste numerous tags on the subject's face
Solution Approach 1:
The patent extracts the tracking function from physical color-dot tags and implements it through virtual control points defined on a 3D model. The system picks up control points from the 3D model that correspond to facial feature locations, eliminating the need to physically attach markers while maintaining tracking capability through camera-based facial feature detection.
Solution Approach 2:
The patent creates a virtual copy of the facial tracking system in the 3D model space. Control points are picked up from the 3D model vertices to correspond with facial feature locations, creating a digital representation that mirrors the physical facial features without requiring physical markers. This virtual copying allows tracking without physical intervention.
2Measurement precision
If numerous color-dot tags are pasted on the subject's face, then measurement precision is improved, but device complexity increases due to the large number of markers required
Solution Approach 1:
The patent removes the physical markers entirely from the system, extracting only the essential tracking function. Instead of using numerous color-dot tags, the system uses a minimal set of control points picked up from the 3D model, significantly reducing the number of tracking elements while maintaining measurement precision through algorithmic facial feature detection.
Solution Approach 2:
The patent replaces the mechanical system of physical tag attachment with an optical and computational system. Control points are picked up from the 3D model and tracked through camera capture and image processing, substituting the mechanical marker-attachment process with a non-contact optical tracking approach that uses facial feature detection algorithms.
3Ease of operation
If control points are picked up from 3D model vertices, then ease of operation is improved by eliminating tag pasting, but manufacturing precision may deteriorate without physical markers
Solution Approach 1:
The patent implements a feedback mechanism where the system captures real-time images, detects facial features, and compares them with the control points picked up from the 3D model. The control points are adjusted based on the detected facial feature positions, creating a closed-loop system that ensures accurate alignment between the virtual control points and actual facial features, thereby maintaining manufacturing precision.
Solution Approach 2:
The patent performs preliminary action by pre-defining control points on the 3D model at locations that correspond to key facial features before actual tracking begins. This preliminary setup allows the system to have accurate reference points ready in advance, ensuring that when tracking starts, the control points are already positioned with the correct spatial relationships, maintaining precision without requiring physical marker placement.
Data Source
AI summary
A method of 3D morphing driven by facial tracking is provided. First, a 3D model is loaded. After that, facial feature control points and boundary control points are picked up respectively. A configure file “A.config” including the facial feature control points and boundary control points data that are picked up corresponding to the 3D avatar is saved. Facial tracking algorithm is started, and then “A.config” is loaded. After that, controlled morphing of a 3D avatar by facial tracking based on “A.config” is performed in real time by a deformation method having control points. Meanwhile, teeth and tongue tracking of the real-time face image, and scaling, translation and rotation of the real-time 3D avatar image is also provided. In addition, a control point reassignment and reconfiguration method, and a pupil movement detection method is also provided in the method of 3D morphing driven by facial tracking.