Eye Expression Driving With Eyelid Spacing and Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to accurately simulate facial expressions, particularly eye expressions, when identity and posture information is unavailable, such as when using virtual reality glasses.
Innovation Solution
Determine expression coefficients, including a first eye coefficient for eyelid spacing and a second eye coefficient for line-of-sight information, using key point identification and line-of-sight tracking algorithms, to drive avatars without requiring identity or posture information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional facial expression simulation methods are used, then identity and posture information can be obtained, but the method cannot work when using virtual reality glasses where such information is unavailable
Solution Approach 1:
The patent extracts only the essential eye region information (eyelid spacing and line-of-sight) from the full facial image, discarding the need for identity and posture data. This allows the system to work with limited input from virtual reality glasses while maintaining expression simulation capability.
Solution Approach 2:
The patent introduces an intermediary processing stage that converts limited eye region data into expression coefficients through key point identification and line-of-sight tracking algorithms, bridging the gap between incomplete input and reliable expression output.
2Measurement precision
If 3D reconstruction is used to improve expression simulation accuracy, then more detailed facial information can be obtained, but the processing complexity and time consumption increase significantly
Solution Approach 1:
The patent extracts only the critical eye region features (eyelid spacing and line-of-sight direction) rather than performing full 3D reconstruction of the face. This selective extraction maintains expression precision while dramatically reducing system complexity and processing requirements.
Solution Approach 2:
The patent segments the facial analysis task into a focused eye region analysis, separating it from the need for complete facial 3D reconstruction. This segmentation allows precise expression coefficient calculation using only eye region key points and line-of-sight tracking.
3Loss of information
If full facial image processing is performed, then comprehensive expression information can be obtained, but the processing time and computational resources increase
Solution Approach 1:
The patent extracts only the essential eye region information needed for expression simulation, obtaining complete expression data from the eye region alone without processing the entire facial image. This maintains information completeness while reducing processing time.
Solution Approach 2:
The patent segments the processing task to focus exclusively on the eye region, dividing it into key point identification and line-of-sight tracking sub-tasks. This segmentation enables rapid processing while capturing all necessary expression information.
Data Source
Figure 1~3
Figure 4~6
AI summary
The present disclosure provides an expression driving method, apparatus, device and storage medium, wherein the method includes: acquiring an image to be processed of a target object, the image to be processed including an eye region of the target object; determining an expression coefficient of at least one dimension of the target object based on the image to be processed, the expression coefficient of at least one dimension including a first eye coefficient and a second eye coefficient, wherein the first eye coefficient is configured for representing an eyelid spacing of the target object, and the second eye coefficient is configured for representing line-of-sight information of the target object; and driving an avatar based on the determined expression coefficient of at least one dimension.