3D Expression Avatar Modeling With Real-Time Facial and Eye Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR systems provide static avatars that cannot be dynamically changed based on user expressions, leading to a poor user experience.
Innovation Solution
A method and apparatus for determining an expression model by acquiring facial and eyeball feature information, classifying expressions, and adjusting parameters to generate a dynamic 3D avatar in real time, enhancing realism and user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a static avatar model is used in VR scenarios, then the system complexity is reduced and rendering is simplified, but the user experience deteriorates due to inability to reflect real-time expressions
Solution Approach 1:
The patent transforms the static avatar model into a dynamic one by introducing real-time expression adjustments. The system continuously acquires facial images and eyeball feature information, classifies expressions, and adjusts 3D model parameters to reflect current user expressions, enabling the avatar to dynamically adapt to user state changes.
Solution Approach 2:
The patent changes the parameters of the 3D avatar model based on expression classification results. By adjusting specific parameters (such as facial feature positions, eye angles, and other expression-related attributes) according to the classified expression type, the system achieves realistic expression rendering without requiring complete model reconstruction.
2Adaptability or versatility
If preset actions and expressions are used for dynamic avatars, then the avatar can change states, but the user experience deteriorates due to limited expression options and inability to freely control
Solution Approach 1:
The system enables the avatar to automatically reflect the user's real-time expressions without requiring manual input. By continuously acquiring facial images and eyeball information, classifying expressions, and automatically adjusting the avatar parameters, the system provides seamless, intuitive interaction where the avatar serves itself based on user state.
Solution Approach 2:
The patent implements a feedback loop where the system continuously monitors user facial expressions and eyeball movements, classifies these expressions, and uses the classification results to adjust the avatar in real-time. This closed-loop feedback mechanism ensures the avatar accurately reflects user state changes.
3Ease of operation
If buttons are used to trigger avatar expressions, then the avatar can respond to user input, but the user experience deteriorates due to unnatural interaction and limited expressiveness
Solution Approach 1:
The patent replaces the mechanical button-based interaction system with an optical/electronic system that captures facial images and eyeball features. This substitution eliminates the need for physical buttons while enabling more natural and expressive interaction through continuous facial expression monitoring and real-time avatar adjustment.
Data Source
AI summary
A method for determining an expression model, an electronic device and a non-transient computer readable storage medium are provided. The method includes: acquiring a facial image and eyeball feature information of a user; determining a corresponding expression classification according to the facial image; determining a corresponding expression model to be adjusted based on the expression classification; adjusting corresponding parameters of the expression model to be adjusted by utilizing the facial image and the eyeball feature information to obtain a target expression model.