Avatar Expression Tracking via Facial Feature Point Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional electronic devices cannot create avatars that make natural and delicate facial expressions that accurately follow variations in a user's facial expressions, failing to provide lifelike representations suited for the characteristics of avatars.
Innovation Solution
An electronic device capable of creating avatars with natural and delicate facial expressions using a plurality of feature points on a user's face, employing a processor to track variations in facial expressions and alter the avatar's expressions in real-time by determining the degree of variation in feature points and applying weights to reference models to generate corresponding motions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional electronic devices use simple facial recognition to create avatars, then the device complexity is low, but the avatar cannot make natural and delicate facial expressions that follow variations in the user's facial expression
Solution Approach 1:
The patent segments the facial expression recognition into multiple feature points (e.g., eyes, eyebrows, mouth corners, nose) rather than treating the face as a single unit. Each feature point is tracked independently to capture subtle variations in facial expressions, enabling the avatar to replicate delicate expressions accurately.
Solution Approach 2:
The patent transitions from 2D image processing to 3D spatial coordinate analysis by establishing a three-dimensional coordinate system for each facial feature point. This dimensional enhancement allows for more precise tracking of facial movements and enables the avatar to express nuanced emotions in three-dimensional space.
2Measurement precision
If conventional electronic devices track facial expressions using basic methods, then the processing speed is fast, but the avatar expressions are not natural and delicate
Solution Approach 1:
The patent performs preliminary actions by pre-establishing a three-dimensional coordinate system for multiple facial feature points and pre-defining the spatial relationships between them. This preparation enables real-time tracking without requiring complex calculations during expression recognition, maintaining fast processing speed while achieving high precision.
Solution Approach 2:
The patent creates a digital copy of the user's facial feature points and their spatial relationships in a virtual three-dimensional space. This copy allows for efficient processing and manipulation of facial expression data without requiring continuous analysis of the original complex facial structures, reducing processing time while maintaining accuracy.
3Adaptability or versatility
If conventional electronic devices use single-point facial tracking, then the system is simple to operate, but the avatar cannot make lifelike expressions suited for the characteristics of avatars
Solution Approach 1:
The patent creates a universal three-dimensional coordinate system that can track multiple facial feature points simultaneously and adapt to various facial expressions. This multi-functional system can handle different types of expressions (smiling, frowning, surprised, etc.) using the same framework, making the avatar highly adaptable while managing complexity through a unified approach.
Solution Approach 2:
The patent changes the parameters of facial tracking from simple 2D coordinates to 3D spatial coordinates with multiple feature points. By adjusting the number and positions of tracked feature points based on the required expression complexity, the system achieves high adaptability while controlling complexity through parameter optimization.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
According to an embodiment, an electronic device comprises a camera, a display, and a processor configured to control the electronic device to: obtain a plurality of images including a first image and a second image corresponding to a user's face using the camera, display, on the display, a first avatar selected from among at least one 3D avatar including model information related to a motion and created based on 3D modeling, determine a degree of variation in at least some feature points among the plurality of feature points of the face based on a comparison between the plurality of feature points of the face included in each of the first image and the second image, determine a weight for at least some of a plurality of reference models related to a motion of the first avatar based at least on the degree of variation determined, and display, on the display, the first avatar on which the motion is performed based on the plurality of reference models and the weight.