Remote Avatar Facial Texture Mapping for Expressive Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote communication technologies struggle to accurately capture and convey emotional expressions and facial characteristics, limiting the ability to reflect a user's personality and character in real-time audio and video connections.
Innovation Solution
A method and system for puppeteering a remote avatar by projecting captured facial images onto a facial framework, determining facial textures, and updating them based on subsequent facial frameworks to generate a three-dimensional avatar, which includes blending facial expression textures and using rendering weights to ensure accurate representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional remote communication methods (email, text, audio, video) are used, then communication can be conducted remotely, but the ability to capture and convey emotional expressions and facial characteristics is lost
Solution Approach 1:
The patent creates a digital copy of the user's face by projecting captured facial images onto a three-dimensional facial framework. This copy, called an avatar, replicates the user's facial characteristics, expressions, and emotions, allowing remote communication to convey emotional information that would otherwise be lost in traditional digital communication
Solution Approach 2:
The system captures changes in facial parameters (such as muscle movements, expression patterns, and facial geometry) and translates these physical changes into corresponding updates of the avatar's appearance. By monitoring and reproducing these dynamic parameters, the system preserves emotional expressions throughout the communication process
2Loss of information
If detailed facial textures and expressions are captured and transmitted, then emotional context and personality representation are improved, but bandwidth consumption and processing requirements increase
Solution Approach 1:
The facial framework is divided into multiple independent texture regions corresponding to different facial areas (eyes, mouth, cheeks, etc.). Each region can be updated independently based on captured facial expressions, allowing the system to transmit only the necessary texture changes rather than entire facial images, thereby reducing bandwidth consumption while maintaining emotional expression fidelity
Solution Approach 2:
The system updates only the portions of the facial texture that have changed or are relevant to current expressions, rather than transmitting or processing complete facial images continuously. This partial updating approach reduces processing requirements and bandwidth usage while still capturing all necessary emotional information
Data Source
AI summary
A method includes receiving a first facial framework and a first captured image of a face. The first facial framework corresponds to the face at a first frame and includes a first facial mesh of facial information. The method also includes projecting the first captured image onto the first facial framework and determining a facial texture corresponding to the face based on the projected first captured image. The method also includes receiving a second facial framework at a second frame that includes a second facial mesh of facial information and updating the facial texture based on the received second facial framework. The method also includes displaying the updated facial texture as a three-dimensional avatar. The three-dimensional avatar corresponds to a virtual representation of the face.