Avatar Lip-Sync Rendering Using Split Voice Processing Circuits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems struggle to accurately display avatars with synchronized mouth movements in real-time, particularly when processing voice data, leading to delays and inconsistencies.
Innovation Solution
An electronic device with multiple processing circuits is used to identify voice data features at different speeds, allowing for the generation of mouth shapes based on these features, and displaying avatars with synchronized mouth movements using a combination of hardware and software components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single processing circuit is used to process voice data for avatar mouth synchronization, then device complexity is reduced, but processing speed and synchronization accuracy deteriorate
Solution Approach 1:
The patent divides the processing circuit into multiple independent processing units, each capable of handling different aspects of voice data processing simultaneously. This segmentation allows parallel processing of voice features, improving overall processing speed while maintaining manageable device complexity through modular architecture
Solution Approach 2:
The processing circuits are designed with multi-functional capabilities to handle various voice processing tasks (feature extraction, analysis, synthesis) within a unified hardware framework. This universality reduces the need for separate dedicated circuits for each function, balancing processing power with device complexity
2Productivity
If multiple processing circuits are used to process voice data in parallel, then processing speed improves, but device complexity increases
Solution Approach 1:
The processing system is segmented into specialized processing circuits that can operate in parallel. Each circuit handles specific voice processing tasks, enabling high-throughput processing while keeping individual circuit complexity low through functional decomposition
Solution Approach 2:
The patent implements a hierarchical processing architecture where simpler processing units are nested within more complex coordination structures. This nesting allows multiple processing circuits to work together efficiently under unified control, managing overall system complexity while maintaining high processing productivity
3Loss of time
If processing speed is increased for real-time avatar display, then user experience improves, but processing accuracy may deteriorate
Solution Approach 1:
The system performs preliminary processing of voice data to extract essential features before full avatar rendering. By pre-processing and identifying key voice characteristics in advance, the system reduces subsequent processing time while maintaining accuracy in mouth shape identification through focused feature analysis
Solution Approach 2:
The patent applies partial processing strategies where not all voice data aspects are processed with equal depth. Critical features for mouth synchronization are processed with high accuracy, while less critical aspects use faster, less computationally intensive methods, achieving acceptable overall accuracy with reduced latency
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
This electronic device may comprise a display, a memory for storing instructions, and at least one processor. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to: with regard to feature value identification of voice data, identify a first processing speed of each of a plurality of processing circuits for processing the voice data; with regard to mouth shape identification of the voice data, identify a second processing speed of each of the plurality of processing circuits; obtain voice information from the outside of the electronic device while displaying an avatar; obtain a plurality of feature values of the voice information by using a first processing circuit identified on the basis of the first processing speed; obtain information for generating mouth shapes on the basis of the plurality of feature values, by using a second processing circuit identified on the basis of the second processing speed; and display, through the display, the avatar including the mouth shapes generated on the basis of the information.