Virtual Anchor Synchronization Using Real-Time Facial Motion Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional live broadcast technologies using 3D characters as virtual anchors struggle to replicate the expression and action reproduction of real anchors, leading to a noticeable gap in picture quality and authenticity.
Innovation Solution
A method that generates a live audio synchronized with a real anchor's speech and matches facial movements of the real anchor to create a second live video, enhancing the virtual anchor's appearance and interaction with the audience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of stationary object
If a 3D character is used as a virtual anchor, then the anchor can be tireless and continuously provide content, but the expression and action reproduction is significantly different from a real anchor
Solution Approach 1:
The patent uses deep learning technology to copy the facial expressions, gestures, and speech patterns of a real anchor onto a virtual anchor. The system captures real anchor performance data through multiple cameras and microphones, then uses neural networks to replicate this behavior in the virtual anchor, making it visually and behaviorally similar to the real anchor while maintaining the ability to broadcast continuously without fatigue
Solution Approach 2:
The patent replaces the mechanical limitation of a real human anchor (physical fatigue, emotional instability) with a digital system. The virtual anchor is controlled by computer algorithms and AI models that can operate continuously without biological constraints, while still reproducing human-like expressions and interactions through digital rendering and motion synthesis
2Adaptability or versatility
If a 3D character is used as a virtual anchor, then technology limitations are overcome, but picture quality and image are significantly different from a real anchor
Solution Approach 1:
The patent dynamically changes visual parameters of the virtual anchor in real-time to match the real anchor's appearance, lighting conditions, and environmental context. The system adjusts facial features, expressions, and lighting parameters frame-by-frame to maintain visual consistency with the real anchor, making the virtual anchor indistinguishable from a real person in terms of picture quality
3Duration of action of stationary object
If traditional 3D virtual anchor technology is used, then the broadcast can run continuously, but the audience can easily distinguish the difference between real and virtual anchor
Solution Approach 1:
The patent implements real-time feedback mechanisms where the system continuously monitors the real anchor's performance and adjusts the virtual anchor's output accordingly. Microphones capture the real anchor's voice, cameras track facial expressions and gestures, and this feedback is immediately fed back into the virtual anchor's rendering system to maintain synchronization and authenticity, making it difficult for the audience to distinguish between real and virtual
Data Source
AI summary
A live broadcast method is provided by present disclosure. The method includes obtaining a live broadcast text and generating a live audio based on the live broadcast text, and determining a first live video that is time-aligned with the live audio and obtaining a second live video; playing the second live video in a live broadcast room; pausing the second live video in response that an audience in the live broadcast room asking a question, and playing a first response video that replies to the question; and continuing to play the second live broadcast video in response that all questions from the audience in the live broadcast room have been answered.