Dynamic Facial Expression Driving via Key Point Difference Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing expression driving techniques lack flexibility and interactivity, relying on preset template expressions, which limits their ability to accurately and interactively drive facial expressions in applications such as short videos and live broadcasts.
Innovation Solution
A method involving facial key point detection, alignment, and inter-frame smoothing to dynamically generate facial expressions by analyzing and processing facial key point sequences from both driven and driving characters, allowing for real-time rendering and filling of meaningless areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If preset template expressions are used for expression driving, then the implementation is simple, but the flexibility and interactivity are poor
Solution Approach 1:
The patent transforms the static preset template approach into a dynamic real-time processing system. The system continuously captures facial images, detects key points, calculates expression differences, and renders driven expressions in real-time, enabling flexible and interactive expression control rather than being limited to fixed templates
Solution Approach 2:
The patent changes the parameter representation from fixed template expressions to continuous facial key point coordinate sequences. By detecting and comparing key point positions between driving and driven characters, the system calculates expression difference parameters that can be dynamically adjusted, providing unlimited expression possibilities rather than being constrained to preset options
2Measurement precision
If real-time facial key point detection and processing is performed, then the flexibility and accuracy are improved, but the processing time and computational complexity increase
Solution Approach 1:
The patent segments the facial expression driving process into distinct modular steps: facial key point detection, expression difference calculation, and expression drive rendering. Each step processes specific information independently, allowing for optimized computation and reduced overall processing time while maintaining high accuracy
Solution Approach 2:
The patent performs preliminary facial key point detection on both the driving character and driven character before calculating expression differences. This preliminary action establishes the baseline key point positions, allowing the subsequent difference calculation to focus only on changes rather than processing entire facial structures from scratch, thereby reducing computational time
3Manufacturing precision
If multiple processing steps including alignment and inter-frame smoothing are added, then the rendering quality and realism are improved, but the device complexity increases
Solution Approach 1:
The patent introduces an intermediary expression difference calculation module that processes the raw key point sequences from both characters. This intermediary step calculates the difference between driving and driven character key points, serving as a mediator that transforms the complex task of realistic expression rendering into a more manageable difference calculation problem, improving quality without proportionally increasing complexity
Data Source
AI summary
A method, an electronic device and a storage medium for expression driving are disclosed. The method may include: performing facial key point detection on a driven character in a first image to obtain a first facial key point sequence; performing the following processing for each second image of a plurality of second images obtained successively: performing facial key point detection on a driving character in the second image to obtain a second facial key point sequence; obtaining a difference between the second facial key point sequence and an expressionless key point sequence which has been determined previously according to an analysis on the second facial key point sequence for a previous second image, and performing expression drive rendering on the driven character based on the difference and the first facial key point sequence. The technical solution may enhance flexibility, interactivity, accuracy etc.