Face Expression Coefficient Extraction via Identity-Texture Decoupling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parameterized face 3D reconstruction algorithms often mix expression information with non-expression information, leading to inaccurate extraction and poor accuracy in information processing tasks such as active speaker detection and expression recognition.
Innovation Solution
An information processing method that utilizes a combination of backbone models and head network models to extract and decouple identity, texture, and expression coefficients from face images, employing a training process with shared coefficients and regularization to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a parameterized face 3D reconstruction algorithm is used to extract expression information, then the reconstruction can be performed in different environments, but the extracted expression information mixes with non-expression information resulting in poor accuracy
Solution Approach 1:
The patent segments the face 3D model parameters into distinct components: identity coefficients, texture coefficients, and expression coefficients. By separating these previously mixed parameters, the system can extract pure expression information while maintaining adaptability across different environments through the parameterized 3DMM framework.
Solution Approach 2:
The patent extracts expression coefficients separately from identity and texture coefficients by introducing expression-specific loss functions and optimization constraints. This extraction process isolates expression information from non-expression information, resolving the accuracy problem while preserving environmental adaptability.
2Ease of manufacture
If 3DMM coefficients are estimated directly from face images, then the process is simple, but the expression coefficients are mixed with identity and texture information leading to poor accuracy
Solution Approach 1:
The patent divides the 3DMM coefficient estimation into separate optimization problems for identity coefficients, texture coefficients, and expression coefficients. This segmentation maintains the simplicity of the overall process while improving accuracy by preventing information mixing through dedicated loss functions for each coefficient type.
Solution Approach 2:
The patent applies different optimization strategies and loss functions to different coefficient types. Expression coefficients use expression-specific loss functions, while identity and texture coefficients use their respective optimization approaches. This local quality differentiation ensures each coefficient type is estimated with appropriate precision without complicating the overall process.
Data Source
AI summary
An information processing method is provided. The method includes obtaining a target video. A first target image feature corresponding to a face image of each frame is obtained. A target identity coefficient and a target texture coefficient corresponding to the target image feature in the face image of a different frame in the target video are obtained. A first target identity feature is obtained according to the target identity coefficient, and a first target texture feature is obtained according to the target texture coefficient. Once a first target feature is obtained by splicing the target image feature, the first target identity feature and the first target texture feature, a first target expression coefficient is obtained based on the first target feature.