Facial Landmark Transformation for Identity-Expression Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial image analysis techniques fail to distinguish between appearance and emotional characteristics, leading to misclassification and deterioration in performance, particularly in few-shot settings where the identity of the subject does not coincide with the driver face.
Innovation Solution
A method and apparatus that utilize a transformation matrix to calculate expression and identity landmarks, generating a reenacted image with high quality by extracting and transforming facial landmarks using a landmark transformer, encoder, and decoder, even in few-shot settings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing facial image analysis techniques are used, then processing is simple, but classification accuracy deteriorates due to inability to distinguish appearance from emotional characteristics
Solution Approach 1:
The patent divides facial landmarks into two distinct components: identity landmarks (capturing appearance characteristics) and expression landmarks (capturing emotional characteristics). This segmentation allows the system to process and analyze each type separately, improving classification accuracy by preventing confusion between appearance and expression while maintaining manageable processing complexity through modular treatment of each landmark type.
2Reliability
If traditional landmark processing is used, then computational resources are saved, but performance deteriorates in few-shot settings with non-matching identities
Solution Approach 1:
The patent extracts and separates expression information from identity information in facial landmarks. By taking out the expression component independently, the system can perform expression analysis that is invariant to identity differences, significantly improving reliability in few-shot settings where the target identity may not be present in training data, while computational overhead remains acceptable due to the efficient extraction process.
Solution Approach 2:
The patent applies different processing strategies to different types of landmarks based on their local quality characteristics. Identity landmarks are processed to preserve appearance-specific features, while expression landmarks are processed to capture emotional states. This localized quality differentiation improves overall system reliability by optimizing each component for its specific function, particularly benefiting few-shot performance through robust expression representation.
3Measurement precision
If facial landmarks are processed without separation, then processing is straightforward, but misclassification occurs between appearance and emotional characteristics
Solution Approach 1:
The patent implements segmentation of facial landmarks into identity-specific and expression-specific components. This division enables precise feature discrimination by ensuring that appearance characteristics and emotional characteristics are analyzed independently, eliminating misclassification issues while maintaining straightforward processing through clear separation of concerns.
Solution Approach 2:
Instead of trying to extract expression from combined landmarks (which causes confusion), the patent inverts the approach by directly computing expression landmarks through transformation matrices that are inherently expression-focused, while identity landmarks capture the residual appearance information. This inverted strategy improves feature discrimination accuracy by design rather than by complex separation algorithms.
Data Source
AI summary
Provided is a method of transforming a landmark including: receiving an input image including a facial image of a first person and a landmark corresponding to the facial image; estimating a transformation matrix corresponding to the landmark; and calculating an expression landmark and an identity landmark corresponding to the input image by using the transformation matrix.