Facial Landmark Transformer for Identity-Expression Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial image analysis techniques fail to distinguish between appearance characteristics and emotional characteristics, leading to misclassification and deterioration in performance, particularly in few-shot settings where the identity of the subject does not coincide with the driver face.
Innovation Solution
A method involving a landmark transformer that extracts expression and identity landmarks using a transformation matrix, generating a high-quality reenacted image by combining driver and target feature maps through an artificial neural network, even in few-shot settings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing facial image analysis techniques are used to process facial landmarks, then the processing can be performed, but the performance deteriorates due to inability to distinguish appearance characteristics from emotional characteristics
Solution Approach 1:
The patent segments facial landmarks into two distinct types: appearance landmarks (capturing identity and structural characteristics) and expression landmarks (capturing emotional and pose characteristics). This segmentation allows independent processing and analysis of each type, resolving the contradiction by enabling precise distinction between appearance and expression characteristics while maintaining processing reliability.
Solution Approach 2:
The patent introduces a transformation matrix as an intermediary that maps between different landmark representations. This transformation matrix serves as a mediator to convert mixed appearance and expression characteristics into separated representations, enabling accurate distinction and processing of each characteristic type without interference from the other.
2Ease of operation
If facial landmarks are processed without distinguishing appearance and expression characteristics, then processing is simplified, but misclassification occurs (e.g., emotionless face classified as surprise)
Solution Approach 1:
By segmenting landmarks into appearance and expression categories, the patent enables precise emotion classification without the confusion of mixed characteristics. The appearance landmarks provide stable identity information while expression landmarks capture emotional cues, allowing accurate classification even when facial configurations are atypical.
Solution Approach 2:
The patent extracts expression-specific characteristics from the mixed facial landmark data by isolating expression landmarks that capture emotional and pose information separate from appearance landmarks. This extraction allows focused analysis on emotional characteristics without the interfering influence of appearance variations.
3Measurement precision
If a transformation matrix is introduced to separate appearance and expression landmarks, then characteristic distinction improves, but device complexity increases
Solution Approach 1:
The transformation matrix serves as a compact intermediary that encapsulates the complex relationship between appearance and expression characteristics. Rather than implementing complex separate processing systems, the transformation matrix provides a unified mathematical framework that achieves characteristic separation through a single learnable parameter set, minimizing overall system complexity.
Data Source
AI summary
Provided is a method of transforming a landmark including: receiving an input image including a facial image of a first person and a landmark corresponding to the facial image; estimating a transformation matrix corresponding to the landmark; and calculating an expression landmark and an identity landmark corresponding to the input image by using the transformation matrix.