Facial Micro-Expression Detection With Motion Retargeting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Facial micro-expression analysis is challenging due to its brief duration, slight variations, and limited action areas, making automatic detection difficult, and existing methods lack robustness and generalization across databases.
Innovation Solution
A system and method for facial expression detection and reconstruction using a deep motion retargeting module with a neural network engine, enhanced by an edge-intensified multi-head self-attention module, to extract and enhance subtle facial movements and generate reconstructed images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If automatic FME analysis is performed, then detection capability is improved, but measurement precision deteriorates due to short time interval and slight variations
Solution Approach 1:
The patent segments the facial region into multiple key facial features (eyes, eyebrows, nose, mouth, cheeks, chin) and further divides them into sub-regions. This segmentation allows the system to focus detection resources on specific areas where micro-expressions occur, improving measurement precision while maintaining automation. The segmented approach enables independent analysis of each facial feature's motion patterns.
Solution Approach 2:
The patent applies local quality by using different processing strategies for different facial regions. The system identifies key facial features and applies specialized motion extraction algorithms tailored to each region's characteristics. This localized approach enhances detection precision for subtle movements in specific areas without compromising overall automation.
2Measurement precision
If deep motion retargeting module is used, then subtle movement detection is improved, but device complexity increases
Solution Approach 1:
The patent introduces an intermediary deep motion retargeting module that acts as a bridge between raw image sequences and motion extraction. This module uses pre-trained neural networks to transform source facial expressions to target facial expressions, enabling subtle movement detection without directly complexifying the entire system. The intermediary handles the complexity internally while providing simplified interfaces.
Solution Approach 2:
The patent uses copying by creating reconstructed images that replicate the motion patterns from source images. The deep motion retargeting module generates copies of facial expressions on different subjects, allowing subtle movements to be detected and analyzed without requiring direct manipulation of the original complex image data multiple times.
3Measurement precision
If edge prediction auxiliary task is performed, then subtle movement extraction is improved, but loss of time increases
Solution Approach 1:
The patent applies preliminary action by performing edge prediction as an auxiliary task during the training phase of the neural network. The edge prediction capability is pre-trained and integrated into the motion extraction model, so that during actual FME detection, the system can leverage these pre-learned edge features without performing separate edge detection operations, thus reducing processing time while maintaining extraction precision.
Solution Approach 2:
The patent merges the edge prediction task with the main motion extraction task by using a multi-task learning approach. The neural network simultaneously performs edge prediction and motion extraction, sharing common features and computational resources. This merging reduces the total processing time compared to performing separate edge detection and motion analysis operations.
4Adaptability or versatility
If reconstructed images are generated, then FME sample diversity is improved, but productivity decreases
Solution Approach 1:
The patent uses copying to generate reconstructed images by applying motion patterns from source images to target images. This copying approach creates diverse FME samples across different subjects while maintaining the original expression dynamics. The system efficiently generates multiple variations by reusing extracted motion features, improving sample diversity without proportionally increasing processing time.
Data Source
AI summary
A system for image processing, a method of facial expression detection and a method of reconstruction image generation. The system comprises a facial expression extraction module arranged to, upon receiving a source sequence of images representing a predetermined facial expression performed by a sample target, extract a motion of at least one key facial feature of the sample target associated with the predetermined facial expression, and to enhance the detection of the motion including at least one subtle movement of the at least one key facial feature; and a facial expression data analyser arranged to collect facial micro-expression data associated with the predetermined facial expression and the motion including subtle movements of all key facial features of the sample target performing the predetermined facial expression.