Eye Movement Metrics for Facial Expression Prediction Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in accurately predicting user expressions based on limited facial images, particularly from the eyes, due to subtle changes and complex linking of facial movements to specific emotions, which are difficult to detect and analyze.
Innovation Solution
A machine learning model is trained to predict user expressions using a series of images of the user's eye, employing facial action coding system (FACS) action units, and adjusting weights based on error data to improve accuracy, allowing for user-specific or general predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If only eye region images are used for expression prediction, then device complexity is reduced and ease of operation is improved, but measurement precision and reliability of expression detection deteriorate due to limited visual information
Solution Approach 1:
The system segments the facial expression analysis task into two parts: (1) detecting subtle movement metrics in the eye region using a simple camera, and (2) using a machine learning model to infer complete facial expressions from these localized metrics. This segmentation allows the physical device to remain simple while achieving accurate expression detection through computational processing.
Solution Approach 2:
The system transforms the expression detection problem by changing from direct visual analysis of entire facial expressions to analyzing subtle movement metrics (parameter changes) in the eye region. The machine learning model learns the mapping between these subtle eye movement parameters and comprehensive facial expressions, enabling accurate detection despite limited input data.
2Measurement precision
If machine learning models are trained with user-specific data, then expression prediction accuracy is improved, but loss of time increases due to calibration requirements
Solution Approach 1:
The system performs preliminary action by pre-training the machine learning model on a diverse dataset of multiple users before deployment. This preliminary training enables the model to generalize expression prediction across different users without requiring extensive user-specific calibration, thus maintaining high accuracy while minimizing setup time.
Solution Approach 2:
The system uses partial action by implementing a two-stage training approach: (1) general training on diverse user data to establish baseline expression patterns, and (2) optional minimal user-specific calibration. This partial customization approach achieves most of the accuracy benefit without the full time cost of complete user-specific training.
Data Source
Figure 1
Figure 2A~2B
Figure 3A~3B
AI summary
Techniques are disclosed for training a machine learning model to predict user expression. A plurality of images are received, each of the plurality of images containing at least a portion of a user's face. A plurality of values for a movement metric are calculated based on the plurality of images, each of the plurality of values for the movement metric being indicative of movement of the user's face. A plurality of values for an expression unit are calculated based on the plurality of values for the movement metric, each of the plurality of values for the expression unit corresponding to an extent to which the user's face is producing the expression unit. The machine learning model is trained using the plurality of images and the plurality of values for the expression unit.