Monocular Eye Gaze Estimation via 3D Face Model Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional eye gaze direction estimation methods from monocular images face limitations in precision due to dependency on accurate iris and head coordinate system localization, and are restricted to specific environments and applications, lacking generalizability to different conditions.
Innovation Solution
A computer-implemented method that reconstructs a common virtual 3D space including the user, imaging device, and potential targets, allowing for eye gaze direction estimation without calibration data, using appearance-based face landmark detection and deep neural networks to estimate 3D gaze vectors from monocular images, reducing dependency on head orientation and iris positioning accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If model-based geometric methods are used for gaze estimation, then the method provides a clear geometric framework, but precision deteriorates because it requires precise localization of iris and head coordinate system which is often impossible to obtain accurately
Solution Approach 1:
The patent replaces the model-based geometric approach (which relies on mechanical localization of iris and head coordinates) with an appearance-based deep learning approach. The neural network directly maps eye image features to gaze direction, substituting the geometric calculation system with a data-driven pattern recognition system that does not require precise anatomical landmark detection.
Solution Approach 2:
The patent introduces an intermediate representation layer (the trained neural network model) that bridges the gap between raw eye images and gaze direction. This intermediary learns the complex mapping relationship during training, allowing the system to achieve high precision without directly solving the difficult problem of precise iris and head coordinate localization.
2Measurement precision
If appearance-based deep learning methods are used with huge datasets, then measurement precision improves, but device complexity increases due to computational requirements
Solution Approach 1:
The patent performs the computationally intensive work in advance by training the deep neural network offline using huge datasets. The trained model is then deployed as a compact representation that can run on resource-constrained devices. This preliminary action transfers the computational burden from the deployment phase to the training phase, enabling high precision on mobile devices.
Solution Approach 2:
The patent creates a simplified copy of the complex mapping relationship through the trained neural network model. Instead of requiring the full computational power needed to process huge datasets in real-time, the system uses the learned parameters and weights from the training data as a compact representation that can be efficiently evaluated on mobile devices.
3Measurement precision
If model-based methods requiring initialization are used, then the system can be calibrated, but adaptability deteriorates because the user must intentionally look at specific points which is not natural
Solution Approach 1:
The patent enables the system to learn gaze patterns automatically from natural user behavior during the training phase. The deep neural network is trained on datasets containing natural gaze sequences, allowing the system to self-calibrate by learning from unforced, natural looking behavior rather than requiring explicit calibration procedures from users.
Solution Approach 2:
The patent performs the calibration process in advance during offline training, where the neural network learns gaze patterns from pre-collected datasets. This preliminary learning eliminates the need for on-device calibration procedures, allowing the system to be immediately adaptable to natural user behavior without requiring intentional calibration actions.
4Device complexity
If monocular images are used for gaze estimation, then device complexity is reduced, but measurement precision deteriorates due to lack of depth information
Solution Approach 1:
The patent replaces the need for multiple imaging devices or depth sensors (which would provide explicit depth information) with a monocular imaging system combined with deep learning. The neural network learns to infer depth-related gaze information from single 2D images by recognizing appearance patterns, substituting the mechanical depth measurement system with a computational inference system.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
A computer-implemented method for estimating eye gaze direction, comprising: fitting (11) a 3D face model (111) to a monocular image (101) obtained from an imaging device, thus obtaining values of a set of face model parameters representing at least one model position parameter (t), at least one orientation parameter (r), at least one shape parameter (s) and at least one action parameter (a), obtaining (12) normalized 3D gaze estimation vectors for the right and left eyes (121) with respect to the imaging device viewpoint; estimating (13) the eye gaze direction with respect to at least one target in the scene. A system comprising at least one processor configured to perform the steps of the method. A computer program product comprising computer program instructions/code for performing the method. A computer-readable memory/medium that stores program instructions/code for performing the method.