Eye Rotation Model for Occlusion-Robust Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional eye tracking systems in devices like glasses-type HMDs face limitations due to structural restrictions, including camera and illumination placement, power, and computational constraints, necessitating a more robust gaze tracking paradigm that can handle occlusions and maintain low latency.
Innovation Solution
The system employs a globe modeling technique using a 3D rotational model with N degrees of freedom, split into a reference estimator for initial N DoF position estimation and a gaze estimator for two DoF rotation, leveraging ambient light and reduced light sources, and combining glint-based and non-glint features for gaze vector estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional eye tracking algorithms are used in glasses-type HMDs, then the system can track gaze direction, but the structural restrictions and occlusions reduce measurement precision and reliability
Solution Approach 1:
The patent changes the mathematical parameters of the eye model from a fixed 5 DoF model to a dynamic 6 DoF globe model with variable center of rotation. This allows the system to adapt to different eye positions and rotations, maintaining measurement precision even when occlusions occur in glasses-type HMDs with limited camera placement options.
Solution Approach 2:
The system transitions from a static eye model with fixed center of rotation to a dynamic model where the center of rotation can move within a defined region. This dynamic adaptation enables the gaze tracking system to maintain reliability under varying occlusion conditions by adjusting the model parameters in real-time based on detected eye features.
2Device complexity
If a single camera is used in glasses-type HMDs, then device complexity is reduced, but occlusions increase making gaze tracking more difficult
Solution Approach 1:
The patent compensates for the limited viewing angle of a single camera by introducing a temporal dimension through frame-to-frame tracking and a spatial dimension through the 6 DoF globe model. This allows the system to reconstruct complete eye geometry from partial views that would be insufficient for conventional 5 DoF models.
Solution Approach 2:
The system creates a virtual 3D copy of the eye globe based on 2D image features from the single camera. This virtual model allows the system to infer eye parameters that are not directly visible in the image, effectively copying the complete eye geometry from partial observations.
3Measurement precision
If 5 DoF eye estimation is performed for every frame, then gaze tracking precision is maintained, but computational power and processing time increase
Solution Approach 1:
The patent segments the gaze estimation process into two distinct modules: a reference estimator that performs computationally intensive 6 DoF eye model fitting at lower frame rates, and a gaze estimator that uses the reference model to rapidly compute gaze vectors at full frame rate. This segmentation maintains precision while reducing overall computational power consumption.
Solution Approach 2:
The system performs preliminary action by pre-computing the reference eye model parameters (center of rotation, radius, orientation) at lower frequency. These pre-computed parameters are then reused for rapid gaze estimation in subsequent frames, avoiding redundant computation while maintaining precision through the constrained 6 DoF model.
4Illumination intensity
If multiple light sources are used for illumination, then eye feature visibility is improved, but device complexity and power consumption increase
Solution Approach 1:
The patent makes the existing display pixels serve a dual function: displaying visual content and providing illumination for eye tracking. By modulating the brightness of display pixels, the system illuminates the eye without requiring separate illumination hardware, thereby improving eye feature visibility while avoiding increased device complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables accurate and precise gaze tracking with low latency while adhering to the architectural and functional restraints of glasses-type HMDs, improving gaze estimation performance and reducing complexity.
Implementation Method 1
images of the eye captured by eye-facing camera(s)
Data Source
AI summary
An eye tracking system that includes a reference estimator that estimates a rotational model of the eye with N (e.g., five) degrees of freedom and a gaze estimator that estimates a gaze vector with the rotational model as a constraint. The rotational model may include a centroid region rather than a fixed eye center. The rotational model may remain static until a trigger event, at which time a new rotational model is estimated. One or both estimators may include a trained neural network. Model and gaze estimation may depend on eye features extracted from images; the eye features may include glints, but other eye features than glints may be used if the device does not include dedicated eye-illuminating light sources. A glint-based gaze vector and a feature-based gaze vector may be estimated, and a final gaze vector may be determined from the two estimated gaze vectors.