Off-Axis Eye Tracking for Glare-Resistant Gaze Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems lack efficient and non-obtrusive methods for eye tracking, particularly in head-mounted devices, which are crucial for accurate gaze estimation and effective rendering of virtual content.
Innovation Solution
A deep neural network is trained to jointly estimate multiple eye and gaze estimation parameters, including pupil center, cornea center, and optical axis, using off-axis camera images, enabling precise gaze vector computation for head-mounted AR/VR/MR devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a camera is positioned directly in front of the display screen to capture user eye movements, then eye tracking accuracy is improved, but the camera captures distracting reflections and glare from the screen that interfere with accurate eye detection
Solution Approach 1:
The patent positions the camera off-axis relative to the display screen, creating an asymmetric geometric relationship between the camera, screen, and user's eyes. This asymmetric positioning allows the camera to capture eye movements while avoiding the symmetric reflection path that would otherwise bounce directly back from the screen surface into the camera lens.
Solution Approach 2:
The patent introduces a new spatial dimension by positioning the camera at an angle rather than directly in front of the screen. This dimensional change in camera placement creates a different viewing geometry that separates the eye tracking function from the harmful screen reflections, allowing both goals to be achieved simultaneously.
2Object-affected harmful factors
If an off-axis camera is used to avoid screen reflections, then distracting reflections are reduced, but the angle at which eye movements must be tracked becomes more challenging
Solution Approach 1:
The patent incorporates feedback mechanisms where the system continuously monitors the user's eye position and adjusts tracking parameters accordingly. The feedback loop allows the off-axis camera system to compensate for the challenging angles by dynamically adapting to maintain accurate eye movement detection despite the non-perpendicular viewing geometry.
Solution Approach 2:
The patent employs parameter changes by adjusting camera angles, focal lengths, and processing algorithms to optimize eye tracking performance from an off-axis position. By varying these parameters, the system overcomes the inherent difficulties of tracking eye movements at oblique angles while maintaining avoidance of screen reflections.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The method enhances the accuracy and robustness of eye tracking, facilitating energy-efficient rendering and improved user interaction in AR/VR/MR environments by decoupling intermediate predictions from final gaze estimation, simplifying data collection, and providing interpretable results.
Implementation Method 1
position information about a user's eye is obtained using a camera. The camera may be positioned to record an image of the user's eye that is reflected from a display screen
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for estimating a gaze vector of an eye using a trained neural network. An input image of the eye may be received from a camera. The input image may be provided to the neural network. Network output data may be generated using the neural network. The network output data may include two-dimensional (2D) pupil data, eye segmentation data, and/or cornea center data. The gaze vector may be computed based on the network output data. The neural network may be previously trained by providing a training input image to the neural network, generating training network output data, receiving ground-truth (GT) data, computing error data based on a difference between the training network output data and the GT data, and modifying the neural network based on the error data.