Glint-Assisted Gaze Tracking in VR Headsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual reality (VR) and augmented reality (AR) head-mounted displays (HMDs) face challenges in accurately tracking user gaze, which affects the rendering of virtual content and interaction with the environment, as existing methods lack precision in determining the point of gaze and visual axis.
Innovation Solution
A glint-assisted gaze tracking system is implemented in HMDs, using cameras and LEDs to detect glints and pupil locations, which are then matched to estimate the cornea and pupil centers in 3D space, reconstructing the optical and visual axes to accurately determine the point of gaze on the display.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional gaze tracking methods are used in HMDs, then the system structure is simpler, but the measurement precision of gaze point and visual axis is insufficient
Solution Approach 1:
The gaze tracking system is segmented into multiple functional components: glint detection module, pupil detection module, cornea center estimation module, and visual axis reconstruction module. Each module processes specific aspects of eye imaging independently, allowing for specialized algorithms optimized for each measurement task while maintaining overall system modularity
Solution Approach 2:
The patent introduces glint detection as an intermediary element that provides reference points for establishing the optical axis. By detecting glints (reflections from the corneal surface) and using them as intermediate reference markers, the system can more accurately determine the visual axis without requiring direct measurement of the entire optical path
2Measurement precision
If glint-assisted gaze tracking is implemented, then the measurement precision of gaze point is improved, but the device complexity increases due to additional processing modules
Solution Approach 1:
The system performs preliminary glint detection and matching before pupil detection. By pre-identifying glint positions and establishing LED-glent correspondences in advance, the system creates a reference framework that simplifies subsequent pupil center detection and visual axis calculation, reducing the computational complexity of later processing stages
Solution Approach 2:
The patent transitions from 2D image plane detection to 3D spatial reconstruction. By detecting glints and pupils in 2D images and then reconstructing their positions in 3D space, the system achieves more accurate gaze point determination while the dimensional transformation provides additional geometric constraints that simplify certain calculation steps
3Reliability
If multiple detection processes (glint and pupil) are used, then the reliability of gaze estimation is improved, but the loss of processing time increases
Solution Approach 1:
The system implements continuous tracking by maintaining glint and pupil detection across successive video frames. By tracking the temporal continuity of eye features and using predictive algorithms, the system can maintain reliable gaze estimates even during brief periods when feature detection is uncertain, reducing the need for repeated full detection cycles
Solution Approach 2:
The system uses feedback from glint detection to guide pupil detection and vice versa. The detected glint positions provide feedback about eye orientation and position, which constrains the search space for pupil detection. This mutual feedback mechanism increases reliability while reducing processing time by avoiding exhaustive search in all possible regions
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This method enhances the accuracy of gaze tracking, allowing for improved rendering of virtual content, focus adjustment, and interaction with the environment, providing a more immersive and interactive VR/AR experience.
Implementation Method 1
detect glints...matches the detected glints to particular ones of the light-emitting elements...estimates the center of the user's cornea in 3D space based on the detected glints and LED correspondences
Implementation Method 2
pupil detection process...detecting the pupil location and contour...estimates the center of the user's pupil in 3D space based on the detected pupil ellipse
Data Source
AI summary
Methods and apparatus for glint-assisted gaze tracking in a VR/AR head-mounted display (HMD). Images of a user's eyes captured by gaze tracking cameras may be analyzed to detect glints (reflections on the cornea of light sources that illuminate the user's eyes) and the pupil. The glints are matched to particular ones of the light sources. The glint-light source matches are used to determine the cornea center of the eye, and the pupil center is determined. The optical axis of the eye is reconstructed from the cornea center and the pupil center, and the visual axis is then reconstructed from the optical axis and a 3D model of the user's eye. The point of gaze on the display is then determined based on the visual axis and a 3D model of the HMD.