AR/VR Eye Tracking via Embedded Content Glints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Eye tracking in augmented and virtual reality (AR/VR) systems typically relies on light-emitting diodes (LEDs) fixed to head-mounted devices (HMDs), increasing hardware cost and footprint, and power consumption.
Innovation Solution
Embedding glints into AR/VR content frames for eye tracking, using image sensors to detect light intensity data from these glints, allowing for the determination of eye orientation without the need for external LEDs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LEDs are fixed to the HMD for eye tracking, then eye tracking accuracy is improved, but hardware cost and footprint increase
Solution Approach 1:
The patent extracts the light source function from external LEDs and embeds it within the display content itself. The display device generates glints directly in the visual content, eliminating the need for separate LED components on the HMD while maintaining eye tracking functionality.
Solution Approach 2:
The display content serves multiple functions: it provides the visual AR/VR experience for the user and simultaneously acts as the light source for eye tracking. The glints embedded in the content fulfill both content delivery and eye tracking illumination purposes.
2Reliability
If LEDs are fixed to the HMD for eye tracking, then eye tracking functionality is achieved, but power consumption increases
Solution Approach 1:
The patent merges the content display function and eye tracking illumination function into a single process. The same display hardware and power supply serve both purposes, eliminating the need for separate LED power consumption.
Solution Approach 2:
The display content serves itself dual purposes: entertaining/informing the user while simultaneously providing the light patterns needed for eye tracking. The system uses its own operational resources (display output) to achieve the eye tracking function.
3Difficulty of detecting and measuring
If external LEDs are used for eye tracking, then glint detection is simplified, but hardware cost increases
Solution Approach 1:
The patent creates optical copies of light sources within the digital content domain. Instead of physical LEDs, virtual light sources (glints) are rendered in the display content, which are then detected by the existing image sensor without requiring additional hardware components.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Reduces hardware complexity and power consumption in HMDs while maintaining accurate eye tracking for interactive AR/VR experiences.
Implementation Method 1
obtaining, via the image sensor, light intensity data corresponding to the one or more content frames of the AR/VR content stream that include the plurality of glints, wherein the light intensity data includes a projection of an eye of the user of the HMD having projected thereon the plurality of glints
Data Source
AI summary
In one implementation, a method includes: synthesizing an AR/VR content stream by embedding a plurality of glints provided for eye tracking into one or more content frames of the AR/VR content stream; displaying, via the one or more AR/VR displays, the AR/VR content stream to a user of the HMD; obtaining, via the image sensor, light intensity data corresponding to the one or more content frames of the AR/VR content stream that include the plurality of glints, wherein the light intensity data includes a projection of an eye of the user of the HMD having projected thereon the plurality of glints; and determining an orientation of the eye of the user of the HMD based on the light intensity data.