Multi-Sensor Eye Tracking for Occlusion-Resilient AR Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing eye-tracking technologies in artificial-reality devices face challenges such as occlusions, side views, and varying light conditions, and the calibration process does not account for physical skewing or changes over time, affecting performance.
Innovation Solution
The integration of a depth sensor, such as a time-of-flight sensor or self mixing interferometer, with a visual camera to generate an improved eye-tracking estimate, combined with a processor that uses machine learning and calibration data to dynamically adjust for changing conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional eye-tracking techniques using light sources and image sensors are used, then eye-tracking functionality is achieved, but performance deteriorates under occlusions, side views, and varying light conditions
Solution Approach 1:
The patent combines multiple sensor types (depth sensor, image sensor, inertial sensor) into an integrated eye-tracking system. The depth sensor provides absolute depth measurements that are fused with image sensor data and inertial data to create a robust multi-modal eye-tracking system that maintains reliability under various challenging conditions.
Solution Approach 2:
The system dynamically adjusts calibration parameters in real-time based on inertial sensor data that detects device movement and positioning changes. This dynamic recalibration allows the system to adapt to varying light conditions, occlusions, and side views by continuously updating calibration data according to the current device state.
2Ease of manufacture
If calibration is performed once at factory, then initial setup is simple, but accuracy deteriorates due to physical skewing and changes over time
Solution Approach 1:
The system performs initial calibration at the factory to establish baseline parameters, then automatically performs additional calibration steps when the device is first worn by a user. This preliminary action ensures both ease of manufacture and maintains measurement precision by preparing the system in advance for personalized calibration.
Solution Approach 2:
The system implements continuous feedback-based calibration where inertial sensor data provides information about device movement and positioning changes. This feedback loop triggers automatic recalibration when skewing or positioning changes are detected, maintaining accuracy over time without requiring manual intervention.
3Measurement precision
If depth sensor is integrated with visual camera and machine learning is used, then eye-tracking accuracy is improved, but device complexity increases
Solution Approach 1:
The depth sensor and inertial sensors serve multiple functions: they provide depth information for eye-tracking, detect device movement for calibration triggers, and monitor environmental conditions. This multi-functionality reduces the need for additional dedicated components, managing device complexity while improving measurement precision.
Solution Approach 2:
The system uses its own inertial sensor data to automatically detect when recalibration is needed and triggers the calibration process autonomously. The machine learning algorithms process sensor data to identify positioning changes and initiate calibration without external intervention, allowing the device to self-correct and maintain accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances eye-tracking accuracy by providing absolute depth measurements and continuous calibration, improving performance in varying light conditions and device positioning changes.
Implementation Method 1
a depth sensor (e.g., a time-of-flight sensor or self mixing interferometer)
Implementation Method 2
a depth sensor (e.g., a time-of-flight sensor or self mixing interferometer)
Data Source
AI summary
Techniques are described for improved eye-tracking through the combination of multiple sensor modalities. Embodiments of the present disclosure may include a wearable device (e.g., artificial-reality device) comprising an eye-tracking module that tracks the positions of either or both of the wearer's eyes based on image data of the eye(s) in addition to other sensor data. The additional sensor data may include, for instance, a depth sensor, an inertial sensor, a face-tracking system, or combinations thereof.