Eye Tracking Alignment for Optical See-Through Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current head-mounted optical see-through displays face challenges in achieving reliable and accurate real-virtual alignment over large working volumes due to limitations in user calibration methods and the assumption of a stationary eye rotation center, leading to inaccurate registration of virtual objects with real objects.
Innovation Solution
A method and system that estimate the 3D location of the Stiles-Crawford Peak (SCP) as the true projection point of the eye using multiple image capturing devices and geometric calibration data, allowing for precise alignment of virtual objects with real objects by calculating the actual SCP position in three-dimensional space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual calibration methods like SPAAM are used to align the HMD with the viewer's eyes, then initial alignment can be achieved, but the calibration becomes unreliable and inaccurate when the display shifts position on the viewer's head
Solution Approach 1:
The system continuously tracks the relative position between the HMD and the viewer's eyes using eye-tracking cameras and processors that calculate transformation matrices. This real-time feedback allows the system to detect when the display shifts and automatically compensate for the shift, maintaining accurate alignment between virtual and real objects throughout the viewing session
Solution Approach 2:
The system performs preliminary calibration to establish the initial transformation matrix between the HMD coordinate system and the eye rotation center coordinate system. This preliminary calibration serves as a baseline that is then continuously refined through real-time tracking, allowing the system to maintain accuracy even as conditions change
2Ease of operation
If the eye rotation center is assumed to be stationary for calibration purposes, then calibration can be performed once, but the registration becomes inaccurate when the user's eye moves
Solution Approach 1:
The system transitions from a static calibration model to a dynamic tracking model. Instead of assuming the eye rotation center remains stationary, the system continuously tracks eye position and updates the transformation matrix in real-time. This dynamic approach maintains registration precision throughout eye movements while requiring only a single initial calibration
Solution Approach 2:
Real-time eye tracking provides continuous feedback on eye position, allowing the system to adjust the virtual object positioning dynamically. The processor calculates updated transformation matrices based on current eye position data, ensuring accurate registration even as the user moves their eyes or head
3Stability of the object's composition
If the HMD is rigidly attached to the head to maintain calibration, then alignment stability improves, but the display cannot be adjusted for different users or comfort preferences
Solution Approach 1:
The system allows the HMD to move freely on the user's head without compromising alignment stability. The real-time eye tracking and dynamic transformation matrix updates compensate for any positional changes, enabling both rigid stability through continuous correction and flexible adaptability for different users and comfort preferences
Solution Approach 2:
Continuous eye tracking provides feedback that allows the system to adapt to different HMD positions on the user's head. The processor automatically recalculates the transformation matrix based on current eye-HMD geometry, maintaining accurate registration regardless of how the display is positioned or adjusted
Data Source
AI summary
The present invention relates to a system for alignment between real and virtual objects in a head-mounted optical see-through display. In an embodiment, the system includes a tracking system including a processor, a headgear attached with the head-mounted optical see-through display, the 5 head-mounted optical see-through display includes at least two cameras mounted on a rigid frame, at least one object may be fixed or mobile including a plurality of marker points, the tracking system is operatively coupled to the headgear and the object, the processor is configured for: capturing two perspective images of the substantially circular entrance pupil of at least one 0 eye and relaying the image data to the processor, a memory device coupled to the processor and containing the geometric calibration data of the at least two cameras and the pre-calibrated transformation between the cameras. The processor extracts the boundary between the entrance pupil and the iris, calculates the projected center of the boundary in the individual images and 5 using the calibration data estimates the center of the entrance pupil in three dimensional space in relation to the cameras.