Foveated HMD Optics for Eye Tracking and Jitter Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current head-mounted displays (HMDs) struggle to provide high-resolution, comfortable, and immersive mixed reality gaming experiences due to limitations in optical systems, inability to sense and adjust to eye movements, project imagery at varying field of view resolutions, and correct for jitter caused by fast head movements.
Innovation Solution
The system employs an eye tracking system, head tracking system, and a controller to generate field mappings with varying resolutions, a light generator to project color-encoded light, and a foveated optical system to provide higher resolution in the central field of view, along with a switchable reflector and liquid crystal wave front correctors to adjust imagery based on eye orientation and movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a conventional optical system is used in head-mounted displays, then the device structure is simple, but the resolution and field of view are limited
Solution Approach 1:
The optical system is divided into multiple independent beam launchers, each responsible for a specific region of the field of view. Each beam launcher contains its own optical path with lenses and mirrors, allowing independent optimization of resolution for different regions while maintaining overall system functionality.
Solution Approach 2:
Different regions of the field of view are assigned different optical characteristics. The central region receives high-resolution imaging through focused optical paths, while peripheral regions use broader beam paths. This local optimization allows high resolution where needed without requiring the entire optical system to be complex.
2Adaptability or versatility
If the display projects imagery at a fixed resolution, then the system is simple to control, but it cannot adapt to eye movements and head motions
Solution Approach 1:
The optical system dynamically adjusts beam paths and focal points based on real-time eye tracking and head motion data. The controller modifies optical element positions and orientations to maintain proper image alignment with the user's gaze direction, enabling adaptation to continuous eye movements and head motions.
Solution Approach 2:
Eye tracking sensors and head motion detectors provide continuous feedback to the controller, which then adjusts the optical system parameters accordingly. This closed-loop feedback mechanism ensures the displayed imagery remains properly aligned with the user's visual attention despite eye movements or head position changes.
3Speed
If fast head movements occur, then the user can quickly change viewing direction, but jitter is introduced into the displayed imagery
Solution Approach 1:
The system predicts and compensates for jitter by continuously tracking head motion and proactively adjusting optical elements before the full effect of head movement is realized. The controller anticipates image displacement based on head velocity and position, pre-positioning optical components to maintain image stability during rapid head movements.
Solution Approach 2:
The mechanical jitter introduced by head movements is replaced by electronic compensation through software control of the optical system. The controller calculates correction values based on head motion sensors and adjusts optical element positions accordingly, substituting mechanical stability with electronic control to maintain image reliability during fast head movements.
4Manufacturing precision
If a single field of view is used, then the optical system is simple, but it cannot provide different resolutions for central and peripheral vision
Solution Approach 1:
The field of view is segmented into multiple regions with different resolution requirements. The optical system divides the display into separate beam paths, with central beams receiving high-resolution processing and peripheral beams receiving lower-resolution processing. This segmentation allows different resolution levels across different spatial regions without requiring the entire system to operate at maximum resolution.
Solution Approach 2:
Each region of the field of view receives optimized optical treatment according to its specific requirements. The central region uses high-resolution optical paths with precise focusing, while peripheral regions use broader, lower-resolution paths. This local quality differentiation achieves the desired resolution distribution while keeping the overall system complexity manageable by applying complexity only where necessary.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables high-resolution, comfortable, and immersive mixed reality gaming by maintaining clear and stable augmentation imagery across a full field of view and range of eye movements, adapting to user-specific eye characteristics, and correcting for jitter.
Implementation Method 1
liquid crystal wave front correctors to adjust imagery based on eye orientation and movement
Implementation Method 2
switchable reflector and liquid crystal wave front correctors to adjust imagery based on eye orientation and movement
Implementation Method 3
a foveated optical system to provide higher resolution in the central field of view
Implementation Method 4
a light generator to project color-encoded light
Data Source
AI summary
A system and method of projecting augmentation imagery in a head-mounted display are disclosed. The system includes an eye tracking system configured to generate eye tracking data at an eye-tracking rate in response to a position of a pupil of a user, a head tracking system configured to estimate a change in orientation of the user's head at a head tracking rate and generate jitter data at a jitter sensing rate, a controller configured generate a field mapping in response to render data at a projector update rate, a light generator configured to generate a color encoded light associated with the augmentation imagery, the light generator includes a 2D pixelated display and an optical system configured to image the color encoded light into a field of view of the user, wherein the optical system includes a foveated optical system.