Head-Mounted Graphics Rendering with Gaze-Driven Foveated Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Designing head-mounted devices with near-eye displays that present virtual content to users without being excessively heavy, bulky, or consuming excessive power, while maintaining sufficient optical performance, is challenging.
Innovation Solution
Incorporating a context-aware rendering configurer and renderer that generate foveated virtual content based on eye position and binocular gaze information, using optics with fixed and removable prescription lenses, to optimize resolution within hardware constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the display and optics are designed to deliver high resolution virtual content to the eye box, then the optical performance is improved, but the device becomes heavier and bulkier
Solution Approach 1:
The system implements foveated rendering that allocates different resolution levels to different regions of the virtual display. High resolution is concentrated only in the foveated region corresponding to the user's gaze, while peripheral regions use lower resolution. This local quality differentiation allows the optics and display to be lighter while still delivering high resolution where needed.
Solution Approach 2:
The system dynamically adjusts the resolution and characteristics of virtual content based on real-time eye position and gaze tracking. The foveated region moves with the user's gaze, and the resolution allocation changes adaptively. This dynamic adaptation allows the system to maintain high optical performance without requiring the entire display and optical system to operate at maximum resolution continuously.
2Manufacturing precision
If the display and optics are designed to deliver high resolution virtual content to the eye box, then the optical performance is improved, but the device consumes excessive power
Solution Approach 1:
By concentrating high resolution rendering only in the foveated region rather than across the entire display, the system significantly reduces the computational load and power consumption of the renderer. The optics also require less power to deliver content at the eye box when the overall resolution requirement is reduced from full-display to localized high resolution.
Solution Approach 2:
The system dynamically adjusts rendering resolution and optical parameters based on real-time gaze tracking. When the user's gaze shifts, the high resolution foveated region repositions and adapts its characteristics. This dynamic adaptation allows the system to maintain high optical performance at the point of gaze while reducing overall power consumption by not maintaining maximum resolution across the entire field of view.
3Manufacturing precision
If the renderer generates virtual content at peak resolution exceeding the hardware limit, then the user experience is improved, but the device complexity increases
Solution Approach 1:
The renderer generates virtual content with varying resolution characteristics across different regions. The foveated region receives high resolution content while peripheral regions receive lower resolution content. This local quality differentiation simplifies the overall rendering system by avoiding the need to render the entire display at maximum resolution, while still providing excellent user experience in the critical foveated area.
Solution Approach 2:
The system dynamically adjusts the resolution characteristics of rendered content based on real-time eye position and gaze data. The foveated region's resolution and characteristics are adaptively modified to match the user's gaze position. This dynamic adaptation simplifies the rendering system by allowing it to optimize performance based on actual usage patterns rather than maintaining maximum complexity throughout.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution ensures that the virtual content is rendered at a peak resolution equal to or above the collective upper resolution limit of the device's hardware, enhancing user experience without increasing size or power consumption.
Implementation Method 1
The device may include optics that direct the light to eye boxes
Data Source
AI summary
An electronic device such as a head-mounted device may include a renderer for generating virtual content, displays that generate light containing the virtual content, and optics that direct the light to eye boxes. The optics may include fixed lenses and optionally removable prescription lenses. The device may include gaze tracking sensors that measure eye position information at the eye boxes and that measure binocular gaze information between the eye boxes. The renderer may generate the virtual content according to a rendering configuration. The rendering configuration may be generated based on the eye position information, the binocular gaze information, hardware constraints of the optics, hardware constraints of the display, and/or information about the virtual content to be displayed. The renderer may render the virtual content with a peak resolution that exceeds a collective upper resolution limit of the optics and the display. The rendered virtual content may include foveated virtual content.