AR Foveated Rendering Using Depth-Based Fixation Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) technologies face challenges in providing a comfortable, natural-feeling, rich presentation of virtual image elements amidst real-world imagery due to the complexity of the human visual perception system.
Innovation Solution
A display system that monitors eye movements to determine a fixation point and adjusts the resolution of virtual objects based on their proximity to this point, rendering high resolution at or near the fixation point and reducing resolution further away, thereby conserving computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high resolution is provided for all virtual objects in the field of view, then the presentation quality is improved, but the computational resources and processing power requirements increase significantly
Solution Approach 1:
The patent implements foveated rendering that provides high rendering resolution locally at the fixation point where the user's eyes are focused, while reducing resolution for virtual objects away from the fixation point. This local quality approach matches the human visual system's natural attention distribution, providing high detail where needed (at the fovea) and lower detail where less important (in peripheral vision), thereby reducing overall computational power requirements while maintaining presentation quality.
Solution Approach 2:
The patent segments the three-dimensional field of view into multiple resolution zones based on proximity to the fixation point. Virtual objects are categorized into different resolution groups: high resolution for objects near the fixation point, and progressively lower resolution for objects farther away. This segmentation allows the system to allocate computational resources efficiently across different spatial regions rather than uniformly treating all objects equally.
2Manufacturing precision
If high resolution is provided for all virtual objects, then the visual quality is improved, but the memory usage increases
Solution Approach 1:
The system applies local quality by maintaining high-resolution rendering data only for virtual objects in the foveal region where the user is fixating, while using lower-resolution data for peripheral objects. This selective memory allocation reduces the total quantity of high-resolution data that must be stored in memory, allowing the system to provide high visual quality where needed without requiring excessive memory capacity.
3Manufacturing precision
If high resolution is provided for all virtual objects, then the presentation richness is improved, but the rendering latency increases
Solution Approach 1:
The patent segments the rendering task into priority-based groups based on object proximity to the fixation point. High-priority rendering is performed for objects near the fixation point where visual quality is critical, while lower-priority rendering is performed for peripheral objects. This segmentation enables the system to reduce overall rendering latency by completing critical high-resolution renders first and using progressively coarser rendering for less time-sensitive peripheral elements.
4Productivity
If eye tracking and foveated rendering are implemented, then the computational efficiency is improved, but the device complexity increases
Solution Approach 1:
The system implements a feedback loop where eye movement data from the eye tracker continuously updates the fixation point location, which then drives the dynamic resolution adjustment of virtual objects. This feedback mechanism enables the system to adapt rendering quality in real-time based on user attention, improving computational efficiency by focusing resources on currently viewed content while managing the added system complexity through automated control.
Data Source
AI summary
Methods and systems for depth-based foveated rendering in a display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include monitoring eye orientations of a user of the display system. A fixation point can be determined based on the eye orientations, the fixation point representing a three-dimensional location with respect to a field of view. Location information of virtual object(s) to present is obtained, with the location information including three-dimensional position(s) of the virtual object(s). A resolution of the virtual object(s) can be adjusted based on a proximity of the location(s) of the virtual object(s) to the fixation point. The virtual object(s) are presented by the display system according to the adjusted resolution(s).