Depth-Based Foveated Rendering for AR Fixation-Driven Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) technologies face challenges in providing a comfortable and natural presentation of virtual image elements amidst real-world imagery due to the complexity of the human visual system, leading to high computational demands and resource usage.
Innovation Solution
A display system that adjusts the resolution of virtual objects based on their proximity to the user's fixation point, rendering high resolution at the fixation point and reducing resolution away from it, utilizing waveguides and sensors to track eye movements and light conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If uniform high resolution is applied to all virtual objects in the field of view, then visual quality is improved, but computational resources and processing power are excessively consumed
Solution Approach 1:
The patent implements local quality by varying the resolution of virtual objects based on their proximity to the user's fixation point. High resolution is applied locally to objects near the fixation point where visual acuity is highest, while lower resolution is applied to objects farther away. This is achieved through depth-based foveated rendering that dynamically adjusts rendering quality according to the user's gaze location and the depth of virtual objects, thereby optimizing the balance between visual quality and computational resource consumption.
2Manufacturing precision
If high resolution is rendered for all virtual objects, then image quality is improved, but memory requirements and processing power increase significantly
Solution Approach 1:
The system applies local quality by allocating high resolution only to virtual objects in the foveal region near the user's fixation point, while using reduced resolution for objects in peripheral vision. This selective resolution allocation significantly reduces the total memory footprint and processing requirements while maintaining high image quality where it matters most - in the user's direct line of sight.
3Manufacturing precision
If full resolution is maintained across the entire field of view, then presentation quality is improved, but energy consumption increases
Solution Approach 1:
The patent implements local quality through depth-based foveated rendering that dynamically adjusts resolution based on the user's fixation point and the depth of virtual objects. By concentrating computational resources and energy on rendering high-resolution images only in the foveal region rather than uniformly across the entire field of view, the system significantly reduces overall energy consumption while maintaining high presentation quality where the user is most likely to focus attention.
Data Source
AI summary
Methods and systems for depth-based foveated rendering in a display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include monitoring eye orientations of a user of the display system. A fixation point can be determined based on the eye orientations, the fixation point representing a three-dimensional location with respect to a field of view. Location information of virtual object(s) to present is obtained, with the location information including three-dimensional position(s) of the virtual object(s). A resolution of the virtual object(s) can be adjusted based on a proximity of the location(s) of the virtual object(s) to the fixation point. The resolution can also be adjusted based on color, ambient illumination level, and/or other considerations. The virtual object(s) are presented by the display system according to the adjusted resolution(s).