Depth-Based Foveated Rendering for AR Display Resolution Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented and virtual display systems require significant computational resources, memory, and processing time, but they do not efficiently simulate three-dimensional imagery, and they are not comfortable or realistic.
Innovation Solution
The display system reduces the resolution of virtual content positioned away from the user's fixation point, using high resolution at or near the fixation point, and lower resolutions farther away, adjusting resolution along the z-axis and optionally the x and y-axes, based on the user's three-dimensional fixation point, and utilizing waveguides to present virtual content on different depth planes with varying wavefront divergence and binocular disparity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high resolution is maintained across the entire field of view, then image quality is improved, but computational resources and energy consumption increase significantly
Solution Approach 1:
The patent applies local quality by rendering different regions of the visual field at different resolutions. The foveal region (center of gaze) is rendered at high resolution to match human visual acuity, while peripheral regions are rendered at progressively lower resolutions. This resolves the contradiction by maintaining high image quality only where the human eye can actually perceive it, thereby reducing overall computational load and energy consumption without sacrificing perceptual image quality.
Solution Approach 2:
The patent segments the visual field into multiple depth planes (e.g., near, intermediate, far) and applies different resolution levels to each plane. By dividing the rendering task into depth-based segments, the system can allocate computational resources efficiently, processing high-resolution content only for the focal plane while using lower resolution for other depth planes, thus reducing total energy consumption while maintaining perceived quality.
2Measurement precision
If full resolution rendering is applied to all depth planes, then visual fidelity is improved, but processing time and computational power requirements increase
Solution Approach 1:
The patent implements local quality by applying high resolution rendering only to the focal depth plane where the user's gaze is directed, while rendering other depth planes at lower resolutions. This approach maintains visual fidelity at the location that matters most to the user while significantly reducing the total processing time required to render the complete scene across all depth planes.
3Measurement precision
If high resolution content is displayed across all depth planes, then realism is improved, but system complexity and processing requirements increase
Solution Approach 1:
The patent applies local quality by rendering high-resolution content only at the focal depth plane corresponding to the user's point of regard, while using lower resolution for other depth planes. This maintains realism where the user's attention is focused while reducing overall system complexity and processing requirements.
Data Source
Figure 1
Figure 2
Figure 3A~3C
AI summary
Methods and systems for depth-based foveated rendering in the display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include determining a fixation point of a user's eyes. Location information associated with a first virtual object to be presented to the user via a display device is obtained. A resolution-modifying parameter of the first virtual object is obtained. A particular resolution at which to render the first virtual object is identified based on the location information and the resolution-modifying parameter of the first virtual object. The particular resolution is based on a resolution distribution specifying resolutions for corresponding distances from the fixation point. The first virtual object rendered at the identified resolution is presented to the user via the display system.