World-Aligned Foveated Rendering for VR HMDs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Foveated rendering in VR and AR systems often results in noticeable aliasing artifacts in low-acuity regions, especially during user head movement, which disrupts the sense of immersion due to dynamic artifacts moving against content.
Innovation Solution
The solution involves 'world-aligning' low-acuity regions to the content of the virtual world, maintaining their position relative to the virtual scene's coordinates, and projecting them onto the display surface, ensuring artifacts shift with the content as the user's field-of-view changes, reducing perceptibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If pixels in low-acuity regions are rendered at lower resolutions to reduce power and computing resources, then power consumption and computing resource usage are reduced, but aliasing artifacts become noticeable and disrupt user immersion
Solution Approach 1:
The system dynamically adjusts the rendering resolution based on the user's gaze direction and head movement. The low-acuity regions are rendered at lower resolution only when the user is not actively looking at them, while high-acuity regions maintain full resolution. This dynamic adaptation allows the system to reduce power consumption without consistently degrading image quality in regions where the user might notice artifacts.
Solution Approach 2:
The patent applies different rendering qualities to different regions of the visual field. The foveal region (central vision) is rendered at full native resolution, while peripheral low-acuity regions are rendered at lower resolutions. This local differentiation allows power savings in peripheral regions without significantly impacting the user's visual experience, as the human eye is less sensitive to artifacts in peripheral vision.
2Productivity
If low-acuity regions are rendered at lower resolutions, then computing resources are reduced, but dynamic aliasing artifacts become noticeable during head movement
Solution Approach 1:
The system continuously tracks head movement and dynamically updates which regions require high-resolution rendering. When the user moves their head, the system recalculates the foveal region and adjusts rendering resolution accordingly. This ensures that low-acuity regions are only rendered at lower resolution when appropriate, reducing computing resources while minimizing the visibility of dynamic artifacts during head movement.
Solution Approach 2:
The system performs preliminary rendering of low-acuity regions at lower resolution before the user's gaze actually reaches those areas. By anticipating where the user will look next based on current gaze direction and head movement trends, the system can prepare lower-resolution content in advance, reducing computing load while ensuring that high-quality rendering is ready when needed.
Data Source
AI summary
A display device, such as a head mounted device (HMD), displays a virtual scene. The display device includes a motion tracker for detecting rotation and/or translation of the display device. The display device also includes a processor that is configured to determine, in response to the detected, an orientation of the display device relative to a plurality of world-aligned viewing frustums that are stationary relative to the virtual scene. The processor is also configured to identify a set of those world-aligned viewing frustums that overlap with an output field of view. The processor is further configured to render pixels of the set of those world-aligned viewing frustums that overlap with an output field of view and upsample the rendered pixels to generate values of display pixels for presentation by the display device.