Display Engine Post-Rendering Processing for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial reality systems face challenges in rendering high-quality, immersive graphics due to computational intensity and latency issues, particularly in head-mounted displays, which can cause discomfort and virtual reality sickness due to geometric distortions and slow response times to user movements.
Innovation Solution
A system and method for generating artificial reality scenes in head-mounted displays, where a primary rendering component handles complex graphics rendering and a display engine refines and finalizes the output, using multiple hardware blocks to correct distortions and rapidly update graphics in response to user movements, enabling high frame rates and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-quality graphics rendering is performed in artificial reality systems, then image quality and immersion are improved, but computational load and processing time increase significantly
Solution Approach 1:
The rendering system is divided into two separate components: a primary rendering component that handles complex graphics rendering tasks, and a display engine in the head-mounted display that performs post-rendering processing including geometric distortion correction, chromatic aberration correction, and other refinements. This segmentation allows each component to specialize in specific tasks, improving overall efficiency and quality while distributing computational load appropriately.
Solution Approach 2:
The primary rendering component performs preliminary rendering of graphics content before it is transmitted to the head-mounted display. By pre-processing the graphics content with basic rendering operations, the system reduces the computational burden on the display engine and ensures that fundamental rendering quality is established before post-processing corrections are applied.
2Manufacturing precision
If geometric distortion correction is applied in the display engine, then image accuracy is improved, but processing complexity increases
Solution Approach 1:
A distortion correction lookup table is introduced as an intermediary data structure that stores pre-computed correction values for geometric distortions and chromatic aberrations. The display engine queries this lookup table during post-processing rather than performing complex real-time calculations, significantly reducing processing complexity while maintaining high correction accuracy. The lookup table acts as a mediator between the rendering component and display output.
3Speed
If rapid graphics updates are performed to respond to user movements, then latency is reduced, but processing power requirements increase
Solution Approach 1:
The rendering pipeline is segmented such that the primary rendering component handles computationally intensive tasks at lower frame rates, while the display engine performs lightweight post-processing operations at higher frame rates. This segmentation enables the head-mounted display to achieve high update speeds for responsive user interaction without requiring the full processing power needed for complete rendering at those same high rates.
Solution Approach 2:
The display engine performs partial rendering operations—specifically post-processing corrections on already-rendered graphics content—rather than performing complete rendering operations. This partial action approach allows the system to achieve high update rates for user movement responses by only processing the necessary corrections rather than full rendering pipelines, reducing the processing power required at high frame rates.
4Manufacturing precision
If chromatic aberration correction is applied, then color accuracy is improved, but processing time increases
Solution Approach 1:
A chromatic aberration correction lookup table serves as an intermediary that stores pre-computed color correction values for different spatial locations and wavelengths. The display engine uses this lookup table to perform rapid color corrections by simple table lookups and interpolations rather than performing complex real-time optical calculations, significantly reducing processing time while maintaining high color accuracy.
Data Source
AI summary
In one embodiment, a computing system may access surfaces and texel data of an artificial reality scene. The surfaces may be generated based on a first viewing position of a viewer. The system may determine tiles on a display to test for a visibility of the surfaces from a second viewing position. The tiles may include first tiles that need more computational resources and second tiles that need less computational resources. The system may determine a tile order which interleaves the first and second tiles. The system may generate rays based on the tile order. The system may determine the visibility of the surfaces from the second viewing position based on the ray-surface intersections. The system may generate color values of a subframe based on the surface visibility and the texel data. The system may provide the color values to the display.