Temporal Foveated Rendering for GPU Workload Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial-reality systems face challenges in efficiently rendering high-quality visuals, particularly in peripheral vision, which leads to increased GPU workload and resource consumption.
Innovation Solution
Temporal foveated rendering reduces GPU workload by alternating between rendering a high-quality 'inset' area and a lower-quality 'outset' area for every other frame, with the outset area being filled using previously rendered frames and utilizing space and time warping parameters to generate a predicted image for compositing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-quality rendering is applied to the entire field of view including peripheral vision, then visual quality is improved, but GPU workload and resource consumption increase
Solution Approach 1:
The patent applies different rendering qualities to different regions of the field of view. The central foveal region receives high-quality rendering with full pixel information, while the peripheral outset region receives lower-quality rendering or is skipped entirely in alternate frames. This local differentiation resolves the contradiction by concentrating computational resources where visual acuity is highest while reducing workload in peripheral areas where the human eye is less sensitive to quality degradation.
Solution Approach 2:
The patent implements temporal foveated rendering where the peripheral outset area is rendered in alternate frames rather than every frame. The system alternates between rendering the full frame and rendering only the central inset area, using previously rendered frames to fill in the peripheral regions. This periodic action reduces the average GPU workload while maintaining acceptable visual quality through temporal redundancy.
2Productivity
If rendering frequency is reduced in peripheral view to save GPU compute, then resource consumption is reduced, but visual completeness may be compromised
Solution Approach 1:
The patent renders the peripheral outset area in advance during frames where it is fully rendered, storing this information for use in subsequent frames. By performing preliminary rendering of the peripheral regions in alternate frames, the system ensures that visual information is available when needed, maintaining visual completeness while reducing the average rendering frequency and GPU compute requirements.
Solution Approach 2:
The patent uses previously rendered frame data to fill in the peripheral outset area in frames where full rendering is skipped. The system copies and reuses visual information from prior frames to maintain visual completeness in the peripheral regions, reducing the need for redundant rendering operations and thereby saving GPU compute resources while preserving visual integrity.
Data Source
AI summary
In one embodiment, a method includes accessing a first image corresponding to a first frame of a video stream, where the first image has complete pixel information, rendering a provisional image corresponding to a second frame of the video stream subsequent to the first frame, where the provisional image has a first area with complete pixel information and a second area with incomplete pixel information, generating a predicted image corresponding to the second frame by re-projecting at least an area of the first image according to one or more warping parameters, and generating a second image corresponding to the second frame by compositing the rendered provisional image and the predicted image.