Foveated Rendering with Variable Framerates for VR Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer graphics rendering techniques are inefficient, leading to increased latency, power consumption, and potential motion-to-photon latency issues, particularly in VR applications, due to the rendering of graphics content at uniform resolutions and framerates without considering the user's gaze direction.
Innovation Solution
Implementing foveated rendering techniques that vary resolution and framerate based on the user's gaze, with higher resolution and framerate applied to the foveal vision region and lower settings for peripheral vision, combined with reprojection algorithms to reduce motion-to-photon latency and artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If uniform resolution and framerate are used for rendering graphics content, then consistent image quality is maintained across all regions, but rendering efficiency decreases and power consumption increases
Solution Approach 1:
The patent applies local quality by rendering different regions of the display at different resolutions and framerates based on user gaze. The foveal region (where the user is looking) is rendered at high resolution and high framerate, while peripheral regions are rendered at lower resolution and framerate. This resolves the contradiction by maintaining high quality where needed while improving overall rendering efficiency and reducing power consumption.
Solution Approach 2:
The patent implements dynamics by making resolution and framerate adaptive rather than static. The rendering parameters dynamically change based on real-time eye tracking data, adjusting which regions receive high-quality rendering resources. This allows the system to optimize rendering efficiency and power consumption while maintaining quality in the user's focal point.
2Manufacturing precision
If high resolution rendering is applied to all regions, then image quality is maintained, but latency increases
Solution Approach 1:
The patent reduces latency by applying high resolution rendering only to the foveal region rather than uniformly across the entire display. Since the human visual system is most sensitive to detail in the central vision area, this selective approach maintains perceived image quality while significantly reducing the computational load and rendering time.
Solution Approach 2:
The patent applies partial action by rendering only the necessary portion of the scene at high quality. Instead of rendering the entire field of view at high resolution, it focuses computational resources on the partial region (foveal area) that the user is actually looking at, thereby reducing latency while maintaining acceptable overall image quality.
3Productivity
If variable framerates are used in different regions, then rendering efficiency improves, but artifacts may appear at region boundaries
Solution Approach 1:
The patent manages artifacts by carefully controlling the transition between high and low framerate regions. The system uses eye tracking data to define smooth boundaries and may apply blending techniques or motion compensation to reduce visible artifacts at the transition zones between regions with different framerate settings.
Solution Approach 2:
The patent employs preliminary action by predicting future gaze positions and pre-rendering content accordingly. This allows the system to maintain variable framerates while reducing artifacts, as the rendering pipeline can prepare content in advance for anticipated eye movements, smoothing out transitions between different framerate regions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Example techniques are described for generating graphics content by obtaining a rendering command for a first frame of the graphics content, rendering a full frame based on the rendering command for the first frame, storing the full frame in a buffer, obtaining a rendering command for a second frame of the graphics content, obtaining an eye position of a user, rendering a partial frame based on the rendering command for the second frame and the eye position of the user, obtaining the full frame from the buffer, and outputting the second frame, wherein the second frame is based on the full frame and the partial frame.