Foveated Compression for VR Rendering Latency and Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graphics processing systems face challenges in efficiently rendering high-resolution graphical content, particularly in virtual reality and augmented reality applications, due to increased computational demands and latency issues, especially when split rendering is employed.
Innovation Solution
Implementing foveated compression techniques that downscale and upscale graphical content based on the user's gaze direction, preserving high quality in the central fovea and reducing resolution in the periphery, thereby optimizing bandwidth and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-resolution graphical content is rendered for VR/AR applications, then visual quality is improved, but computational demand and latency increase
Solution Approach 1:
The patent applies different resolution levels to different regions of the displayed image based on the user's gaze direction. The central foveal region maintains high resolution for sharp visual quality, while peripheral regions use lower resolution to reduce computational load and bandwidth requirements. This local quality differentiation directly resolves the contradiction by optimizing visual quality where needed without uniformly increasing computational demand across the entire image.
Solution Approach 2:
The image is divided into multiple regions (foveal and peripheral zones) that are processed independently at different resolution levels. This segmentation allows the system to handle computational tasks more efficiently by processing only the necessary high-resolution portions, thereby reducing overall computational demand while maintaining quality in critical viewing areas.
2Manufacturing precision
If high-resolution graphical content is rendered, then visual quality is improved, but bandwidth requirements increase
Solution Approach 1:
Different resolution levels are applied locally to different image regions based on gaze direction. The central foveal region maintains high resolution to preserve visual quality, while peripheral regions use lower resolution to reduce the total data volume transmitted, thereby reducing bandwidth requirements without sacrificing quality in the critical viewing area.
Solution Approach 2:
The image data is segmented into high-resolution foveal portions and low-resolution peripheral portions, allowing the system to transmit only the necessary amount of data for each region. This segmentation reduces the total bandwidth consumption while maintaining visual quality where the user is looking.
3Manufacturing precision
If uniform resolution is applied across the entire frame, then visual quality is consistent, but computational efficiency decreases
Solution Approach 1:
Instead of applying uniform resolution, the system dynamically adjusts resolution locally based on the user's gaze direction. The foveal region receives high resolution for sharpness, while peripheral regions use lower resolution, thereby improving computational efficiency without compromising quality in the critical viewing area.
Solution Approach 2:
The resolution settings are made dynamic rather than static, adapting in real-time based on gaze direction. This dynamic adjustment allows the system to optimize computational efficiency by allocating resources where they are most needed, rather than maintaining constant uniform resolution across the entire frame.
Data Source
AI summary
The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can render at least one frame including display content at a server. Aspects of the present disclosure can also downscale the at least one frame including the display content, where a downscaling rate of one or more portions of the at least one frame is based on a location of each of the one or more portions. Moreover, aspects of the present disclosure can communicate the downscaled at least one frame including the display content to a client device. Aspects of the present disclosure can also encode the downscaled at least one frame including the display content. Further, aspects of the present disclosure can decode the encoded at least one frame including the display content. Aspects of the present disclosure can also upscale the at least one frame including the display content.