Foveated Rendering for VR Eye Tracking Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphical display devices, particularly head-mounted displays (HMDs) for virtual reality (VR), face challenges in efficiently rendering images due to the need for high frame rates and high resolution, which increases computational load and power consumption.
Innovation Solution
The implementation of foveated rendering techniques, which adjust pixel resolution and vertex density based on the user's gaze direction, allowing for reduced computational resources on non-foveal regions and adaptive compression of images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If planar rendering is used with uniform pixel resolution across the entire screen, then the rendering process is simple and computationally efficient, but the visual quality and user experience in the foveal region are insufficient
Solution Approach 1:
The patent implements foveated rendering that applies different pixel resolution to different regions of the display. The foveal region receives high-resolution rendering to maintain visual quality, while peripheral regions use lower resolution to reduce computational load. This local differentiation of rendering quality directly resolves the contradiction between maintaining high pixel resolution and preserving computational efficiency.
2Reliability
If high frame rate rendering is implemented to prevent motion sickness in VR, then user comfort is improved, but power consumption and computational load increase significantly
Solution Approach 1:
By applying foveated rendering, the system maintains high frame rates in the foveal region where users need the most visual fidelity for comfort, while reducing frame rates in peripheral regions. This selective approach allows the system to meet the high frame rate requirement for user comfort without proportionally increasing overall power consumption across the entire display.
3Use of energy by moving object
If foveated rendering is implemented to reduce computational load, then power consumption decreases, but the complexity of the rendering system increases
Solution Approach 1:
The patent implements dynamic foveated rendering that adapts the rendering quality based on real-time eye tracking data. The system dynamically adjusts which regions receive high-resolution rendering and which regions use lower resolution, based on the user's gaze position. This dynamic adaptation allows the system to optimize the balance between power consumption and rendering quality without requiring complex manual configuration.
4Productivity
If eye tracking is added to enable foveated rendering, then rendering efficiency is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses eye tracking technology as an intermediary to bridge the gap between the user's visual attention and the rendering system. The eye tracker provides gaze position data that serves as input for the foveated rendering algorithm, allowing the system to automatically identify which regions need high-resolution rendering. This intermediary approach enables rendering efficiency improvements without requiring direct complex interaction between the display and rendering components.
Data Source
AI summary
Gaze tracking data representing a user's gaze is analyzed to determine one or more regions of interest. One or more gaze tracking parameters are determined from the gaze tracking data. Adjusted foveation data is determined representing an adjusted size and/or shape of one or more regions of interest in one or more images to be subsequently presented to the user based on the one or more gaze tracking parameters. The compression of the one or more transmitted images is adjusted so that fewer bits are needed to transmit data for portions of an image outside the one or more regions of interest than for portions of the image within the one or more regions of interest. Adjusting compression of the transmitted image(s) includes decreasing a size of the foveal region for a subset of the one or more images that are presented to the user during the saccade or blink.