Predicted-Gaze Foveated Rendering for Low-Latency Image Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image rendering systems suffer from latency issues, causing areas of interest to be incorrectly rendered due to the user's gaze shifting before the image update can occur, especially in computer-generated graphics where rendering times are long.
Innovation Solution
A method and system that predicts the user's point of regard at a future time by analyzing gaze data and image content, allowing for dynamic adjustment of image parameters such as resolution and detail around the predicted point of regard, while reusing previous frames and rendering only the foveated area with high quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If foveated rendering is used to render high resolution around the point of regard, then image quality in the area of interest is improved, but latency increases due to the time needed for calculating the point of regard and rendering the scene
Solution Approach 1:
The system performs preliminary calculations of the point of regard and begins rendering the foveated area in advance before the actual gaze point is known. By predicting where the user will look and pre-rendering that region, the system reduces latency while maintaining high image quality in the area of interest.
Solution Approach 2:
The rendering resolution dynamically adapts based on the predicted point of regard. The system renders high resolution only in the predicted foveated area while using lower resolution elsewhere, and this dynamic adjustment occurs continuously as the user's gaze moves, optimizing both quality and performance.
2Manufacturing precision
If the resolution is increased around the point of regard, then image clarity is improved, but computational resources and processing time increase
Solution Approach 1:
The system applies different quality levels to different regions of the display. High resolution rendering is applied locally only to the predicted foveated area where the user is expected to look, while peripheral areas are rendered at lower resolution. This local quality differentiation maintains image clarity where needed while significantly reducing overall computational resource consumption.
Data Source
AI summary
Method for subsequently displaying a stream of images in a display area of a display device, wherein a first area within the display area is determined around a point of regard. A first image of the stream of images is displayed in the display area such that a first part of the first image, which is displayed in the first area, is displayed according to a first parameter value of at least one parameter and a second part of the first image, which is displayed in at least one second area outside the first area, is displayed according to a second parameter value of the at least one parameter. Moreover, the determining of the point of regard is performed as predicting the point of regard for a certain future point of time, at which the first image is displayed.