Predictive Foveated Rendering for Low-Latency Image Display
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image rendering technologies suffer from latency issues, causing areas of interest to be incorrectly rendered due to user gaze movement, especially in computer-generated graphics, leading to suboptimal display quality.
Innovation Solution
A system and method that predicts the user's point of regard for a future time by analyzing gaze data and image content, allowing for dynamic adjustment of image parameters such as resolution and detail around the predicted point of regard, while reusing previous frames and rendering only the foveated area with high quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If foveated rendering is used to render high resolution around the point of regard, then image display quality is improved, but latency increases due to calculation time
Solution Approach 1:
The system performs preliminary actions by predicting the future point of regard before the actual gaze occurs. Eye movement parameters are analyzed in advance to forecast where the user will look, allowing the rendering system to prepare high-resolution images for the anticipated area of interest before the user's gaze actually reaches that location, thereby reducing perceived latency.
Solution Approach 2:
The system dynamically adjusts the point of regard and first area based on real-time eye tracking data and predicted gaze movements. Instead of using a static rendering approach, the system continuously updates the high-resolution rendering region to match the user's dynamic gaze patterns, ensuring optimal image quality is delivered to the correct location at the right time.
2Manufacturing precision
If the point of regard is determined and rendered with high resolution, then image display quality is improved, but the user gaze may have moved away by display update time
Solution Approach 1:
The system performs preliminary gaze prediction to determine where the user will look at the time of display update, rather than relying on the current gaze position. By analyzing eye movement parameters and predicting future gaze location, the system ensures that high-resolution rendering is directed to the correct area when the image is actually displayed, maintaining both rendering accuracy and gaze position accuracy.
Solution Approach 2:
The system uses continuous feedback from eye tracking devices to monitor and analyze eye movement parameters. This feedback loop allows the system to detect changes in gaze direction and adjust the predicted point of regard accordingly, ensuring that the high-resolution rendering area remains aligned with the user's actual gaze position despite the time delay between rendering and display update.
3Manufacturing precision
If resolution is increased around the point of regard, then image quality is improved, but computational load increases
Solution Approach 1:
The system applies local quality by rendering only the first area around the predicted point of regard with high resolution, while rendering the rest of the display area with lower resolution. This selective rendering approach ensures that computational resources are concentrated on the area that will actually be perceived by the user, significantly reducing overall computational load while maintaining high image quality where it matters most.
Solution Approach 2:
By predicting the point of regard in advance, the system can prepare the high-resolution rendering area beforehand, allowing computational tasks to be distributed over time. This preliminary prediction enables the system to perform intensive rendering calculations during periods when the user's gaze is stable, rather than rushing to render the entire high-resolution image at the last moment.
Data Source
AI summary
Method for subsequently displaying a stream of images in a display area of a display device, wherein a first area within the display area is determined around a point of regard. A first image of the stream of images is displayed in the display area such that a first part of the first image, which is displayed in the first area, is displayed according to a first parameter value of at least one parameter and a second part of the first image, which is displayed in at least one second area outside the first area, is displayed according to a second parameter value of the at least one parameter. Moreover, the determining of the point of regard is performed as predicting the point of regard for a certain future point of time, at which the first image is displayed.