Foveated Rendering via Gaze-Dependent Sampling Patterns
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image processing technologies face challenges in efficiently reducing computational complexity and power consumption while maintaining visual quality, particularly in rendering graphics, as they often require uniform resolution across the entire screen, which is not aligned with human visual acuity patterns.
Innovation Solution
The method involves tracking a user's gaze point and determining sampling patterns for primitives based on their distance and direction from the gaze point, allowing for varying resolutions that prioritize higher resolution near the gaze point and lower resolution further away, effectively implementing foveated rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If uniform resolution is applied across the entire screen, then visual quality is maintained, but computational complexity and power consumption increase
Solution Approach 1:
The patent applies different rendering resolutions to different regions of the display based on the user's gaze point. The foveal region (center of gaze) is rendered at high resolution while peripheral regions are rendered at progressively lower resolutions. This local differentiation maintains visual quality in the important central region while reducing computational complexity in less critical peripheral regions.
Solution Approach 2:
The patent divides the display area into multiple regions based on distance from the gaze point, with each region assigned a different rendering resolution. This segmentation allows the system to apply appropriate computational resources to each region, maintaining high visual quality where needed while reducing overall computational complexity.
2Measurement precision
If uniform resolution is applied across the entire screen, then visual quality is maintained, but power consumption increases
Solution Approach 1:
The patent renders only the foveal region at high resolution while using lower resolutions for peripheral regions. Since the human eye has highest acuity at the center of gaze, this local quality approach maintains perceived visual quality while significantly reducing the total number of pixels that require high-resolution rendering, thereby reducing power consumption.
Solution Approach 2:
The patent applies high-resolution rendering only partially - specifically to the foveal region where it is most needed - rather than applying it excessively across the entire display. This partial action maintains visual quality in critical areas while avoiding the excessive power consumption that would result from uniform high-resolution rendering.
3Measurement precision
If higher resolution is applied near the gaze point, then visual quality is improved, but device complexity increases
Solution Approach 1:
The patent dynamically adjusts rendering resolution based on the user's gaze point position. As the gaze point moves, the high-resolution rendering region dynamically follows it, with peripheral regions automatically transitioning to lower resolutions. This dynamic adaptation maintains visual quality where needed while managing rendering complexity through automated region classification.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of processing an image includes tracking (S810) a gaze point of a user on a screen of a display device; determining (S820) sampling patterns for primitives forming a frame, respectively, based on a distances between the gaze point and the primitives; and rendering (S830) the primitives according to the sampling patterns determined for the primitives, respectively.