Gaze-Based Frame Refoveation for Lower-Energy Post-Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Devices with displays face resource-intensive post-rendering operations that drain the battery, necessitating a reduction in resource consumption.
Innovation Solution
Devices refoveate rendered frames based on gaze data to reduce the size of the foveation plateau, resampling and performing post-rendering operations on lower resolution frames to conserve resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If post-rendering operations are performed on high-resolution rendered frames, then visual quality is maintained, but resource consumption increases and battery life decreases
Solution Approach 1:
The patent applies local quality by performing post-rendering operations selectively only on the foveated portion of the display where the user is looking, rather than on the entire high-resolution frame. This maintains visual quality in the critical foveal region while reducing resource consumption in peripheral regions, thereby extending battery life.
Solution Approach 2:
The patent segments the display into foveated and non-foveated portions based on eye tracking data. Post-rendering operations are applied only to the segmented foveated region, separating the processing requirements to optimize both visual quality and energy efficiency.
2Measurement precision
If the foveation plateau size is maintained large, then visual quality in peripheral regions is improved, but resource consumption for post-rendering operations increases
Solution Approach 1:
The patent dynamically adjusts the foveation plateau size based on real-time eye tracking data. The size and position of the foveation plateau change as the user's gaze moves, allowing the system to optimize peripheral visual quality when needed while maintaining high processing efficiency during normal operation.
Solution Approach 2:
The patent changes the parameters of the foveation plateau (size, position, shape) based on eye tracking measurements. By adjusting these parameters dynamically, the system balances peripheral visual quality with processing efficiency, applying post-rendering operations only when and where necessary.
Data Source
AI summary
A method includes obtaining a first frame that is associated with a first foveation parameter. The first frame is rendered based on first eye tracking data obtained at a first time. The method includes capturing, via the eye tracker, second eye tracking data at a second time that occurs after the first time. The method includes refoveating the first frame based on the second eye tracking data in order to generate a second frame with a second foveation parameter that is different from the first foveation parameter. The method includes performing a post-rendering operation on the second frame.