Dynamic Foveated Pipeline for Simulated Reality Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rendering images for simulated reality (SR) experiences is computationally expensive, particularly due to the need for high resolution in focused areas and varying peripheral vision capabilities of humans, leading to increased processing power and bandwidth requirements.
Innovation Solution
Implementing a method that renders images with varying resolutions based on a rendering resolution function, where areas of focus receive higher resolution and peripheral areas receive lower resolution, using a warped image representation with scaling factors to optimize processing and bandwidth, and employing foveation techniques to reduce computational burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high resolution is used for focused areas in simulated reality rendering, then image quality in areas of interest is improved, but computational cost and processing power requirements increase
Solution Approach 1:
The patent applies local quality by rendering different portions of the simulated reality scene at different resolutions. Specifically, areas corresponding to the user's foveal vision (central gaze area) are rendered at high resolution, while peripheral areas are rendered at lower resolution. This selective quality approach maintains image quality where the user actually looks while reducing overall computational cost.
Solution Approach 2:
The patent implements dynamic resolution adjustment based on real-time eye tracking data. The rendering system continuously adapts which areas receive high resolution treatment as the user's gaze moves. This dynamic approach ensures computational resources are always allocated to the currently relevant areas rather than statically allocating resources to fixed regions.
2Measurement precision
If high resolution is used throughout the entire simulated reality image, then overall image quality is improved, but bandwidth requirements for transmission increase
Solution Approach 1:
Instead of transmitting the entire high-resolution image, the system transmits only the high-resolution portion corresponding to the user's foveal area at full resolution while peripheral areas are transmitted at lower resolution. This significantly reduces the total data quantity that needs to be transmitted while maintaining perceptual quality.
3Productivity
If varying resolution rendering is implemented, then computational load is reduced, but system complexity increases due to additional processing steps
Solution Approach 1:
The system performs preliminary actions by pre-processing the scene to identify areas of interest based on eye tracking data before the actual rendering occurs. This preparation step allows the main rendering process to focus computational resources efficiently on only the necessary high-resolution areas, reducing overall complexity through better organization of operations.
Data Source
AI summary
In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.