Foveated Anti-Aliasing With Peripheral Frame Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Foveated imaging causes flickering in the periphery due to temporal aliasing, which degrades the user experience, particularly noticeable despite humans' relatively weak peripheral vision.
Innovation Solution
Generate an output image by blending at least a portion of the periphery of the currently rendered image with an accumulation image based on previously rendered images, using a weighted sum of the currently rendered image pixel value and the accumulation image pixel value, with the weighting factor determined by the scaling factor at the pixel location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If foveated imaging is used to reduce computational complexity and energy consumption, then rendering performance is improved, but temporal aliasing causes flickering in the periphery
Solution Approach 1:
The patent applies preliminary action by accumulating frame buffers from previous low-resolution renders before the current high-resolution render completes. The accumulation image is prepared in advance and then blended with the current render, allowing the system to maintain anti-aliasing benefits without requiring every single frame to be rendered at full resolution, thus reducing energy consumption while preventing flickering.
Solution Approach 2:
The patent applies local quality by differentiating the rendering resolution based on spatial location. High resolution is applied locally to the fovea region where users focus their attention, while low resolution is applied to the periphery regions. The accumulation blending is selectively applied only to peripheral regions where temporal aliasing occurs, preserving energy savings in the periphery while maintaining quality where needed.
2Productivity
If foveated imaging is used to reduce computational complexity, then rendering speed is improved, but temporal aliasing causes flickering in the periphery
Solution Approach 1:
The accumulation image is prepared in advance from previous frames and stored in a frame buffer. When generating the current output image, the system blends the pre-prepared accumulation image with the current render, eliminating the need to wait for previous frames to complete fully, thus maintaining high rendering speed while reducing flickering through the anti-aliasing blend operation.
Solution Approach 2:
The patent maintains continuity of useful action by continuously accumulating frame buffers from all previous renders. This continuous accumulation process ensures that the anti-aliasing blend always has access to historical data, smoothing out temporal variations and eliminating flickering while the system maintains high rendering throughput through the optimized foveated rendering pipeline.
3Measurement precision
If high resolution is applied to the entire image, then image quality is improved, but computational complexity and energy consumption increase
Solution Approach 1:
The patent implements local quality by applying high resolution only to the fovea region and low resolution to the periphery region. The accumulation blending is applied selectively to peripheral regions where temporal aliasing occurs, while the fovea region maintains high resolution directly. This spatially adaptive approach improves overall image quality in critical regions while reducing computational complexity in less critical regions.
Solution Approach 2:
The patent segments the image into different regions based on resolution requirements. The fovea region is separated from the periphery region, with each region processed at different resolution levels. The accumulation frame buffer is also segmented, with blending applied only to peripheral regions. This segmentation allows the system to optimize computational resources by applying high computational complexity only where it is most needed for perceived image quality.
Data Source
AI summary
In one implementation, a method of generating an output image is performed by a device including a display, one or more processors, and non-transitory memory. The method includes obtaining a currently rendered image. The method includes obtaining an accumulation image based on previously rendered images. The method includes generating an output image based on the currently rendered image and the accumulation image, wherein the output image includes: (1) a first output pixel at a first pixel location associated with a first resolution greater than or equal to a threshold, the first output pixel having a first output pixel value that is a currently rendered image pixel value of a currently rendered image pixel at the first pixel location; (2) a second output pixel at a second pixel location associated with a second resolution less than the threshold, the second output pixel having a second output pixel value that is a weighted sum of a currently rendered image pixel value of a currently rendered image pixel at the second pixel location and an accumulation image pixel value of an accumulation image pixel at a corresponding pixel location corresponding to the second pixel location; and (3) a third output pixel at a third pixel location associated with a third resolution less than the threshold, the third output pixel having a third output pixel value that is a currently rendered image pixel value of a currently rendered image pixel at the third pixel location. The method includes displaying, on the display, the output image.


