Foveated Rendering Border Tile Blending for Artifact-Free VR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Foveated rendering in AR/VR headsets using mobile GPUs results in abrupt transitions between high- and low-pixel-density tiles, causing visual artifacts like 'tunnels' or 'windows', which degrade the user experience due to the larger tile sizes of mobile GPUs compared to traditional GPUs.
Innovation Solution
Implement a blending process for border tiles by rendering them in both high- and low-pixel-density and combining these renderings to create a gradual transition, or use mipmap texture levels to downgrade pixel densities based on object distance, ensuring a smooth transition between tile densities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If foveated rendering is implemented with large tile sizes to reduce processing workload, then power consumption and processing requirements are reduced, but abrupt transitions between high- and low-pixel-density tiles cause visual artifacts
Solution Approach 1:
The rendering process is divided into multiple passes: a first pass renders all tiles at low pixel density, while a second pass renders only border tiles at high pixel density. This segmentation allows the system to reduce overall processing workload and power consumption while still providing high quality rendering where needed, avoiding the visual artifacts that would result from uniformly rendering all tiles at high density.
Solution Approach 2:
Different pixel densities are applied to different regions of the display based on the user's gaze location. Border tiles surrounding the high-pixel-density foveal region are rendered at high pixel density, while peripheral tiles are rendered at low pixel density. This local quality approach ensures that visual artifacts are eliminated at the critical transition zones while maintaining power efficiency in the periphery.
2Object-affected harmful factors
If border tiles are rendered in both high- and low-pixel-density and blended, then visual artifacts are reduced, but rendering time and processing complexity increase
Solution Approach 1:
The system performs a preliminary rendering pass of all tiles at low pixel density before performing a second targeted pass on only the border tiles at high pixel density. This preliminary action ensures that the base image is already rendered, and only the necessary border regions require additional processing, thereby minimizing the overall rendering time increase while still achieving artifact reduction through blending.
Solution Approach 2:
Instead of rendering all tiles at high pixel density (excessive action), the system renders only the necessary border tiles at high pixel density (partial action). The blending operation combines the high-density border tile rendering with the low-density peripheral rendering, achieving sufficient visual quality without the excessive processing time that would result from uniform high-density rendering.
3Weight of moving object
If mobile GPU is used instead of traditional GPU, then device weight and power consumption are reduced, but tile rendering capabilities differ causing abrupt density transitions
Solution Approach 1:
The system changes the rendering parameters specifically for border tiles by rendering them in a second pass at high pixel density, then blending these results with the low pixel density peripheral tiles. This parameter change approach compensates for the mobile GPU's tile-based rendering characteristics, ensuring rendering consistency across tile boundaries without requiring a traditional GPU.
Solution Approach 2:
The blending operation acts as an intermediary process that combines the high-pixel-density border tile rendering with the low-pixel-density peripheral rendering. This intermediary blending step smooths the transition between regions rendered by the mobile GPU's tile-based architecture, ensuring visual consistency despite the hardware limitations.
Data Source
AI summary
In one embodiment, a method includes by a computing system, rendering an image using a tile-based graphics processing unit. Determining a gaze location of a user wearing a head-mounted device and using the gaze location to select, from the multiple tiles, central tiles in which the user's gaze location is located, periphery tiles outside of the central tiles, and border tiles located between the central tiles and the periphery tiles. Instructing the GPU to render (a) the central tiles in a first pixel-density, (b) the periphery tiles in a second pixel-density, and (c) the border tiles in the first pixel-density and in the second pixel-density and then blending the border tiles rendered in the first pixel-density and the border tiles rendered in the second pixel-density to create blended border tiles. Then, outputting the central tiles, the periphery tiles, and the blended border tiles using a display of the head-mounted device.