Dynamic Tiling for Foveated Rendering in AR VR Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR), mixed reality (MR), and virtual reality (VR) systems face challenges in efficiently rendering high-quality images while minimizing processing power and power consumption, especially when dealing with foveated rendering and eye tracking.
Innovation Solution
The system employs a dynamic tiling approach, where tiles are defined and resized based on the user's gaze direction and error, with higher resolution tiles centered on the gaze point and lower resolution tiles in the periphery, optimizing image quality and processing demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If foveated rendering is implemented with static tile positioning, then processing power is reduced by rendering only high-resolution central region, but image quality deteriorates when gaze shifts occur between tile updates
Solution Approach 1:
The patent implements dynamic tile positioning where the high-resolution tile continuously adjusts its position and size based on real-time gaze tracking data. When the user's gaze moves, the system recalculates and repositions the high-resolution tile to center on the new gaze point, ensuring the foveal region always receives high-quality rendering while maintaining processing efficiency.
Solution Approach 2:
The system uses eye tracking feedback to continuously monitor gaze position and dynamically adjust tile positioning accordingly. The gaze data feeds into the rendering pipeline, triggering tile repositioning when gaze movement is detected, creating a closed-loop system that maintains image quality adaptivity.
2Manufacturing precision
If high-resolution rendering is applied to the entire display, then image quality is maintained across all regions, but processing power consumption increases significantly
Solution Approach 1:
The patent applies different rendering qualities to different spatial regions of the display based on gaze position. The high-resolution tile is positioned at the gaze point to provide detailed rendering where the user is looking, while peripheral regions use lower-resolution rendering, optimizing the balance between perceived image quality and processing power consumption.
Solution Approach 2:
The display is segmented into multiple resolution zones: a high-resolution tile positioned at the gaze point and lower-resolution peripheral regions. This segmentation allows the system to allocate processing resources selectively, rendering high detail only where necessary while reducing computational load in less critical areas.
3Productivity
If tile size is increased to cover larger display areas, then processing power is reduced by fewer tiles to render, but image quality in peripheral regions deteriorates
Solution Approach 1:
The system dynamically adjusts tile size and position based on gaze position and display geometry. The high-resolution tile size is optimized to cover the foveal region appropriately, while peripheral regions use adaptive lower-resolution rendering. This dynamic adjustment ensures processing efficiency without sacrificing necessary image quality in different display regions.
Data Source
Figure 1
Figure 2
Figure 3~7
AI summary
A method for providing imagery to a user on a display includes receiving eye tracking data and determining a gaze location on the display or a gaze vector using the eye tracking data. The method can also include defining a first tile using the gaze location on the display or the gaze vector. The first tile may have a height and a width, the height and width being determined using the eye tracking data. The method can further include defining multiple additional tiles to fill an entire area of the display. The method can also include providing a portion of an image using the first tile at a first image quality and providing another portion of the image at a second image quality using at least one of the multiple additional tiles.