Distributed Foveated Rendering for Wearable Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing foveated rendering techniques for computing devices, such as virtual reality headsets, are inefficient in distributing processing and memory usage between local and remote computers, leading to increased battery consumption and latency in wearable devices.
Innovation Solution
A distributed foveated rendering method where the end user device sends gaze and orientation data to a remote computer, which generates low-resolution background and high-resolution foreground images, allowing the end user device to composite and process these images, reducing redundant data transfer and processing load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If foveated rendering is implemented on the end user device, then image quality in the foveated region is improved, but processing load and battery consumption increase
Solution Approach 1:
The display area is segmented into a foveated region (where the user is gazing) and a peripheral region. The system renders the foveated region at high resolution and the peripheral region at low resolution, thereby improving image quality in the important area while reducing overall processing load and battery consumption.
Solution Approach 2:
Different rendering qualities are applied to different regions of the display. The foveated region receives high-resolution rendering to ensure image quality, while the peripheral region uses low-resolution rendering to reduce computational burden, achieving local optimization of both quality and energy consumption.
2Device complexity
If all rendering is performed on the remote computer, then processing load on the end user device is reduced, but bandwidth usage and latency increase
Solution Approach 1:
The system extracts only the essential high-resolution foveated region data from the remote computer and transmits it to the end user device. The peripheral region data is either not transmitted or transmitted at lower resolution, reducing bandwidth usage and latency while still providing sufficient visual information when combined with locally rendered content.
Solution Approach 2:
Instead of transmitting complete high-resolution data for the entire display area, the system transmits only partial data corresponding to the foveated region at high resolution. This partial action approach reduces data transmission requirements and latency while maintaining perceptual quality through local compositing.
3Manufacturing precision
If high resolution is maintained across the entire display, then image quality is improved, but processing load and bandwidth usage increase
Solution Approach 1:
The display is segmented into foveated and peripheral regions with different resolution requirements. By rendering only the foveated region at high resolution and the peripheral region at low resolution, the system maintains image quality where it matters most while significantly improving processing efficiency and reducing bandwidth usage.
Solution Approach 2:
The system dynamically changes the resolution parameter based on the display region. The foveated region uses high resolution parameters to ensure image quality, while the peripheral region uses low resolution parameters to improve processing efficiency. This parameter adaptation resolves the contradiction between quality and productivity.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Techniques for distributed foveated rendering based on user gaze are described. In an example, an end user device is communicatively coupled with a remote computer and presents images on a display based on gaze data. The user device receives a low resolution background image and high resolution foreground image from the remote computer based on the gaze data. The foreground image is constrained to a foveated region according to the gaze data. The end user device generates a composite image by scaling up the background image and overlaying the foreground image. The composite image is then presented on the display.