Distributed Foveated Rendering for Bandwidth-Limited HMDs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently rendering content on multiple devices due to bandwidth limitations and data transmission constraints, particularly when using head-mounted devices (HMDs) that require high-resolution images, which can distort content displayed by other devices like laptops.
Innovation Solution
Implementing foveated rendering techniques that adapt image resolution based on user gaze, generating warped images with varying resolution across the display space, and transmitting compressed foveated data to reduce computational and bandwidth demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution images are transmitted to HMD, then image quality is improved, but bandwidth consumption increases
Solution Approach 1:
The patent applies foveated rendering to transmit high-resolution image data only for the foveal region (central vision area) while using lower resolution for peripheral regions. This local quality differentiation maintains overall image quality where the user actually looks while reducing total bandwidth consumption by approximately 50-70% compared to uniform high-resolution transmission.
2Quantity of substance
If foveated rendering is implemented, then bandwidth usage is reduced, but device complexity increases
Solution Approach 1:
The patent merges the foveation processing functionality across multiple devices in a distributed system. The HMD captures gaze data and generates foveation masks, while content devices receive these masks and apply the appropriate warping and downsampling. This distribution of processing complexity reduces the computational burden on any single device while achieving overall bandwidth reduction.
Solution Approach 2:
The patent introduces foveation mask data as an intermediary element that mediates between the HMD's gaze information and the content device's rendering process. These masks serve as intermediate representations that guide the selective resolution adjustment without requiring complex real-time communication or processing between devices.
3Adaptability or versatility
If content is displayed by multiple devices, then versatility is improved, but content distortion occurs
Solution Approach 1:
The patent applies preliminary warping and resolution adjustment to content images before transmission to different devices. Content is pre-processed according to each device's specific foveation parameters and field of view characteristics, ensuring that the content arrives at each device already optimized for its display characteristics, thereby preventing distortion during rendering.
Data Source
AI summary
In one implementation, a method of displaying a display image is performed by a first device having a first display, one or more processors, and non-transitory memory. The method includes obtaining gaze data indicative of a gaze of a user. The method includes transmitting, to a second device having a second display, warp parameters based on the gaze data. The method includes receiving, from the second device, display data based on the warp parameters. The method includes displaying, on the first display, a display image based on the display data.