Axis-Based Compression for VR Foveated Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtual reality systems, latency between user movement and image rendering can cause judder, leading to motion sickness and degraded user experience due to the computational exhaustive process of generating and transmitting high-resolution images within the frame time.
Innovation Solution
The system employs axis-based compression and decompression techniques, where images are divided into areas or blocks along geometric or curved axes, with higher compression levels applied to peripheral areas, reducing communication bandwidth while preserving the fidelity of the foveated area, allowing for efficient image rendering and transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution images are generated and transmitted within frame time, then image quality is maintained, but latency increases and user experience degrades
Solution Approach 1:
The image is divided into multiple regions of interest (ROIs) based on depth information, with different compression levels applied to each region. Foreground objects are segmented from background, allowing selective quality preservation where needed while reducing overall data transmission requirements.
Solution Approach 2:
Different compression levels are applied to different spatial regions of the image based on their importance. Foreground regions maintain higher quality while background regions use lower quality, optimizing the balance between perceived image quality and transmission latency.
2Measurement precision
If computational exhaustive processes are used for image rendering, then image fidelity is preserved, but processing time exceeds frame time
Solution Approach 1:
The rendering process is segmented into depth-based regions, allowing selective application of computational resources. Only foreground regions requiring high fidelity receive exhaustive processing, while background regions use simplified rendering paths.
Solution Approach 2:
Instead of applying full computational processing to the entire image, the system applies partial processing only to critical foreground regions, achieving acceptable overall image quality with reduced processing time that fits within frame time constraints.
3Measurement precision
If full bandwidth communication is used for image transmission, then image quality is maintained, but communication bandwidth consumption increases
Solution Approach 1:
The transmitted image data is segmented into depth-based regions with different compression levels. Foreground regions are transmitted with higher fidelity while background regions use aggressive compression, reducing total bandwidth consumption while preserving perceptual quality.
Solution Approach 2:
The system transmits high-quality data only for important foreground regions and lower-quality data for background regions, optimizing bandwidth utilization by matching transmission quality to regional importance rather than uniformly transmitting full quality across the entire image.
Data Source
AI summary
Disclosed herein are related to a system and a method of remotely rendering an image. In one approach, a console device generates an image according to a gaze direction of a user of a head mounted display (HMD). In one aspect, the image includes a first area and a second area disposed along an axis, where the second area is located farther away from a foveated area of the image than the first area. In one aspect, the foveated area corresponds to the gaze direction of the user of the HMD. In one aspect, the console device compresses the image according to the axis, where the second area is compressed at a higher level than the first area. In one aspect, the compressed image is transmitted to the HMD. The HMD may decompress the compressed image according to the axis, and render the decompressed image.