Foveated Video Link for VR with Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression methods for wide field of view displays, such as HMDs for VR, are inefficient in terms of bandwidth and computational load, as they apply a single compression algorithm across the entire screen, failing to account for varying visual acuity and resulting in increased motion sickness risk.
Innovation Solution
Implementing foveated video technology that uses eye gaze tracking to determine the region of interest, allowing high resolution only in the foveal region and reducing resolution outside, thereby reducing bandwidth and computational requirements while minimizing motion sickness through dynamic adjustment of the region of interest.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single compression algorithm is applied across the entire screen, then the device complexity is reduced, but the bandwidth efficiency deteriorates
Solution Approach 1:
The patent applies different compression qualities to different regions of the video frame based on the foveal region identification. The central foveal region receives high-quality compression while peripheral regions receive lower-quality compression, optimizing bandwidth usage according to human visual perception characteristics.
Solution Approach 2:
The video frame is divided into multiple regions with different compression qualities. The system segments the display area into a foveal region (requiring high quality) and peripheral regions (tolerating lower quality), allowing differential compression strategies to be applied to each segment.
2Manufacturing precision
If high resolution video is displayed across the entire wide FOV, then the visual quality is improved, but the bandwidth requirement increases
Solution Approach 1:
High resolution video is rendered only in the foveal region where the user's visual acuity is highest, while peripheral regions are rendered at lower resolution. This approach maintains perceived visual quality while significantly reducing the total bandwidth requirement for wide FOV displays.
Solution Approach 2:
The system dynamically changes the resolution parameter across different spatial regions of the video frame. The foveal region maintains high resolution parameters while peripheral regions use lower resolution parameters, adapting the quality to the local visual importance.
3Manufacturing precision
If the foveal region size is increased, then the detail preservation is improved, but the bandwidth consumption increases
Solution Approach 1:
The foveal region size and position are dynamically adjusted based on real-time eye tracking data. The system continuously adapts the high-resolution region to follow the user's gaze, ensuring that detail preservation is applied only where the user is currently looking, thereby optimizing bandwidth consumption.
Solution Approach 2:
The system uses eye tracking feedback to determine the appropriate foveal region boundaries and adjust the differential compression parameters accordingly. This closed-loop feedback mechanism ensures that high detail preservation is applied precisely to the region of visual interest while minimizing overall bandwidth consumption.
Data Source
Figure 1A
Figure 1B
Figure 2A~2B
AI summary
Gaze tracking data is analyzed to determine one or more regions of interest within an image of a video stream. The video stream data is selectively scaled so that sections within the regions of interest maintain high resolution while areas not within the region of interest are down-scaled to reduce bandwidth cost of transmission. A scheme for reduction of motion sickness by reducing the size of the high resolution area is also claimed.