Video Stream Splitting for VR Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for video data transmission in virtual reality applications, such as near-eye displays, face bandwidth and computing power limitations, leading to noticeable lag and visual artifacts when changing viewing angles, as they require large amounts of data to maintain seamless high-resolution images across all viewing angles.
Innovation Solution
The technique involves partitioning video images into regional cross sections based on user viewport, foveal vision, and scenes of interest, encoding these sections into multiple video sub-streams at varying spatiotemporal resolutions, allowing for efficient data transmission and seamless high-resolution rendering without lag.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video data is transmitted at high resolution for all viewing angles, then visual quality is improved, but bandwidth requirements increase enormously
Solution Approach 1:
The patent segments the video content into multiple pyramid faces, where only the base pyramid face (corresponding to the current FOV) is transmitted at high resolution, while other faces are transmitted at lower resolutions or not at all. This segmentation allows the system to provide high visual quality for the active viewing area while significantly reducing overall bandwidth requirements.
Solution Approach 2:
The patent applies local quality by transmitting high-resolution video data only for the base pyramid face that corresponds to the user's current field of view, while other pyramid faces receive lower resolution data. This ensures that computational and bandwidth resources are concentrated on the visually critical regions, maintaining high visual quality where needed while reducing overall data transmission requirements.
2Productivity
If pyramid partitioning is used to reduce bandwidth, then data transmission efficiency is improved, but lag increases when changing FOV to non-base faces
Solution Approach 1:
The patent applies preliminary action by pre-computing and preparing video data for multiple pyramid faces, including potential future FOV directions. When the user changes their field of view, the system can quickly switch to the pre-prepared base pyramid face data, minimizing lag. This preliminary preparation ensures that high-resolution data is ready for rapid transmission when needed.
Solution Approach 2:
The patent implements dynamics by making the base pyramid face selection adaptive and dynamic based on the user's current and predicted field of view. The system continuously monitors FOV changes and dynamically adjusts which pyramid face becomes the base, ensuring that high-resolution data is always available for the active viewing area while maintaining efficient bandwidth utilization.
3Productivity
If pyramid partitioning is used to reduce bandwidth, then transmission efficiency is improved, but FOV may intersect two pyramid faces causing resolution loss
Solution Approach 1:
The patent applies partial or excessive action by transmitting slightly more data than strictly necessary for the current FOV. When the FOV intersects multiple pyramid faces, the system transmits high-resolution data for the entire base pyramid face area, which may extend beyond the strict boundaries of the current FOV. This excessive transmission ensures complete coverage and maintains resolution continuity, preventing the visual artifacts that would result from precise but incomplete face boundary adherence.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Scenes in video images are identified based on image content of the video images. Regional cross sections of the video images are determined based on the scenes in the video images. Image portions of the video images in the regional cross sections are encoded into multiple video sub-streams at multiple different spatiotemporal resolutions. An overall video stream that includes the multiple video sub-streams is transmitted to a streaming client device.