Scalable Video Coding for Virtual Reality View Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Streaming virtual reality content is challenging due to the significant amount of visual information required to simulate a user's presence in a 3D environment, often resulting in data disruptions and insufficient bandwidth or processing resources, leading to jarring visual experiences when switching between different streams or perspectives.
Innovation Solution
The use of a scalable video encoding standard like SHVC, which encodes virtual reality content into base layers and enhancement layers, allowing for improved frame rate, resolution, and quality by selectively decoding and downloading only the necessary layers based on the user's perspective, reducing data processing and bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If virtual reality content is streamed with high visual information requirements to simulate user presence in 3D environment, then the quality and immersion of virtual reality experience is improved, but the bandwidth requirements and data processing loads increase significantly causing disruptions and insufficient resources
Solution Approach 1:
The video content is segmented into multiple layers with different quality levels (base layer and enhancement layers). Each layer can be independently decoded and displayed, allowing the system to provide acceptable quality even when only lower-bitrate layers are transmitted. This segmentation enables adaptive streaming where layers are selectively transmitted based on available bandwidth, resolving the contradiction between maintaining high quality and reducing bandwidth requirements.
2Adaptability or versatility
If multiple video streams are prepared for different perspectives and resolutions to enable smooth switching, then the adaptability and user experience are improved, but the device complexity and processing requirements increase
Solution Approach 1:
The system dynamically selects which video layers to decode and display based on current user perspective, device capabilities, and network conditions. Instead of statically preparing and switching between complete separate video streams, the scalable video codec allows dynamic adaptation by selectively processing only the necessary enhancement layers needed for the current viewing scenario, reducing processing complexity while maintaining adaptability.
3Manufacturing precision
If enhancement layers are decoded to improve frame rate, resolution, and quality, then the visual quality is improved, but the data processing time and computational resources increase
Solution Approach 1:
The video content is pre-encoded into a scalable structure with base layers and enhancement layers during the encoding phase. This preliminary organization allows the decoder to quickly assemble high-quality output by simply combining pre-prepared layers rather than performing complex real-time processing, significantly reducing decoding time while maintaining the ability to deliver high visual quality when computational resources and time are available.
Data Source
AI summary
Techniques to enable virtual reality content to be delivered using a video codec that operates according to a scalable video encoding standard. Base layer frames for the different views of the virtual reality content are downloaded by a client device. The views are prioritized using a prediction model that assigns priorities based on the likelihood that a corresponding view will be selected within a particular period of time. Enhancement layer frames are then selected and downloaded based on the priorities.