Scalable Video Coding for Virtual Reality View Prioritization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Streaming virtual reality content is challenging due to the significant amount of visual information required to simulate a user's presence in a 3D environment, often resulting in data disruptions and insufficient bandwidth or processing resources, leading to jarring visual experiences when switching between different streams or perspectives.

Innovation Solution

The use of a scalable video encoding standard like SHVC, which encodes virtual reality content into base layers and enhancement layers, allowing for improved frame rate, resolution, and quality by selectively decoding and downloading only the necessary layers based on the user's position and perspective, minimizing data processing and bandwidth requirements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If virtual reality content is streamed with high resolution and frame rate to simulate user presence in 3D environment, then visual quality is improved, but bandwidth requirements and data processing load increase significantly

Engineering Contradiction:

Improvevisual qualityVSAvoidbandwidth requirements

Core Design Contradiction:

Manufacturing precisionVSQuantity of substance

Solution Approach 1:

The video content is segmented into multiple layers with different quality levels (base layer and enhancement layers). The base layer provides essential visual information at lower resolution, while enhancement layers provide additional details for higher quality. This segmentation allows progressive transmission where essential content is delivered first, and additional quality is added as bandwidth permits.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically adjusts which layers are transmitted and decoded based on real-time bandwidth availability, user device capabilities, and network conditions. This dynamic adaptation allows the system to maintain optimal visual quality within available bandwidth constraints, switching between different layer combinations as conditions change.

Inventive Principle:

Principle #15Dynamics

2Adaptability or versatility

If all possible views are pre-loaded to accommodate user movement in virtual environment, then adaptability to user interaction is improved, but data transmission volume and processing time increase

Engineering Contradiction:

Improveadaptability to user interactionVSAvoidprocessing time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The base layer is pre-loaded and made immediately available, providing a complete but lower-quality view of all possible perspectives. Enhancement layers are prepared but not fully transmitted until needed. This preliminary action ensures that users can immediately access any view at acceptable quality, with the option to enhance specific views as they are actually viewed.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of uniformly enhancing all views, the system applies enhancement layers selectively to only those specific views that the user is currently looking at or is predicted to look at next. This local quality enhancement minimizes data transmission by focusing computational and bandwidth resources only on the locally relevant portions of the virtual environment.

Inventive Principle:

Principle #3Local quality

3Adaptability or versatility

If multiple video streams are maintained for different perspectives and resolutions, then adaptability to user selection is improved, but device complexity and switching disruptions increase

Engineering Contradiction:

Improveadaptability to user selectionVSAvoiddevice complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The scalable video coding structure provides a universal framework where a single encoded bitstream contains multiple quality and resolution variations through different layer combinations. Instead of maintaining separate complete video streams for each perspective and resolution, the system uses one multi-layer encoded stream that can be decoded at multiple quality levels, reducing device complexity while maintaining adaptability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Manufacturing precision

If enhancement layers are decoded to improve frame rate and resolution, then visual quality is improved, but processing resources and energy consumption increase

Engineering Contradiction:

Improvevisual qualityVSAvoidprocessing resources

Core Design Contradiction:

Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The system decodes only the necessary portion of enhancement layers required to achieve acceptable visual quality for the current viewing conditions. Instead of fully decoding all enhancement layers for maximum quality, the system selectively applies partial enhancement based on bandwidth availability and device capabilities, reducing processing resource consumption while maintaining adequate visual quality.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11228774B1Scalable video coding techniques

Publication Date: 2022.01.18 AMAZON TECH INC

AI summary

Techniques to enable virtual reality content to be delivered using a video codec that operates according to a scalable video encoding standard. Base layer frames for the different views of the virtual reality content are downloaded by a client device. The views are prioritized using a prediction model that assigns priorities based on the likelihood that a corresponding view will be selected within a particular period of time. Enhancement layer frames are then selected and downloaded based on the priorities.