Layered Video Stream for VR Bandwidth Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Virtual reality systems face limitations in providing immersive experiences due to bandwidth, storage, and processing constraints, leading to reduced quality and freedom of motion when capturing and displaying real-world content with traditional methods.

Innovation Solution

The video stream is divided into layers, allowing progressive retrieval and processing, with vantages and tiles assigned to layers based on importance and quality, enabling enhanced quality and motion freedom by incorporating higher layers, and using tiled light-field cameras to capture continuous light-field data for accurate virtual views.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional video streaming methods are used for virtual reality content, then bandwidth and storage requirements are high, but quality and motion freedom are reduced

Engineering Contradiction:

Improvequality of virtual reality experienceVSAvoidbandwidth and storage requirements

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The video stream is divided into multiple layers, where each layer contains vantages and tiles at different quality levels. This segmentation allows the system to transmit only the necessary layers based on client capabilities, reducing bandwidth requirements while maintaining quality for capable devices.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different regions of the video content (vantages and tiles) are assigned to different layers based on their importance and quality requirements. Critical regions receive higher quality treatment in lower layers, while less critical regions can be enhanced in higher layers, optimizing bandwidth usage.

Inventive Principle:

Principle #3Local quality

2Adaptability or versatility

If high quality virtual reality content is provided to all users, then user experience is improved, but devices with limited capabilities cannot benefit

Engineering Contradiction:

Improvecompatibility across different device capabilitiesVSAvoidquality of virtual reality experience

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system dynamically adapts the delivered content quality based on client device capabilities. Clients with higher processing power and bandwidth receive more layers and higher quality vantages, while limited devices receive only essential lower layers, ensuring optimal performance across all device types.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The layered video stream structure serves multiple functions: it provides baseline quality for all devices, enables progressive enhancement for capable devices, and allows flexible adaptation to different network conditions. This universal structure benefits both constrained and unconstrained clients.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Reliability

If complete video streams with all vantages and tiles are transmitted, then viewing quality is maximized, but bandwidth consumption increases

Engineering Contradiction:

Improveviewing qualityVSAvoidbandwidth consumption

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The system transmits only the necessary portion of the complete video stream based on client needs. Essential vantages and tiles are included in lower layers, while additional higher-quality vantages and tiles are provided in higher layers only when bandwidth allows, avoiding transmission of redundant data.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The video stream is pre-organized into layers with vantages and tiles prioritized by importance before transmission. This preliminary structuring enables the client to quickly assemble a usable video stream from the most important layers without waiting for or requiring transmission of all possible content.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10546424B2Layered content delivery for virtual and augmented reality experiences

Publication Date: 2020.01.28 GOOGLE LLC

AI summary

A virtual reality or augmented reality experience of a scene may be presented to a viewer using layered data retrieval and/or processing. A first layer of a video stream may be retrieved, and a first viewer position and/or orientation may be received. The first layer may be processed to generate first viewpoint video of the scene from a first virtual viewpoint corresponding to the first viewer position and/or orientation. The first viewpoint video may be displayed for the viewer. Then, a second layer of the video stream may be retrieved, and a second viewer position and/or orientation may be received. The second layer may be processed to generate second viewpoint video of the scene from a second virtual viewpoint corresponding to the second viewer position and/or orientation, with higher quality than the first viewpoint video. The second viewpoint video may be displayed for the viewer.