Viewport-Predictive Tiled Video Streaming for Bandwidth Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding techniques struggle to efficiently manage the transmission and rendering of multi-directional video content, particularly in scenarios where the viewer's viewport changes, leading to inefficiencies in bandwidth usage and visual artifacts due to non-uniform compression quality across tiles.
Innovation Solution
The technique involves parsing multi-directional video into spatial tiles and temporal segments, applying a multi-tier coding protocol to adapt bit allocation based on viewport prediction and image content analysis, and dynamically selecting tier collections to optimize bandwidth usage and visual quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If uniform compression quality is applied to all tiles in multi-directional video, then visual quality is consistent across the rendered image, but bandwidth usage is inefficient
Solution Approach 1:
The patent applies different compression qualities to different spatial regions (tiles) of the multi-directional video based on their importance. Viewport tiles that are likely to be displayed receive higher compression quality, while non-viewport tiles receive lower compression quality. This resolves the contradiction by making compression quality non-uniform in a controlled manner that improves bandwidth efficiency while maintaining visual quality where it matters most.
2Manufacturing precision
If high bit allocation is used for all tiles, then visual quality is maintained across the entire multi-directional video, but bandwidth consumption increases
Solution Approach 1:
The patent segments the multi-directional video into multiple independent tiles and applies different bit allocations to different tiles based on their viewport importance. This segmentation allows the system to concentrate bandwidth on important tiles while reducing or eliminating transmission of less important tiles, thereby maintaining visual quality where needed while reducing overall bandwidth consumption.
Solution Approach 2:
Different bit allocation strategies are applied to different tiles based on their likelihood of being in the viewport. Tiles with high viewport probability receive higher bit allocation for better visual quality, while tiles with low probability receive reduced bit allocation. This local differentiation resolves the contradiction between maintaining visual quality and reducing bandwidth consumption.
3Device complexity
If viewport prediction is not used, then coding simplicity is maintained, but bandwidth efficiency decreases due to inability to adapt to viewport changes
Solution Approach 1:
The patent performs preliminary viewport prediction before encoding the video tiles. By predicting the future viewport location in advance, the system can pre-determine which tiles will be important and allocate resources accordingly. This preliminary action enables adaptive bit allocation and tile selection that improves bandwidth efficiency without requiring complex real-time adjustments during encoding.
Solution Approach 2:
The system uses viewport prediction feedback to continuously adapt the encoding parameters. The predicted viewport information feeds back into the encoding process to dynamically adjust tile selection and bit allocation. This feedback mechanism enables the system to adapt to viewport changes while maintaining manageable coding complexity through systematic adaptation rules.
4Manufacturing precision
If all tiles are transmitted at full quality, then visual artifacts are minimized, but bandwidth usage is excessive
Solution Approach 1:
The patent applies different quality levels to different tiles based on their viewport importance. Tiles that are predicted to be in the viewport maintain high visual quality with minimal artifacts, while tiles outside the viewport use lower quality encoding. This local quality differentiation minimizes visual artifacts where they would be most noticeable while dramatically reducing overall bandwidth usage.
Data Source
AI summary
In communication applications, aggregate source image data at a transmitter exceeds the data that is needed to display a rendering of a viewport at a receiver. Improved streaming techniques that include estimating a location of a viewport at a future time. According to such techniques, the viewport may represent a portion of an image from a multi-directional video to be displayed at the future time, and tile(s) of the image may be identified in which the viewport is estimated to be located. In these techniques, the image data of tile(s) in which the viewport is estimated to be located may be requested at a first service tier, and the other tile in which the viewport is not estimated to be located may be requested at a second service tier, lower than the first service tier.