Tile-Based Video Coding for Seamless Viewport Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Viewport-adaptive streaming for 360-degree video in virtual reality experiences significant delays due to the need for frequent changes in streaming quality, which is costly in terms of rate-distortion performance and results in noticeable lag when the user changes their viewing orientation, as existing methods to reduce switching delay increase the streaming bitrate.
Innovation Solution
A method for reducing streaming bitrate delays by merging tile-based encoded bitstreams with different segment access point intervals, allowing for seamless quality adjustments without increasing the overall bitrate, using a hierarchical data model and efficient encoding techniques to manage quality changes in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If representations of more frequent SAPs are made available to reduce switching delay, then the viewing orientation change delay is reduced, but the streaming bitrate increases significantly
Solution Approach 1:
The patent merges tile rectangles from different Representations with different SAP intervals into a single combined bitstream. This allows the decoder to access tiles from either Representation depending on which SAP is needed, reducing switching delay without requiring separate bitstreams for each SAP interval, thereby controlling bitrate
Solution Approach 2:
The patent divides the 360-degree video content into tile-based segments that can be independently coded and accessed. By organizing content into tiles with different SAP intervals, the system enables selective streaming of specific tile segments, reducing the overall bitrate while maintaining fast switching capability for viewport changes
2Adaptability or versatility
If intra-coded SAPs are used in single-layer video bitstreams, then random access is enabled, but the rate-distortion performance deteriorates and switching delay increases
Solution Approach 1:
The patent implements dynamic SAP intervals where different tile rectangles have different SAP distances based on their importance and access patterns. Frequently accessed tiles have shorter SAP intervals while less critical tiles have longer intervals, optimizing both random access capability and switching delay dynamically
Solution Approach 2:
The patent applies different coding qualities and SAP intervals to different regions (tiles) of the 360-degree video based on their local importance. Primary viewport tiles receive higher quality with frequent SAPs, while peripheral tiles use lower quality with less frequent SAPs, optimizing overall performance
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A method comprising: requesting a first time segment of a first set of a plurality of independently coded tile set from a server; merging a first and a second coded tile set belonging to viewport into the first time segment of a first bitstream; merging a third and a fourth coded tile set into the first time segment of a second bitstream; requesting, in response to a viewport change, a second time segment of a second set of the plurality of independently coded tile set and/or rectangle sequences wherein the first and third coded tile set are predicted from the first and third coded tile set of the first time segment, and a second and fourth coded tile set of the second and fourth spatial region start with a SAP; merging the first and fourth coded tile sets into the second time segment of the first bitstream; and merging the third and the second coded tile set into the second time segment of the second bitstream.