Tile-Based Video Coding for Viewport Quality Update Delay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards face challenges in reducing the viewport quality update delay in viewport-adaptive streaming for 360-degree video content, particularly when the viewing orientation changes, due to the limitations of mixing random-access and non-random-access pictures in the same coded picture.
Innovation Solution
The method involves encoding at least four bitstream versions of the same content, with independently coded tile sets representing different spatial regions at varying quality levels and random access picture intervals. These bitstreams are then grouped into collocated sub-picture tracks, and instructions are generated to merge tile sets from different spatial locations into a single coded picture, allowing a tile set from a random access picture to be decoded as if it were from a non-random-access picture when merged with a non-random-access picture tile set.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If SAP intervals are extended to reduce streaming bitrate, then bandwidth consumption is reduced, but viewport quality update delay increases
Solution Approach 1:
The video content is segmented into independently coded tile sets representing different spatial regions. Each tile set can be decoded independently, allowing selective transmission of only those tiles needed for the current viewport. This segmentation enables frequent quality updates without requiring full SAP intervals, as individual tiles can be updated separately.
Solution Approach 2:
Different quality levels are applied to different spatial regions through the tile-based structure. The primary viewport region receives high-quality independently coded tiles, while peripheral regions use lower quality or predictive coding. This allows the system to maintain high quality where needed while reducing overall bitrate, and enables faster quality updates in the viewport region without being constrained by global SAP intervals.
2Loss of time
If multiple bitstream versions are created with different random access picture intervals, then viewport quality update frequency is increased, but device complexity increases
Solution Approach 1:
The patent merges independently coded tile sets from different bitstream versions (with different random access picture intervals) into a single coded picture. The decoding device receives multiple bitstreams with different RAP intervals and combines their tile sets, allowing frequent quality updates while managing complexity through standardized merging procedures defined in the video coding standard.
Solution Approach 2:
The patent introduces a new dimension of organization by grouping independently coded tile sets into collocated sub-picture tracks. This track-based organization allows the system to manage multiple bitstream versions with different RAP intervals in a structured way, where tracks are grouped by spatial region and can be selectively decoded and merged, reducing the perceived complexity at the decoding device.
3Loss of time
If tile sets from different random access pictures are merged, then quality update frequency is improved, but compliance with video coding standards becomes challenging
Solution Approach 1:
The patent dynamically adjusts the decoding process based on the random access picture type of each incoming tile set. When a tile set from a random access picture is merged with tiles from non-random-access pictures, the decoding device applies appropriate initialization and reference picture management dynamically. This dynamic adaptation allows standard compliance while enabling frequent quality updates through merged tile sets from different RAP intervals.
Data Source
AI summary
Four or more bitstream versions of a same content are encoded and divided into segments of independently coded tile sets representing multiple spatial regions. First and second bitstreams include independently coded tile sets encoded at a first quality. Third and fourth bitstreams include independently coded tile sets encoded at a second quality. First and third bitstreams have first random access picture interval. Second and fourth bitstreams have second random access picture interval. Independently coded tile sets are grouped into multiple groups of collocated sub-picture tracks, only one of said tile sets per group is intended to be received and/or decoded per any segment. Instruction(s) are generated for merging tile sets of different spatial locations into coded picture(s), causing a tile set originating from a random access picture to be decoded as a tile set originating from a non-random-access picture when merged with a tile set originating from a non-random-access picture.