Subpicture Boundary Alignment for Layered Video Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards struggle with efficiently handling the extraction and decoding of specific regions of a bitstream, particularly in applications like 360-degree video streaming, where users with varying bandwidth and computational resources require flexible tile resolution adaptation and efficient merging of sub-pictures without loss of decoding accuracy.
Innovation Solution
Implementing additional signaling mechanisms in video coding standards to align sub-picture boundaries across layers, use unique sub-picture IDs, and manage dummy data to ensure constant bitrate, allowing for efficient extraction and decoding of specific regions of interest.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If motion vectors are constrained to not reference tiles different than the current tile in HEVC, then tiles can be extracted and merged into bitstreams without affecting decoding accuracy, but this constraint causes efficiency loss in the coding process
Solution Approach 1:
The video content is divided into independent decodable units called sub-pictures, which are smaller than full pictures. Each sub-picture can be independently extracted and decoded, enabling flexible tile extraction and merging while maintaining decoding accuracy without requiring motion vector constraints that reduce coding efficiency.
Solution Approach 2:
The patent introduces a new dimensional approach by treating sub-pictures as independent decodable units with their own boundaries, rather than constraining motion vectors within existing tile structures. This allows sub-pictures to be extracted and merged across different bitstreams while maintaining decoding accuracy, resolving the contradiction between reliability and productivity.
2Productivity
If sub-pictures are used instead of constrained motion vectors, then coding efficiency is improved without losing the ability to extract and merge regions, but additional signaling mechanisms are required to align sub-picture boundaries across layers
Solution Approach 1:
Sub-picture boundaries are pre-defined and signaled in advance through additional signaling mechanisms, allowing the decoder to know in advance where sub-picture boundaries are located. This preliminary action enables efficient extraction and merging operations without requiring complex runtime constraints on motion vectors, thus improving coding efficiency while managing signaling complexity.
3Adaptability or versatility
If tiles outside the current viewport are downloaded and decoded to handle sudden orientation changes, then user experience is improved, but computational load and power consumption increase on mobile devices
Solution Approach 1:
The patent enables extraction of specific sub-picture regions from the bitstream, allowing the decoder to process only the necessary sub-pictures required for the current viewport and anticipated orientation changes. This selective extraction reduces the computational load and power consumption compared to decoding all tiles, while still maintaining adaptability to handle sudden orientation changes.
Data Source
AI summary
Concepts are described, including encoding, processing and decoding of a data stream having a video encoded thereinto, wherein the video comprises a plurality of pictures, wherein the data stream comprises a plurality of pictures in at least two layers, wherein the pictures of at least one layer are split into a predetermined layer-specific number of subpictures, one or more of the pictures or subpictures of one layer corresponds to one picture or subpictures in one or more other layer, and at least one of the subpictures comprises boundaries for boundary extension for motion compensation, and an indication that at least one of the boundaries of corresponding subpictures or corresponding pictures in different layers are aligned with each other.