Multi-Layer Video Decoding Unit Interleaving for Low-Delay Playback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-layer video coding technologies face challenges in achieving ultra-low end-to-end delay while maintaining compatibility with decoders that may not support advanced low-delay concepts, particularly in Scalable Video Coding (SVC), MV-HEVC, and 3D-HEVC extensions.
Innovation Solution
The proposed solution involves encoding video content in sub-portions of pictures using inter-layer prediction, with each sub-portion encoded into payload packets associated with specific layers, and subdividing access units into decoding units that are interleaved across layers, accompanied by advanced timing control information for decoder buffer management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If decoding units of different layers are interleaved across layers to enable ultra-low delay, then end-to-end delay is reduced, but decoder complexity increases due to advanced timing control requirements
Solution Approach 1:
The access unit is segmented into multiple decoding units (DUs), each associated with a specific layer. These DUs are arranged in interleaved order across layers in the bitstream, allowing the decoder to process and output decoded data from different layers in parallel, thereby reducing end-to-end delay while maintaining manageable decoder complexity through structured organization
Solution Approach 2:
The patent introduces dynamic timing control mechanisms including per-decoding-unit removal time indicators and coded picture buffer (CPB) operation parameters that adapt to the interleaved DU structure. This dynamic control allows the system to optimize buffer management and timing for ultra-low delay operation without requiring fundamentally complex decoder architecture
2Productivity
If advanced timing control information is introduced for per-decoding-unit buffer management, then decoding efficiency is improved, but compatibility with legacy decoders deteriorates
Solution Approach 1:
The patent designs the timing control information structure to serve multiple functions: it provides detailed per-DU removal time control for advanced decoders to achieve ultra-low delay, while simultaneously maintaining compatibility with legacy decoders through fallback mechanisms. The same bitstream structure can be interpreted differently by different decoder types, allowing universal support across decoder generations
Solution Approach 2:
The patent introduces intermediary elements such as SEI (Supplementary Enhancement Information) messages and optional parameters that carry advanced timing control information. These intermediaries provide the additional control functionality for modern decoders without fundamentally altering the core H.265 bitstream structure, thus maintaining compatibility with legacy decoders that ignore the optional enhancements
3Ease of manufacture
If decoding units are arranged in sequential layer order, then decoder implementation is simplified, but parallel processing capability is reduced
Solution Approach 1:
The access unit is divided into separate decoding units, each clearly associated with a specific layer. This segmentation allows the bitstream to be organized in interleaved order (layer 0 DU, layer 1 DU, layer 0 DU, etc.) rather than sequential layer order, enabling the decoder to initiate processing of multiple layers simultaneously while maintaining clear boundaries that simplify implementation
Solution Approach 2:
The patent transitions from a single-dimensional sequential processing model to a multi-dimensional organization where decoding units are arranged in both layer dimension and temporal dimension. This allows parallel processing across layers while maintaining sequential access within each DU, effectively adding a layer dimension to the processing architecture without complicating the basic decoder logic
Data Source
AI summary
An interleaved multi-layered video data stream with interleaved decoding units of different layers is provided with further timing control information in addition to the timing control information reflecting the interleaved decoding unit arrangement. The additional timing control information pertains to either a fallback position according to which all decoding units of an access unit are treated at the decoded buffer access unit-wise, or a fallback position according to which an intermediate procedure is used: the interleaving of the DUs of different layers is reversed according to the additionally sent timing control information, thereby enabling a DU-wise treatment at the decoder's buffer, however, with no interleaving of decoding units relating to different layers. Both fallback positions may be present concurrently. Various advantageous embodiments and alternatives are the subject of the various claims attached herewith.