Multi-layer HEVC Bitstream Parameter Set Placement for Random Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing file formats for multi-layer HEVC bitstreams do not adequately ensure that parameter sets necessary for decoding Intra Random Access Point (IRAP) pictures are provided to video decoders, leading to decoding failures, especially in devices that cannot parse the bitstream, and result in increased complexity and file size.
Innovation Solution
A file format restriction is implemented where each IRAP picture's parameter set is included in its respective track, a temporally collocated sample in a reference layer track, or the sample itself, ensuring that parameter sets are available for decoding, reducing unnecessary copies and file size.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If parameter sets are included in every track for each IRAP picture, then decoding reliability is improved, but device complexity and file size increase
Solution Approach 1:
The patent segments the bitstream into multiple tracks, with each track containing parameter sets relevant to specific layers. This allows devices to selectively access only the parameter sets needed for their specific decoding requirements, reducing the complexity of processing all parameter sets while maintaining reliable decoding through organized accessibility.
Solution Approach 2:
Parameter sets are prepared and organized in advance within track structures before decoding occurs. This preliminary organization allows devices to efficiently locate and access required parameter sets without performing complex parsing operations during real-time decoding, thereby reducing operational complexity while ensuring reliability.
2Device complexity
If parameter sets are stored in separate NAL units away from encoded pictures, then bitstream structure is simplified, but random access capability deteriorates
Solution Approach 1:
The patent merges parameter sets with their corresponding encoded pictures by including them within the same track structures. This combination ensures that when a device accesses an encoded picture, the required parameter sets are immediately available in the same location, enabling efficient random access without requiring separate retrieval operations from distant NAL units.
Solution Approach 2:
Tracks serve as intermediary structures that bridge the gap between encoded pictures and their associated parameter sets. By organizing both elements within the same track, the system provides a natural mapping relationship that simplifies random access operations while maintaining a structured bitstream organization.
3Reliability
If devices are configured to parse the L-HEVC bitstream, then parameter set availability is improved, but processing speed deteriorates
Solution Approach 1:
The patent extracts parameter sets from the complex L-HEVC bitstream structure and places them in organized track structures that are easier to access. This extraction allows devices to obtain required parameter sets without performing complex parsing operations on the entire bitstream, thereby maintaining parameter set availability while significantly improving processing speed.
Solution Approach 2:
Different tracks are designed with different levels of parameter set inclusion based on local requirements. Devices can access only the parameter sets locally relevant to their decoding needs within specific tracks, avoiding the overhead of parsing the entire bitstream structure while ensuring all necessary parameter sets are available.
4Reliability
If multiple copies of parameter sets are included in the file, then parameter set accessibility is improved, but file size increases
Solution Approach 1:
Parameter sets organized in track structures serve multiple functions simultaneously: they are accessible to devices needing random access, they maintain a compact representation avoiding redundant copies, and they can be selectively accessed based on device capabilities. This multi-functional organization achieves parameter set accessibility without proportionally increasing file size through unnecessary repetitions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A device generates, stores, or obtains a file that complies with a restriction specified in a file format to which the file conforms. The restriction requires that for each respective Intra Random Access Point (IRAP) picture, a parameter set required for decoding the respective IRAP picture is in at least one of the following: a sample entry of the respective IRAP pictures track that is applicable to the respective IRAP pictures sample, a sample entry of a non-corresponding track for the respective IRAP picture carrying a reference layer of the respective IRAP pictures layer, the respective IRAP pictures sample, or a temporally collocated sample of the respective IRAP pictures sample in a track of that carries the reference layer.