Multi-layer Video File Format Sub-sample Information Boxes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The ISO base media file format (ISOBMFF) and its extensions are inadequate for effectively storing video data conforming to Multi-View High Efficiency Video Coding (MV-HEVC), 3D High Efficiency Video Coding (3D-HEVC), and Scalable High Efficiency Video Coding (SHVC) standards, as they fail to properly identify Intra Random Access Point (IRAP) pictures and manage multiple coded pictures within access units, leading to issues with random access and layer switching.
Innovation Solution
Generating files with additional metadata boxes that document IRAP pictures and using sub-sample information boxes to specify the type of sub-sample information, allowing for efficient random access and layer switching without parsing NAL units, and employing multiple tracks to manage pictures with different output flags.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional ISOBMFF format is used for storing multi-layer video data, then file compatibility is maintained, but the ability to identify IRAP pictures and manage multiple coded pictures is insufficient
Solution Approach 1:
The patent divides the sample information into multiple sub-samples, where each sub-sample corresponds to a specific coded picture (VCL or non-VCL). This segmentation allows individual identification and management of different picture types within a sample, enabling proper identification of IRAP pictures and non-VCL NAL units without requiring complex parsing of the entire bitstream.
Solution Approach 2:
The patent introduces sub-sample information boxes that contain flags specifying the type of sub-sample information (VCL or non-VCL) in advance. This preliminary action allows computing devices to pre-identify the nature of each coded picture before processing, enabling efficient random access and layer switching without requiring complex real-time parsing of NAL units.
2Measurement precision
If computing devices parse NAL units to identify IRAP pictures and manage layers, then accurate identification is achieved, but processing time and complexity increase
Solution Approach 1:
The patent introduces sub-sample information boxes as intermediary structures that contain flags specifying the type of sub-sample information. These boxes act as mediators between the raw NAL units and the computing device's processing logic, providing pre-processed identification information that eliminates the need for complex NAL unit parsing while maintaining accurate identification of IRAP pictures and non-VCL NAL units.
3Ease of operation
If multiple tracks are used to manage pictures with different output flags, then proper management of output and reference pictures is achieved, but file structure complexity increases
Solution Approach 1:
The patent segments the media data into multiple tracks, where each track manages a specific subset of pictures (e.g., output pictures vs. reference pictures). This segmentation allows independent management of pictures with different output flags, enabling proper identification and processing of IRAP pictures and non-VCL NAL units without requiring complex parsing of the entire bitstream.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computing device generates a file that comprises a track box that contains metadata for a track in the file. Media data for the track comprises a sequence of samples, each of the samples being a video access unit of multi-layer video data. As part of generating the file, the computing device generates, in the file, a sub-sample information box that contains flags that specify a type of sub-sample information given in the sub-sample information box. When the flags have a particular value, a sub-sample corresponding to the sub-sample information box contains exactly one coded picture and zero or more non-Video Coding Layer (VCL) Network Abstraction Layer (NAL) units associated with the coded picture.