Multi-layer Video Codec SEI Extraction Mode Indication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding systems face challenges in determining the extraction mode used to produce sub-bitstreams, leading to inconsistencies in decoding capabilities, as fully-extractable and size-optimized sub-bitstreams may not conform to video coding standards, affecting decoding efficiency and bandwidth optimization.
Innovation Solution
Incorporating Supplemental Enhancement Information (SEI) messages to indicate the extraction mode used, ensuring that sub-bitstreams contain sufficient Network Abstraction Layer (NAL) units for correct decoding and optimizing bitstream size by excluding unnecessary pictures, while allowing flexible extraction modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If sub-bitstreams are extracted using size-optimized mode to reduce bandwidth, then bitstream size is reduced, but decoding compliance with video coding standards cannot be guaranteed
Solution Approach 1:
An intermediate device (e.g., network element, gateway) is introduced to perform bitstream extraction and mode indication. This intermediary adds metadata (extraction mode indicators) to the bitstream without significantly increasing size, enabling decoders to properly handle extracted sub-bitstreams while maintaining compliance with video coding standards
Solution Approach 2:
The extraction mode indication is prepared and embedded in the bitstream during the extraction process itself, before the bitstream reaches the decoder. This preliminary action ensures that decoding compliance is maintained from the outset rather than requiring post-processing or complex decoder logic
2Adaptability or versatility
If multiple extraction modes are supported to increase flexibility, then adaptability is improved, but system complexity increases
Solution Approach 1:
The extraction functionality is segmented into distinct modes (fully-extractable, size-optimized, etc.), each with clearly defined behavior and requirements. This segmentation allows the system to support multiple modes without requiring complex logic to handle all possibilities simultaneously, as each mode can be processed according to its specific rules
Solution Approach 2:
The patent uses lightweight metadata structures (e.g., single-bit flags, simple parameters) to indicate extraction modes. These minimal indicators provide the necessary information for decoder behavior without adding significant complexity to the codec structure or processing requirements
3Reliability
If fully-extractable mode is used to ensure decodability, then decoding reliability is improved, but bitstream size increases
Solution Approach 1:
The patent introduces parameters (extraction mode indicators, target layer identifiers) that control the extraction behavior. By changing these parameters, the system can switch between fully-extractable mode (higher reliability, larger size) and size-optimized mode (lower reliability, smaller size), allowing flexible trade-offs based on specific application requirements
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computing device may obtain, from a first bitstream that includes a coded representation of the video data, a Supplemental Enhancement Information (SEI) message that includes an indication of an extraction mode that was used to produce the first bitstream. If the extraction mode is the first extraction mode, the first bitstream includes one or more coded pictures not needed for correct decoding of the target output layer set. If the extraction mode is the second extraction mode, the first bitstream does not include the one or more coded pictures not needed for correct decoding of the target output layer set.