HEVC SEI Message Handling in Multi-Layer Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of HEVC SEI messages in a multi-layer context presents challenges such as increased complexity, syntax inconsistencies, and errors that can cause multi-layer video codecs to malfunction, including issues with layer identification, nesting, and error resilience.
Innovation Solution
Applying constraints on certain syntax elements, such as disallowing SEI NAL units to follow VCL NAL units of higher layers, specifying the set of applicable layers for decoded picture hash and active parameter sets, and controlling the placement of prefix and suffix SEI messages, to ensure correct handling and minimize errors in multi-layer coding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If SEI messages are used in multi-layer video coding, then supplemental enhancement information can be conveyed, but complexity increases and syntax inconsistencies arise
Solution Approach 1:
The patent segments SEI message handling by layer identifier, separating the processing of SEI messages into layer-specific operations. Each SEI message is associated with a specific layer through layer identification syntax elements, allowing independent processing and validation of SEI messages for each layer without affecting other layers, thereby reducing overall system complexity while maintaining error resilience.
Solution Approach 2:
The patent applies local quality by making SEI message syntax and processing layer-specific. Different layers can have different SEI message configurations, validity flags, and processing rules tailored to their specific requirements. This allows optimization of error resilience for each layer individually without imposing unnecessary complexity on all layers.
2Adaptability or versatility
If SEI NAL units are freely placed in bitstream, then flexibility is improved, but syntax inconsistencies and errors increase
Solution Approach 1:
The patent implements preliminary action by establishing layer identification syntax elements and validity flags before SEI message processing occurs. The layer identifier is assigned to each SEI NAL unit in advance, and validity flags are set beforehand to indicate which layers should process which SEI messages. This pre-configuration ensures syntax consistency is maintained while allowing flexible placement of SEI NAL units throughout the bitstream.
Solution Approach 2:
The patent introduces layer identifier syntax elements and validity flags as intermediary mechanisms between SEI NAL units and the decoding process. These intermediaries carry layer-specific information that mediates the interaction between SEI messages and multi-layer video data, ensuring that SEI messages are correctly associated with their target layers without requiring rigid bitstream structure, thus maintaining both flexibility and consistency.
3Productivity
If SEI messages are processed without layer constraints, then processing speed is improved, but decoding accuracy decreases
Solution Approach 1:
The patent implements self-service by enabling each layer to independently process only the SEI messages designated for it through layer identification and validity flags. Each layer's decoder can autonomously determine which SEI messages apply to it without requiring complex inter-layer coordination or validation, thereby maintaining high processing speed while ensuring decoding accuracy through layer-specific message handling.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In an example, a method of coding video data includes obtaining one or more video coding layer (VCL) network abstraction layer (NAL) units of an access unit and a first layer of a multi-layer bitstream of video data. The method also includes only coding one or more non-VCL NAL units containing an SEI message applicable to the VCL NAL units of the first layer together with the VCL NAL units of the first layer such that within the access unit the bitstream does not contain any coded pictures of any other layer of the multi-layer bitstream between the VCL NAL units of the first layer and the non-VCL NAL units containing the SEI message applicable to the VCL NAL units of the first layer.