SEI Message Handling in Layered Video Bitstreams for Conformance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in bitstream conformance, particularly with Picture Order Count (POC) constraints and subpicture sub-bitstream extraction processes, leading to non-conforming bitstreams during sub-bitstream extraction and merging of video content.
Innovation Solution
The proposed solution involves modifying the POC constraint rules to allow for correct POC derivation in multi-layer bitstreams and optimizing the sub-bitstream extraction process by removing unnecessary SEI messages and replacing parameter sets, ensuring that all SEI messages with a particular payload type have the same content across associated layers or access units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If POC constraints are applied during sub-bitstream extraction, then decoding accuracy is improved, but bitstream conformance deteriorates
Solution Approach 1:
The patent changes the parameter constraints for POC MSB signaling based on the extraction context. During sub-bitstream extraction, the patent relaxes certain POC constraints while maintaining others, allowing the POC derivation to adapt to the extracted bitstream conditions rather than strictly following original bitstream constraints. This resolves the contradiction by making the POC parameters flexible rather than fixed.
Solution Approach 2:
The patent introduces dynamic handling of POC constraints during sub-bitstream extraction. The decoding process dynamically adjusts POC derivation based on whether the bitstream is being extracted or originally decoded, and whether POC MSB is signaled. This dynamic adaptation allows the system to maintain POC accuracy while ensuring conformance to extraction-specific requirements.
2Loss of information
If SEI messages are retained in the bitstream, then information completeness is improved, but bitstream complexity deteriorates
Solution Approach 1:
The patent selectively extracts and removes certain SEI messages from the bitstream during sub-bitstream extraction processes. Specifically, it removes SEI messages that are not relevant to the extracted content or that would cause conformance issues, while retaining essential SEI messages. This extraction approach maintains information completeness for the extracted sub-bitstream while reducing overall bitstream complexity.
Solution Approach 2:
The patent applies partial retention of SEI messages rather than complete retention or complete removal. It keeps only the necessary SEI messages required for the extracted sub-bitstream's proper decoding and conformance, removing redundant or excessive SEI messages. This partial action optimizes the balance between information completeness and complexity reduction.
3Productivity
If sub-bitstream extraction removes all SEI NAL units satisfying a condition, then extraction efficiency is improved, but conformance reliability deteriorates
Solution Approach 1:
The patent performs preliminary assessment of SEI NAL units before removal during sub-bitstream extraction. It evaluates each SEI message against conformance criteria before extraction, ensuring that removing the message will not violate bitstream conformance requirements. This preliminary check maintains conformance reliability while still achieving extraction efficiency by removing only appropriate messages.
Solution Approach 2:
The patent implements a feedback mechanism during sub-bitstream extraction where the conformance status is continuously monitored. When SEI messages are considered for removal, the system checks whether removal would maintain conformance, and adjusts the extraction decision based on this feedback. This ensures extraction efficiency is achieved without compromising conformance reliability.
Data Source
AI summary
Systems, methods and apparatus for encoding, decoding or transcoding digital video are described. One example method of video processing includes performing a conversion between a video and a bitstream of the video including multiple layers, wherein the bitstream includes a plurality of supplemental enhancement information, SEI, messages associated with an access unit, AU, or a decoding unit, DU, of a particular output layer set, OLS, or a particular layer, wherein the plurality of SEI messages, including a message type different from scalable nesting type, is based on a format rule, and wherein the format rule specifies that each of the plurality of SEI messages has a same SEI payload content due to the plurality of SEI messages being associated with the AU or the DU of the particular OLS or the particular layer.