Recovery Point SEI Message Signaling POC MSB in Multi-Layer Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for implementing recovery point SEI messages in video coding can cause inconsistencies in picture order count (POC) values, leading to incorrect referencing of pictures during decoding, especially in multi-layer video scenarios where the most significant bits (MSBs) of POC are inferred rather than signaled, potentially resulting in the assumption of picture loss.
Innovation Solution
Signaling the MSB part of the POC in the recovery point SEI message for layers with nuh_layer_id greater than 0, or encoding constraints that prevent the recovery point SEI message from being present unless specific conditions are met, such as the presence of POC MSB signaled or specific access unit configurations, to ensure accurate POC values are communicated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If recovery point SEI message is included in multi-layer video without signaling POC MSB, then bitstream compatibility is maintained, but picture order count accuracy deteriorates causing incorrect picture referencing
Solution Approach 1:
The POC value is segmented into two parts: POC MSB (most significant bits) and POC LSB (least significant bits). The POC LSB is signaled in the slice header, while the POC MSB is signaled in the recovery point SEI message. This segmentation allows the system to maintain both bitstream compatibility and POC accuracy by distributing different parts of the POC value across different message structures.
Solution Approach 2:
The recovery point SEI message acts as an intermediary carrier for the POC MSB information. Instead of directly embedding POC MSB in the slice header (which would increase complexity), the system uses the SEI message as an intermediate structure to convey the POC MSB, thereby resolving the contradiction between reliability and complexity.
2Loss of information
If POC MSB is inferred rather than signaled, then bitstream overhead is reduced, but picture referencing accuracy deteriorates leading to assumption of picture loss
Solution Approach 1:
The POC MSB value is preliminarily determined and stored in the recovery point SEI message before the actual picture decoding process. This preliminary action ensures that when the slice header is processed, the POC MSB is already available to combine with POC LSB, preventing the need to infer POC values and avoiding incorrect assumptions of picture loss.
Solution Approach 2:
The system implements a feedback mechanism where the recovery point SEI message provides POC MSB information back to the decoding process. This feedback loop ensures that the decoder has accurate POC information without requiring additional bitstream overhead for direct POC signaling, thus resolving the contradiction between information loss and reliability.
3Reliability
If recovery point SEI message is always included, then picture order count accuracy is maintained, but bitstream efficiency deteriorates due to unnecessary message inclusion
Solution Approach 1:
Instead of always including the recovery point SEI message (excessive action) or never including it (insufficient action), the system applies partial action by including the message only when specific conditions are met - namely when POC MSB needs to be signaled for accurate picture referencing. This conditional inclusion maintains POC accuracy while avoiding unnecessary bitstream overhead and improving encoding efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A device for video encoding multi-layer video data includes a memory configured to store at least a portion of a multi-layer bitstream of video data and one or more processors configured to: encode a first access unit comprising at least a layer and a reference layer of the layer; determine if the first access unit is a recovery point; in response to the first access unit being a recovery point, include in the first access unit, a recovery point SEI message that applies to at least the layer and the reference layer; and generate the first access unit with the SEI message.