Absolute Reference Picture Signaling for Video Decoder Resilience
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding and decoding technologies, such as H.264/AVC, are vulnerable to losses of pictures containing Memory Management Control Operations (MMCO) types 2, 3, 4, 5, or 6, leading to incorrect reference picture lists and out-of-sync behavior, which cannot be recovered through conventional means like constrained intra blocks or Instantaneous Decoder Refresh (IDR) pictures.
Innovation Solution
The proposed solution involves absolute and explicit reference picture signaling, where the encoded representation of a picture contains buffer description information defining multiple reference pictures, allowing the decoder to update the decoded picture buffer based on determined picture identifiers, independent of correctly received previous pictures, and using delta picture identifiers to reduce bitstream overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If relative reference picture signaling is used (H.264/AVC approach), then the bitstream overhead is reduced, but the system becomes vulnerable to picture losses containing MMCO commands, leading to out-of-sync behavior between encoder and decoder
Solution Approach 1:
The patent applies preliminary action by explicitly signaling the buffer description and reference picture identifiers in advance within the picture data itself, before decoding occurs. This ensures that the decoder has all necessary information to correctly identify reference pictures without relying on implicit MMCO commands that may be lost during transmission.
Solution Approach 2:
The patent creates a copy of the reference picture identification information by embedding the buffer description and picture identifiers directly in the picture data. This redundant copying of essential reference information ensures that even if other parts of the bitstream are lost, the decoder can still correctly identify reference pictures.
2Device complexity
If implicit MMCO operations are used for reference picture management, then the encoding complexity is reduced, but the decoder cannot recover from picture losses containing MMCO commands of type 2, 3, 4, 5, or 6
Solution Approach 1:
The patent implements feedback by having the decoder explicitly read and verify reference picture identifiers from the picture data itself. This creates a self-checking mechanism where the decoder can independently confirm the correctness of reference picture selection without relying on implicit commands, enabling automatic error recovery.
Solution Approach 2:
The picture data serves itself by containing all necessary buffer description and reference picture identification information within its own structure. This self-service approach eliminates the need for separate MMCO command structures and allows the picture to be correctly decoded even if other parts of the bitstream are lost.
3Reliability
If absolute reference picture signaling is implemented, then error resilience is improved, but the bitrate requirements for signaling increase
Solution Approach 1:
The patent applies local quality by differentiating how reference picture identifiers are signaled based on their position and importance. The buffer description uses absolute values for critical reference pictures, while other identifiers may use relative or differential encoding, optimizing the balance between error resilience and bitrate efficiency for different parts of the reference structure.
Data Source
AI summary
An encoded representation (60) of a picture (10) of a video stream (1) is decoded by retrieving buffer description information identifying a buffer description defining multiple reference pictures (40, 42) of the video stream (1) from the encoded representation (60). A picture identifier of a reference picture (42) is determined based on picture identifier information retrieved from the buffer description for the reference picture (42) and a reference picture preceding the reference picture according to a list order in the buffer description. The determined picture identifier is used to update a decoded picture buffer (230, 350) that stores reference pictures (40, 42) to be used as decoding reference for the picture (10) and/or any subsequent picture (50) of the video stream (1).