Inter-layer RPS Derivation Using Sub-layer Reference Prediction Dependency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current HEVC extension methods for signaling and reference picture set (RPS) derivation in scalable video coding (SVC) are inefficient as they do not allow for signaling overhead savings when some pictures in sub-layers are restricted from inter-layer prediction, leading to high memory and signaling costs.
Innovation Solution
The proposed solution involves a video encoder determining whether direct reference layer pictures can be used for inter-layer prediction based on their temporal identification values and setting flags to indicate their inclusion in an inter-layer reference picture set, allowing for more frequent signaling overhead savings even when some pictures are restricted.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If current HEVC extension methods are used for signaling and RPS derivation, then inter-layer prediction can be performed, but signaling overhead and memory costs are high
Solution Approach 1:
The patent extracts and separately identifies restricted pictures from the general reference picture set. By introducing a specific flag (restricted_il_ref_pics_flag) and separate handling for restricted inter-layer reference pictures, the solution isolates the problematic element (restricted pictures) from the overall RPS management, allowing efficient signaling only when necessary while maintaining full inter-layer prediction functionality.
Solution Approach 2:
The patent implements dynamic signaling by conditionally setting the restricted_il_ref_pics_flag based on whether restricted pictures exist in the reference picture set. The system adapts its signaling behavior: when restricted pictures are present, the flag is set to indicate their presence; when absent, the flag remains unset. This dynamic approach optimizes signaling overhead while ensuring reliable inter-layer prediction based on current video content characteristics.
2Measurement precision
If all direct reference layer pictures are included in inter-layer RPS subset, then prediction accuracy is improved, but memory allocation increases
Solution Approach 1:
The patent applies local quality by differentiating between restricted and non-restricted reference pictures within the reference picture set. Instead of treating all pictures uniformly, the solution assigns different properties and handling methods to restricted pictures (which should not be used for inter-layer prediction) versus non-restricted pictures (which can be used). This localized differentiation optimizes memory usage by excluding inappropriate references while maintaining prediction accuracy using suitable pictures.
3Reliability
If temporal identification threshold is lowered, then more pictures are restricted from inter-layer prediction, but signaling overhead increases
Solution Approach 1:
The patent implements self-service by allowing the decoder to automatically determine the presence of restricted pictures through the restricted_il_ref_pics_flag and existing RPS indexing mechanisms. The encoder sets the flag based on whether restricted pictures exist, and the decoder uses this flag combined with standard RPS parsing to identify restricted pictures without requiring additional explicit signaling. This self-service approach maintains prediction reliability while minimizing signaling overhead.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Systems and methods for inter-layer reference picture set derivation based on sub-layer reference prediction dependency are described herein. One aspect of the subject matter described in the disclosure provides a video encoder comprising a memory configured to store one or more direct reference layer pictures of one or more current pictures in a sequence, wherein the one or more current pictures are associated with a current layer, the current layer being associated with the one or more direct reference layers. The video encoder further comprises a processor in communication with the memory unit. The memory unit is configured to set an indication associated with a current picture to indicate whether all of the one or more direct reference layer pictures of the current picture that are not restricted for use in inter-layer prediction are included in an inter-layer reference picture set associated with the current picture.