Temporal Scalability Syntax for Video Bitstream Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression technologies, such as H.264, face challenges in efficiently identifying and pruning non-base layer pictures required for prediction, especially in scenarios with limited computational resources or variable network bitrates, leading to potential drops in visual quality and frame rate.
Innovation Solution
Incorporating temporal_id, temporal_nesting_flag, and temporal_switching_point_flag in the normative parts of the bitstream, such as the sequence parameter set and slice header, allows for efficient identification and pruning of temporal layers, enabling flexible prediction techniques and layer switching without deep bitstream analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep bitstream analysis is performed to identify non-base layer pictures required for prediction, then pruning accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies preliminary action by embedding temporal scalability information (temporal_id, temporal_nesting_flag, temporal_switching_point_flag) into the bitstream structure in advance. This allows the decoder to identify reference picture requirements without performing deep analysis during decoding, thus resolving the contradiction between pruning accuracy and computational complexity
Solution Approach 2:
The patent introduces temporal scalability syntax elements as intermediaries between the encoded video data and the decoder. These intermediaries carry pre-computed temporal relationship information, enabling the decoder to make pruning decisions based on simple flag checks rather than complex analysis, thereby reducing computational complexity while maintaining pruning accuracy
2Reliability
If all temporal layers are decoded to maintain visual quality, then visual quality is improved, but bitrate consumption increases
Solution Approach 1:
The patent enables dynamic adaptation of temporal layer decoding based on network conditions and playback requirements. The temporal_switching_point_flag and temporal_nesting_flag allow the system to dynamically switch between decoding all layers (for high quality) and pruning certain layers (for lower bitrate), thus resolving the contradiction between visual quality and bitrate consumption
Solution Approach 2:
The patent changes the parameter of temporal layer selection by introducing flags that indicate which temporal layers are necessary for prediction. This allows the system to adjust the number of decoded layers based on available bitrate while maintaining visual quality through intelligent selection of essential layers
3Productivity
If temporal scalability information is embedded in normative bitstream parts, then bitstream manipulation efficiency is improved, but conformance requirements increase
Solution Approach 1:
The patent achieves universality by embedding temporal scalability information in normative bitstream parts (sequence parameter set, slice header) that are already required by H.264 standards. This multi-functional approach allows the same bitstream structures to serve both standard compliance and temporal scalability indication purposes, thus improving manipulation efficiency without significantly increasing conformance complexity
Data Source
AI summary
Disclosed are techniques for temporal scalable video coding. By including a temporal_switching_point_flag and/or a temporal_nesting_flag into a normative part of the scalable bitstream, reference picture management in decoder can be simplified, and media-aware network elements and decoder can easily identify those pictures from which onward a higher temporal layer can be added.