Temporal Picture Coding Structure Signaling in Video Bitstreams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression standards like H.264 lack a mechanism to explicitly describe temporal picture coding structures, making it difficult for decoders and Media-Aware Network Elements (MANEs) to associate coded pictures with temporal layers, which limits flexibility in bitstream management and resource allocation.
Innovation Solution
A new pattern description syntax is introduced that can be part of an SEI message or parameter set, allowing decoders and MANEs to easily identify and utilize temporal picture coding structures, enabling efficient transrating, bitstream extraction, and resource management by providing information on picture types, temporal layers, and prediction relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If explicit signaling of temporal picture coding structure is not provided in the bitstream, then bitstream overhead is reduced, but decoders and MANEs cannot easily identify and associate coded pictures with temporal layers
Solution Approach 1:
The patent applies preliminary action by encoding temporal picture coding structure information (such as temporal_id, picture_type, and prediction relationships) in advance within the bitstream syntax structure. This allows decoders and MANEs to readily access and associate coded pictures with their temporal layers without requiring complex post-processing or deep bitstream inspection, thus preventing information loss while avoiding excessive complexity.
2Loss of information
If deep bitstream inspection is used by decoders to derive coding structure, then temporal layer association can be obtained, but decoding complexity and processing time increase
Solution Approach 1:
The patent enables self-service by incorporating explicit temporal picture coding structure syntax elements directly into the bitstream that decoders can access without performing deep inspection. The coding structure information is made self-evident through standardized syntax fields (such as temporal_id, pic_order_cnt, and prediction mode indicators), allowing decoders to efficiently derive temporal layer associations through simple parsing rather than complex analysis, thus maintaining high decoding speed while preventing information loss.
3Quantity of substance
If reference picture storage is limited by H.264 profiles and levels, then memory usage is controlled, but flexibility in temporal scalability is reduced
Solution Approach 1:
The patent applies dynamics by enabling adaptive reference picture management through explicit temporal picture coding structure signaling. The system can dynamically adjust which pictures are stored as references and their associated temporal layer information based on the encoded syntax elements, allowing flexible temporal scalability within H.264 profile and level constraints. This prevents loss of temporal scalability flexibility while controlling reference picture storage capacity through efficient memory management guided by the explicit structure information.
Data Source
AI summary
The disclosed subject matter describes a new pattern description that can be part of an SEI message or a parameter set, and can be used to describe a temporal picture coding structure and associated bitrates and frame rates. The knowledge of the coding structure can benefit transraters, bitstream extractors, and digital video recorders. Decoders can utilize the coding structure information for example, to decode pictures in parallel, using multiple threads/cores.