Backward-Compatible RTP Payload Characterization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video transport and storage technologies face challenges in efficiently determining which coded data units to forward or process, particularly in multipoint communication systems, file playback, layer or view switching, and bitrate adaptation, due to complexities in accessing necessary information within NAL units and aggregation packets.
Innovation Solution
A system and method that conveys easy-to-access information within RTP payloads or file format data units to help network middleboxes and media players identify redundant coded pictures, temporal level switching points, gradual decoding refresh access points, view identifiers, and view random access points, using extended PACSI NAL units or aggregator NAL units to facilitate decision-making on data units to be transmitted or processed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If NAL units are aggregated into RTP packets without additional characterization information, then packet transmission efficiency is maintained, but network middleboxes cannot efficiently decide which packets to forward or discard
Solution Approach 1:
The patent applies preliminary action by inserting characterization information (CI) into the packet structure in advance, before the middlebox needs to make forwarding decisions. This pre-packaging of metadata about NAL unit types, scalability layers, and view identifiers allows middleboxes to quickly assess packet importance without parsing the entire packet contents, thus improving decision-making ease while maintaining manageable packet structure complexity.
Solution Approach 2:
The characterization information acts as an intermediary layer between the NAL units and the middlebox decision-making process. This intermediate metadata structure provides a simplified interface that translates complex NAL unit characteristics into easily interpretable indicators, enabling middleboxes to make informed forwarding decisions without directly analyzing the underlying video data structures.
2Adaptability or versatility
If all NAL units are transmitted to ensure compatibility with all receivers, then receiver compatibility is improved, but bandwidth utilization deteriorates due to transmitting unnecessary data to certain receivers
Solution Approach 1:
The patent applies local quality by enabling different portions of the bitstream to be treated differently based on their characteristics. The characterization information identifies specific NAL units with particular properties (e.g., base layer vs. enhancement layer, mandatory vs. optional), allowing receivers to selectively process only the portions relevant to their capabilities while ignoring unnecessary data, thus optimizing bandwidth utilization while maintaining adaptability.
Solution Approach 2:
The patent changes the parameter of data transmission by using the characterization information to dynamically adjust which NAL units are transmitted based on receiver capabilities. Middleboxes can modify transmission parameters (such as dropping enhancement layer packets for basic receivers) guided by the CI, enabling efficient bandwidth utilization while maintaining compatibility with diverse receiver capabilities.
3Measurement precision
If middleboxes parse each NAL unit header to identify characteristics, then accurate identification is achieved, but processing time increases significantly
Solution Approach 1:
The patent extracts the essential characteristic information from the NAL units and places it in a readily accessible location within the packet structure. Instead of requiring middleboxes to parse each NAL unit header to identify characteristics, the extraction of key metadata (NAL unit type, scalability information, view identifiers) into the packet header or preceding fields allows for rapid identification with high accuracy, significantly reducing processing time while maintaining measurement precision.
4Reliability
If receivers process all received NAL units, then complete video playback is ensured, but device resource consumption increases unnecessarily
Solution Approach 1:
The patent applies partial action by enabling receivers to process only the necessary subset of NAL units based on their capabilities and the characterization information provided. Receivers can perform partial processing of the bitstream, focusing computational resources on decoding essential base layer data while optionally skipping enhancement layers or non-essential NAL units, thus ensuring reliable video playback while significantly reducing device energy consumption.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and method for conveying information that is helpful for a network middlebox or a media player to decided which coded data units to forward or process within an RTP payload or a file format data unit in an easy-to-access manner. This mechanism can be used to provide indications of items such as redundant coded pictures, temporal level switching points, gradual decoding refresh access points, view identifiers, and view random access points. A middlebox and/or receiver can then use this information to determine whether certain coded data units need to be processed and/or transmitted.