Media Extractor Tracks for Non-Consecutive NAL Unit Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards face challenges in efficiently encoding and decoding multi-track video data, particularly in supporting non-consecutive network access layer (NAL) units and adapting to varying bandwidth and decoder capabilities, which affects the transmission and playback of multiview video coding (MVC) content.
Innovation Solution
The introduction of media extractor tracks in the ISO base media format that can reference non-consecutive NAL units, allowing for efficient extraction and decoding of MVC operation points, and the modification of the 3GPP file format to include frame rate as a track selection attribute, enabling better adaptation to bandwidth and decoder capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional video coding standards are used, then compatibility with existing systems is maintained, but the ability to efficiently extract non-consecutive NAL units and support operation point selection is limited
Solution Approach 1:
The patent segments the video bitstream into separate tracks, with at least one track containing only extractors that reference non-consecutive NAL units. This segmentation allows independent extraction of specific NAL units without processing the entire bitstream, enabling efficient operation point selection while maintaining manageable track structures through systematic organization of extractors and samples.
Solution Approach 2:
The patent introduces extractor objects as intermediaries between the bitstream and the decoding process. These extractors act as mediators that selectively retrieve non-consecutive NAL units from the bitstream without requiring complex processing, simplifying the extraction mechanism while enabling flexible operation point adaptation through the intermediary extractor layer.
2Productivity
If all NAL units are extracted and decoded, then complete video data is obtained, but bandwidth consumption and processing time increase
Solution Approach 1:
The patent applies extraction by selectively removing and retrieving only the necessary non-consecutive NAL units from the bitstream through the extractor mechanism. This allows the system to obtain complete video data when needed while significantly reducing the quantity of NAL units processed during operation point selection, thereby increasing decoding speed and reducing bandwidth consumption.
Solution Approach 2:
The patent enables partial action by extracting only the necessary NAL units required for a specific operation point rather than processing the entire bitstream. This partial extraction approach allows the system to decode video at lower quality or reduced resolution when bandwidth is limited, improving productivity by reducing processing time and resource consumption.
3Adaptability or versatility
If fixed track selection is used, then decoding process is simple, but adaptability to varying bandwidth and decoder capabilities is reduced
Solution Approach 1:
The patent introduces dynamics by enabling dynamic track selection based on available bandwidth and decoder capabilities. The system can switch between different tracks containing different sets of extractors and NAL units, allowing adaptive operation point selection that responds to changing network conditions and hardware capabilities while maintaining ease of operation through automated adaptation mechanisms.
Data Source
AI summary
A video coding apparatus may be configured to utilize media extractors in a media extractor track that reference two or more non-consecutive network access layer (NAL) units of a separate track. An example apparatus includes a multiplexer to construct a first track including a video sample comprising NAL units, based on encoded video data, wherein the video sample is included in an access unit, construct a second track including an extractor that identifies at least first one of the NAL units in the video sample of the first track, and wherein the extractor identifies a second NAL unit of the access unit, wherein the first identified NAL unit and the second identified NAL unit are non-consecutive, and include the first track and the second track in a video file conforming at least in part to ISO base media file format. The identified NAL units may be in separate tracks.