Multi-View Layer Coding With Interleaved NAL Units for Low Delay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-view/layer coding technologies, such as HEVC SVC/NA/0 profiles, face challenges in efficiently enabling parallel decoding/encoding while maintaining low delay processing, particularly due to inter-view prediction dependencies across spatial segments and strict sequential arrangements of NAL units, which hinder further reduction of end-to-end delay.
Innovation Solution
Implementing a layer-identification extension mechanism for signaling inter-view prediction changes at spatial segment boundaries and allowing interleaving of NAL units within access units, while optimizing layer identification signaling based on application complexity, thereby reducing inter-view decoding delay and end-to-end delay without compromising coding efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If inter-view prediction is used across spatial segments to improve coding efficiency, then compression performance is improved, but decoding delay increases due to sequential processing requirements
Solution Approach 1:
The patent divides the video data stream into multiple independent NAL unit types (VPS, SPS, PPS, RAP, non-RAP) and allows parallel processing of different NAL unit types. By segmenting the decoding pipeline into independent processing streams for different NAL unit categories, the system maintains coding efficiency through inter-view prediction while enabling parallel execution to reduce overall decoding delay.
Solution Approach 2:
The patent implements dynamic buffering and reordering mechanisms that adaptively manage the timing and sequence of NAL unit processing. The buffer management system dynamically adjusts to the arrival and processing rates of different NAL unit types, allowing flexible reordering and parallel processing while maintaining the required decoding sequence, thus reducing delay without sacrificing coding efficiency.
2Device complexity
If NAL units are arranged in strict sequential order to simplify processing, then device complexity is reduced, but end-to-end delay increases
Solution Approach 1:
The patent segments the NAL unit stream into distinct categories (VPS, SPS, PPS, RAP, non-RAP) that can be processed in parallel independent streams. Each segment type has its own processing pipeline, allowing simultaneous handling of multiple NAL units without complex inter-dependency management, thus reducing both device complexity and end-to-end delay.
Solution Approach 2:
The patent performs preliminary classification and buffering of NAL units by type before processing. By pre-organizing the data stream into categorized buffers and establishing processing priorities in advance, the system eliminates the need for complex real-time sequencing decisions, reducing processing complexity while enabling parallel execution to minimize delay.
3Reliability
If layer identification is signaled for every packet to ensure accurate layer tracking, then reliability is improved, but data stream overhead increases
Solution Approach 1:
The patent applies layer identification signaling selectively rather than universally. Full layer identification is provided only where necessary for accurate layer tracking, while in other cases layer information is inferred or omitted. This partial signaling approach maintains reliability for critical operations while significantly reducing the overall overhead in the data stream.
Solution Approach 2:
The patent uses reference copying mechanisms where layer identification information from previously decoded packets is reused and copied for subsequent packets when the layer structure remains unchanged. This avoids redundant signaling while maintaining accurate layer tracking, reducing overhead without sacrificing reliability.
Data Source
AI summary
Various concepts which further improve multi-view/layer coding concepts, are described.