Video Bitstream Tranche Coding for Low-Latency Parallel Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current parallel processing techniques in video encoding and decoding, such as those used in HEVC, face challenges in reducing coding delay while maintaining coding efficiency, especially in multi-threaded environments where data locality and transmission delays become critical, particularly in high-resolution video streaming applications like video conferencing and gaming.
Innovation Solution
The approach involves segmenting data into smaller tranches within WPP substreams or tiles, allowing for continued context-adaptive binary arithmetic coding (CABAC) probability adaptation across tranche boundaries, enabling earlier transmission and decoding of these tranches, which are interleaved and packetized to match maximum transfer unit sizes, thereby reducing overall delay and enabling parallel processing without compromising entropy coding efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If data is segmented into smaller tranches within WPP substreams or tiles, then transmission delay is reduced and parallel processing is enabled, but coding efficiency may be compromised due to interrupted CABAC probability adaptation
Solution Approach 1:
The video data is segmented into smaller tranches within WPP substreams or tiles, allowing independent transmission and parallel processing of multiple tranches simultaneously, thereby reducing transmission delay and enabling multi-threaded decoding
Solution Approach 2:
CABAC probability adaptation is maintained across tranche boundaries through preliminary state preservation, where the entropy decoder continues probability adaptation from previous tranches rather than resetting, preventing coding efficiency loss that would normally result from segmentation
2Loss of energy
If CABAC probability adaptation is continued across tranche boundaries, then coding efficiency is maintained, but complexity of the decoding process increases
Solution Approach 1:
The CABAC probability adaptation process continues continuously across tranche boundaries without interruption or reset, maintaining the entropy coding efficiency that would otherwise be lost due to segmentation, while the decoder thread simply continues processing without additional complex operations
3Productivity
If multiple decoder threads are used for parallel processing, then productivity is improved, but data locality requirements increase transmission complexity
Solution Approach 1:
The bitstream is segmented into tranches that can be independently assigned to different decoder threads, enabling parallel processing and improved productivity while maintaining manageable data transmission through structured segmentation
Solution Approach 2:
The tranche structure acts as an intermediary between the encoded bitstream and multiple decoder threads, organizing data into manageable units that can be efficiently distributed and processed in parallel while maintaining the necessary data locality for each thread
Data Source
AI summary
A raw byte sequence payload describing a picture in slices, WPP substreams or tiles and coded using context-adaptive binary arithmetic coding is subdivided into tranches with continuing the context-adaptive binary arithmetic coding probability adaptation across tranche boundaries. Thereby, tranche boundaries introduced within slices, WPP substreams or tiles do not lead to a reduction in the entropy coding efficiency of these entities. However, the tranches are smaller than the original slices, WPP substreams or tiles and accordingly they may be transmitted with a lower delay, than the un-chopped original entities. According to another aspect combinable with the first aspect, substream marker NAL units are used within a sequence of NAL units of a video bitstream to enable a transport demultiplexer to assign data of slices within NAL units to the corresponding substreams or tiles to be able to, in parallel, serve a multi-threaded decoder with the corresponding substreams or tiles.