Sub-picture CPB Buffering for Low-Delay Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding methods face challenges in achieving reduced codec delay, particularly in conversational applications where low end-to-end delay is crucial, and in implementing sub-picture based coded picture buffer (CPB) behavior due to requirements such as identical bit allocation across treeblock groups and difficulties in splitting and sending coded bits separately.
Innovation Solution
A generic design for sub-picture based CPB behavior is introduced, allowing each sub-picture to have continuous coding blocks, variable bit allocation, and signaling of CPB removal times, enabling byte alignment and independent operation of sub-pictures within a slice or frame.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If sub-picture based CPB behavior is implemented with identical bit allocation across treeblock groups, then low-delay video transmission is achieved, but device complexity and ease of manufacture worsen due to requirements for splitting and sending coded bits separately
Solution Approach 1:
The patent divides a picture into multiple decoding units (sub-pictures, slices, or other partitions) that can be independently decoded and removed from the CPB. Each decoding unit has its own CPB removal delay parameter, allowing selective removal without requiring the entire picture to be processed uniformly. This segmentation enables low-delay operation by allowing early removal of independently decodable units while avoiding the complexity of splitting coded bits across multiple transmission channels.
2Loss of time
If sub-picture based CPB behavior is implemented with identical bit allocation, then interoperable low-delay transmission is achieved, but ease of operation worsens due to restrictions on bit allocation flexibility
Solution Approach 1:
The patent allows different CPB removal delay parameters to be assigned to different decoding units within the same picture. This enables local optimization where each decoding unit can have its own timing characteristics based on its specific decoding requirements, content complexity, and priority. The encoder can signal different cpb_removal_delay values for different decoding units, providing fine-grained control over decoding timing without requiring identical bit allocation across all units.
3Loss of time
If decoding units are removed from CPB at different times, then codec delay is reduced, but reliability worsens due to potential desynchronization between encoder and decoder
Solution Approach 1:
The patent employs explicit signaling of CPB removal delay parameters in the bitstream for each decoding unit. The encoder signals cpb_removal_delay values that indicate when each decoding unit should be removed from the CPB at the decoder. This feedback mechanism ensures that both encoder and decoder maintain synchronized understanding of the decoding timeline, preventing desynchronization even when different decoding units are removed at different times. The signaled parameters act as a coordination protocol between encoder and decoder.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
As one example, a method of coding video data includes storing one or more decoding units of video data in a picture buffer. The method further includes obtaining a respective buffer removal time for the one or more decoding units, wherein obtaining the respective buffer removal time comprises receiving a respective signaled value indicative of the respective buffer removal time for at least one of the decoding units. The method further includes removing the decoding units from the picture buffer in accordance with the obtained buffer removal time for each of the decoding units. The method further includes coding video data corresponding to the removed decoding units, wherein coding the video data comprises decoding the at least one of the decoding units.