Decoded Picture Buffer Partial Output for Low-Latency Video Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards, such as the HEVC HRD model, assume instantaneous decoding, which does not account for the time required for decoding processes, leading to limitations in accessing decoded data before the entire picture is fully decoded, particularly in applications requiring fast access to partial video data.
Innovation Solution
The decoder enables partial output of decoded picture data before the entire picture is decoded by allowing copying of decoded units from the decoded picture buffer to a data sink while other units are still being processed, enabling applications to access and process partial picture data during decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the decoder waits for the entire picture to be decoded before outputting data, then decoding accuracy is maintained, but access time to decoded data increases
Solution Approach 1:
The picture is divided into multiple decoding units (DUs) that can be decoded and output independently. The DPB is segmented to store individual DUs, allowing selective output of decoded portions without waiting for the complete picture decoding process to finish.
Solution Approach 2:
Decoded units are prepared and stored in the DPB in advance before the entire picture decoding is complete. Applications can access these preliminarily decoded units immediately when they become available, rather than waiting for full picture decoding to finish.
2Productivity
If the decoder outputs the entire picture only after complete decoding, then data consistency is ensured, but processing efficiency decreases
Solution Approach 1:
The decoding output is segmented into individual decoding units rather than requiring complete picture output. Each DU can be independently transferred to the data sink, enabling incremental processing and improving overall system throughput.
Solution Approach 2:
The decoding process and output process run continuously and independently. While decoding continues on remaining units, already decoded units are immediately output and processed, eliminating idle time and maintaining continuous useful action throughout the system.
3Loss of time
If all decoding units are decoded before application access, then data completeness is guaranteed, but access delay increases
Solution Approach 1:
Instead of requiring complete picture decoding before output, the system performs partial decoding and outputs the decoded portion. Applications receive and process available data incrementally, accepting that not all data may be complete but gaining immediate access to what is available.
Solution Approach 2:
The decoded picture buffer serves as an intermediary structure that decouples the decoding process from the output process. It stores decoded units temporarily, allowing applications to access data as it becomes available without blocking on complete picture decoding.
Data Source
AI summary
A method of processing video data of a picture is described, the method including: allocating memory for a decoded picture in a decoded picture buffer, DPB, the decoded picture comprising pixels representing video data; receiving a bitstream comprising decoding units, DUs, and storing the DUs in a coded picture buffer, CRB, the DUs representing a coded picture that needs to be decoded into the decoded picture, each of the DUs representing a coded block of pixels; determining if, during decoding of the coded picture, at least one partial output can be performed, the at least one partial output including copying the one or more decoded DUs from the DPB to a data sink, while one or more DUs of the coded picture are not yet decoded and removed the CPB, the one or more decoded DUs representing a part of the decoded picture; and, or performing the at least one partial output if the processor determines that the at least one partial output can be performed, the performing including marking the one or more decoded DUs stored in the DPB as being ready for partial output, the marking signaling the decoder apparatus not to remove the one or more decoded DUs from the DPB; and, copying the one or more marked decoded DUs from the DPB to the data sink without removing the one or more decoded DU from the DPB.