Scalable Video Buffer Signaling for Lower DPB and Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-loop scalable video coding, the increased memory bandwidth and decoded picture buffer (DPB) size requirements pose challenges, particularly in decoders with limited resources, as they need to handle multiple layers with inter-layer prediction.
Innovation Solution
A multi-loop scalable encoder signals an indication of the highest temporal level of reference data used for inter-layer prediction and the maximum decoded picture buffer (DPB) size needed for decoding enhancement layers, allowing decoders to optimize memory usage and reduce unnecessary decoding of non-target layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multi-loop scalable video coding is used to support multiple layers with inter-layer prediction, then video quality and scalability are improved, but memory bandwidth and decoded picture buffer size requirements increase
Solution Approach 1:
The patent applies local quality by differentiating the treatment of reference layers and target layers. Reference layers maintain full decoding to provide accurate prediction data, while non-target enhancement layers use constrained intra prediction and skip mode to reduce memory usage. This selective quality approach allows the system to maintain high scalability while reducing overall memory bandwidth requirements.
Solution Approach 2:
The patent extracts and removes unnecessary decoding operations for non-target enhancement layers. By identifying which layers are actually needed for the current decoding target, the system can eliminate redundant picture buffer allocations and memory bandwidth consumption for layers that will not be used, thereby reducing the quantity of memory resources required.
2Adaptability or versatility
If multi-loop scalable video coding is used to support multiple layers with inter-layer prediction, then video quality and scalability are improved, but decoded picture buffer size increases
Solution Approach 1:
The patent implements dynamic buffer management where the decoded picture buffer size is adjusted based on the actual decoding target. The buffer allocation is not fixed but dynamically determined by which enhancement layer is the current target, allowing the system to scale buffer usage according to the specific decoding needs rather than allocating maximum buffer size for all possible layers simultaneously.
Solution Approach 2:
The patent uses disposable buffer allocation for non-target layers, where buffer memory is allocated only when needed for reference purposes and immediately released afterward. This approach allows the system to support multiple layers in the scalable structure while using minimal buffer size at any given time, effectively treating buffer resources as temporary rather than permanent allocations.
3Measurement precision
If all layers are fully decoded to support inter-layer prediction, then prediction accuracy is improved, but computational complexity and memory usage increase
Solution Approach 1:
The patent applies local quality by providing full decoding and high prediction accuracy only for reference layers that are actually needed, while using simplified decoding methods for non-target enhancement layers. This selective approach maintains prediction accuracy where it matters most while reducing overall computational complexity and decoder resource requirements.
Solution Approach 2:
The patent uses partial action by decoding only the necessary portions of each layer required for the current target. Instead of fully decoding all layers regardless of need, the system performs partial decoding operations focused specifically on providing the minimum necessary prediction data, thereby reducing computational complexity while maintaining adequate prediction accuracy.
Data Source
AI summary
A method for encoding a video sequence in a scalable video encoder to generate a scalable bitstream is provided that includes encoding the video sequence in a first layer encoder of the scalable video encoder to generate a first sub-bitstream, encoding the video sequence in a second layer encoder of the scalable video encoder to generate a second sub-bitstream, wherein portions of the video sequence being encoded in the second layer encoder are predicted using reference portions of the video sequence encoded in the first layer encoder, combining the first sub-bitstream and the second sub-bitstream to generate the scalable bitstream, and signaling in the scalable bitstream an indication of a maximum decoded picture buffer (DPB) size needed for decoding the second sub-bitstream and the first sub-bitstream when the second sub-bitstream is a target sub-bitstream for decoding.