Multi-hypothesis Video Decoding for Block Size Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-Efficiency Video Coding (HEVC) faces challenges in achieving optimal coding efficiency due to limitations in inter prediction modes, particularly in Skip and Merge modes, where motion inference methods rely on spatial or temporal neighboring blocks, which can lead to suboptimal performance in certain scenarios.
Innovation Solution
The implementation of a Multi-hypothesis (MH) mode in video decoders, which generates a second prediction using intra or inter prediction modes in addition to the primary prediction, allowing for combined prediction and improved motion compensation, especially for blocks larger than a specified threshold.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If traditional inter prediction modes (Skip/Merge) are used with motion inference from neighboring blocks, then device complexity is reduced, but coding efficiency deteriorates due to suboptimal performance in certain scenarios
Solution Approach 1:
The prediction process is segmented into multiple hypotheses (first hypothesis from inter prediction, second hypothesis from intra prediction), where each hypothesis is evaluated independently and then combined. This segmentation allows the system to explore multiple prediction paths without overwhelming complexity, resolving the contradiction by dividing the prediction task into manageable segments that can be selectively applied.
Solution Approach 2:
The patent introduces a block size parameter threshold that dynamically changes the prediction approach. For blocks exceeding the threshold, multi-hypothesis mode is enabled to improve coding efficiency, while for smaller blocks, traditional modes are maintained to preserve low complexity. This parameter-based adaptation resolves the contradiction by adjusting the prediction strategy based on block characteristics.
2Productivity
If multi-hypothesis mode is enabled for all block sizes, then coding efficiency is improved, but device complexity increases due to additional prediction calculations
Solution Approach 1:
The patent applies multi-hypothesis mode selectively based on local block characteristics (size threshold) rather than uniformly across all blocks. This local quality approach ensures that the complex multi-hypothesis processing is applied only where it provides significant benefit (larger blocks), while smaller blocks continue to use simpler prediction modes, thus improving coding efficiency without proportionally increasing overall device complexity.
Solution Approach 2:
The patent implements partial action by enabling multi-hypothesis mode for only those blocks that exceed the size threshold, rather than applying it to all blocks. This partial application of the complex prediction method achieves most of the potential coding efficiency gains while avoiding the full complexity overhead, effectively resolving the contradiction between improved productivity and increased device complexity.
3Ease of operation
If a fixed threshold is used to enable multi-hypothesis mode, then ease of operation is improved, but adaptability deteriorates for different block size scenarios
Solution Approach 1:
The patent introduces dynamic adaptability by allowing the threshold parameter to be adjusted based on picture type (I, P, B slices) and block characteristics. Rather than using a completely fixed threshold, the system dynamically adapts the threshold value to suit different scanning directions and picture types, thereby maintaining ease of operation through simple threshold comparison while significantly improving adaptability to various block size scenarios.
Solution Approach 2:
The patent employs parameter changes by adjusting the threshold value based on different picture types and scanning directions. For example, different thresholds are used for I slices, P slices, and B slices, and for different scanning directions (horizontal vs. vertical). This parameter adaptation resolves the contradiction by allowing the system to maintain simple threshold-based operation while adapting to diverse block size scenarios through parameter variation.
Data Source
AI summary
A video decoder that decodes a current block of pixels by using multi-hypothesis combined prediction mode is provided. The video decoder generates a first prediction of the current block based on an inter prediction mode. The video decoder enables the combined prediction mode for the current block based on a block size of the current block determined according to a width and a height of the current block. The combined prediction mode is disabled when the width of or the height of the current block is greater than a threshold length. When the combined prediction mode is enabled, the video decoder generates a second prediction based on an intra prediction mode that is inferred to be a planar mode, and subsequently a combined prediction for the current block based on the first prediction and the second prediction. The video decoder reconstructs the current block by using the combined prediction.