Sub-PU Motion Vector Prediction in HEVC
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies, such as the High Efficiency Video Coding (HEVC) standard, face limitations in motion prediction refinement and deblocking filtering, particularly in the 3D extension for multiview video plus depth format, where sub-prediction units (PUs) are not adequately refined and deblocking filters do not account for sub-PU boundaries.
Innovation Solution
The implementation of advanced temporal motion vector prediction (TMVP) modes that allow for two-stage motion vector determination and refinement for sub-PUs, along with the storage of separate motion information for each sub-PU, and the conversion of sub-PUs to a deblocking-friendly structure to enable effective filtering across sub-PU boundaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sub-PUs are used for motion prediction in HEVC, then motion prediction accuracy is improved, but device complexity increases due to additional motion information storage and processing
Solution Approach 1:
The prediction unit is divided into multiple sub-PUs, allowing independent motion vector determination for each sub-PU. This segmentation enables more precise motion prediction by capturing local motion variations within the PU, while the systematic two-stage process manages the complexity of handling multiple sub-PUs.
Solution Approach 2:
Motion information for sub-PUs is determined and stored in advance during the encoding process. This preliminary action allows the decoder to efficiently retrieve and use pre-computed sub-PU motion information without performing complex calculations, thereby improving prediction accuracy while managing device complexity.
2Productivity
If conventional deblocking filters are applied, then filtering speed is maintained, but filtering effectiveness deteriorates at sub-PU boundaries
Solution Approach 1:
The deblocking filter is modified to apply different filtering strategies at sub-PU boundaries compared to regular PU boundaries. This local quality approach recognizes that sub-PU boundaries have unique characteristics requiring specialized filtering, thereby improving filtering effectiveness at these critical locations while maintaining overall processing efficiency.
Solution Approach 2:
Virtual sub-PU boundaries are introduced as intermediary structures to guide the deblocking filter. These virtual boundaries serve as mediators that enable the filter to systematically identify and process actual sub-PU boundaries, improving filtering effectiveness without significantly increasing computational complexity.
3Measurement precision
If two-stage TMVP mode is implemented, then motion vector refinement is improved, but encoding complexity increases
Solution Approach 1:
The motion vector determination process is divided into two stages: first determining motion vectors for the entire PU, then refining them for each sub-PU. This segmentation of the encoding process allows systematic refinement of motion vectors with improved precision while managing encoding complexity through a structured approach.
Solution Approach 2:
Coarse motion vectors are determined in advance for the entire PU before sub-PU level refinement. This preliminary action provides a foundation for the second stage, reducing the search space and computational requirements for fine-tuning motion vectors at the sub-PU level, thereby balancing refinement quality with encoding complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques are described for sub-prediction unit (PU) based motion prediction for video coding in HEVC and 3D-HEVC. In one example, the techniques include an advanced temporal motion vector prediction (TMVP) mode to predict sub-PUs of a PU in single layer coding for which motion vector refinement may be allowed. The advanced TMVP mode includes determining motion vectors for the PU in at least two stages to derive motion information for the PU that includes different motion vectors and reference indices for each of the sub-PUs of the PU. In another example, the techniques include storing separate motion information derived for each sub-PU of a current PU predicted using a sub-PU backward view synthesis prediction (BVSP) mode even after motion compensation is performed. The additional motion information stored for the current PU may be used to predict subsequent PUs for which the current PU is a neighboring block.