Affine Block Prediction Refinement for Efficient Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in efficiently predicting motion vectors for affine blocks, leading to suboptimal compression and increased data requirements, particularly in high-resolution video applications.
Innovation Solution
The implementation of a syntax element to enable affine prediction refinement (PROF) with spatial gradients generated using a gradient filter on reference samples, allowing for improved inter prediction of sub-blocks within affine coded blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If motion estimation and compensation processes are used to reduce temporal redundancy, then video quality is improved, but encoding complexity and processing time increase significantly
Solution Approach 1:
The patent applies preliminary action by performing motion estimation on only a subset of blocks (e.g., every other block or selected important blocks) rather than all blocks. This preliminary partial processing reduces the overall computational complexity while still providing sufficient motion compensation information to maintain acceptable video quality. The selected blocks are processed in advance to establish motion vectors that can be reused or interpolated for neighboring blocks.
Solution Approach 2:
The patent implements partial action by selectively applying motion estimation and compensation only to certain blocks that contribute most to video quality, rather than processing every block equally. This partial processing approach focuses computational resources on critical areas while reducing overall complexity. The patent may also use simplified motion compensation for less important blocks, applying just enough processing to maintain acceptable quality without excessive computation.
2Productivity
If intra-prediction mode is used to encode blocks, then coding efficiency is improved, but the number of reference samples required increases processing complexity
Solution Approach 1:
The patent applies segmentation by dividing the prediction process into different stages or regions. It separates the identification of reference samples from the actual prediction computation. The patent may also segment the processing by handling different block types or positions differently, applying simplified reference sample identification for certain regions while using more complex methods only where necessary, thus reducing overall processing complexity while maintaining coding efficiency.
3Measurement precision
If more reference samples are used for prediction, then prediction accuracy is improved, but memory access requirements and processing time increase
Solution Approach 1:
The patent applies preliminary action by pre-identifying and storing reference sample positions and values in a prepared data structure before the actual prediction process. This preliminary organization of reference samples reduces the time required during the prediction computation phase, as the data is already arranged for efficient access. The patent may also pre-filter or pre-select the most relevant reference samples to reduce the number that need to be processed.
Solution Approach 2:
The patent implements partial action by using a limited set of reference samples that provide sufficient prediction accuracy for most cases, rather than always using all available reference samples. This partial usage reduces memory access requirements and processing time while maintaining acceptable prediction accuracy. The patent may selectively increase the number of reference samples only for specific block types or difficult-to-predict regions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus for video decoding includes processing circuitry. The circuitry can be configured to determine whether to apply the PROF to an affine coded block, and responsive to a determination to apply the PROF to the affine coded block, perform a PROF process. For example, during the PROF process, a prediction sample I(i,j) at a sample location (i,j) in the affine coded block can be generated. Spatial gradients g_x (i,j) and g_y (i,j) at the sample location (i,j) in the affine coded block can be generate. A prediction refinement ΔI(i,j) based on the spatial gradients g_x (i,j) and g_y (i,j) can be generated. The prediction refinement ΔI(i,j) can added to the prediction sample I(i,j) to generate a refined prediction sample. The PROF for affine prediction can be enabled according to a syntax element received at the decoder, or can be enabled by default.