Affine Prediction Refinement Optical Flow Video Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding techniques face challenges in efficiently reducing redundancy in video signals, particularly in achieving high compression ratios while maintaining acceptable distortion levels, especially in applications requiring low bandwidth and storage space.
Innovation Solution
The method involves performing sub-block based affine motion compensation and prediction refinement using optical flow processes within a video decoder, which generates a final prediction for each pixel in the current block by determining spatial gradients and incorporating constraints from the coded video bitstream.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If lossy compression is employed to reduce bandwidth and storage requirements, then compression ratio is improved, but distortion between original and reconstructed signals increases
Solution Approach 1:
The current block is divided into multiple sub-blocks, and each sub-block is processed independently with its own motion compensation parameters. This segmentation allows for more precise local prediction, reducing the distortion introduced by compression while maintaining high compression ratios through efficient redundancy removal.
Solution Approach 2:
The patent applies optical flow-based parameter refinement to motion vectors and motion compensation parameters. By continuously optimizing these parameters using gradient descent and energy minimization, the system achieves better prediction accuracy, thereby reducing distortion while maintaining compression efficiency.
2Quantity of substance
If motion compensation with motion vectors is used to compress video signals, then redundancy is reduced and compression is achieved, but prediction accuracy deteriorates leading to higher distortion
Solution Approach 1:
The patent implements an iterative optimization process where prediction errors are calculated and used to refine motion parameters through optical flow analysis. This feedback loop continuously improves prediction accuracy by adjusting motion vectors and compensation parameters based on the actual prediction error, thereby reducing distortion while maintaining compression.
Solution Approach 2:
The system performs preliminary motion compensation using initial motion vectors, then refines these parameters through optical flow analysis before final reconstruction. This preliminary action followed by refinement ensures that the initial compression achieves good redundancy reduction while the subsequent refinement minimizes prediction errors and distortion.
3Productivity
If simple motion compensation is applied to reduce computational complexity, then processing speed is improved, but compression efficiency deteriorates requiring higher bandwidth
Solution Approach 1:
By dividing the block into sub-blocks and applying motion compensation independently to each, the system achieves better local prediction accuracy without proportionally increasing overall computational complexity. The segmented approach allows parallel processing while reducing the residual data that needs to be transmitted, thereby improving compression efficiency without sacrificing processing speed.
Solution Approach 2:
The patent applies different motion compensation strategies to different sub-blocks based on local characteristics. Each sub-block receives customized motion parameters optimized for its specific content, improving compression efficiency locally. This local quality approach ensures better overall compression without requiring excessive computational resources for uniform high-quality processing throughout.
Data Source
AI summary
A method of video decoding performed in a video decoder includes receiving a coded video bitstream including a current block that is divided into a plurality of sub-blocks. Sub-block based affine motion compensation is performed on the current block to generate a sub-block prediction for each pixel in each sub-block of the current block. One or more spatial gradients are determined for each sub-block prediction. For each sub-block prediction, prediction refinement with optical flow process is performed using the respective determined one or more spatial gradients and at least one constraint included in the coded video bitstream. For each sub-block prediction, an output of the respective prediction refinement is added to the respective sub-block prediction to generate a final prediction for each pixel in each sub-block of the current block.