Hierarchical Motion Vector Processing for Artifact Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Motion-compensated frame interpolation (MCFI) using received motion vectors often suffers from artifacts like blockiness and ghost effects due to irregular motions and the inability to accurately represent complex motion fields, especially in resource-limited devices.
Innovation Solution
A hierarchical motion vector processing method that determines reliability levels based on residual energy, merges macroblocks, and iteratively assigns single motion vectors to minimize prediction differences, while accounting for occlusions and using motion vector correlation to refine and smooth motion vectors on finer block sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If direct MCFI uses received motion vectors directly, then frame rate doubling is achieved, but blockiness and ghost effects occur
Solution Approach 1:
The patent segments the motion vector field into multiple reliability levels (high, medium, low) based on residual energy analysis. Different motion vector regions are processed differently according to their reliability, allowing frame rate doubling while reducing artifacts by applying appropriate processing to each segment.
Solution Approach 2:
The patent applies local quality by treating different regions of the motion vector field differently based on their reliability characteristics. High reliability regions use direct interpolation while low reliability regions undergo refinement, creating spatially adaptive processing that reduces artifacts without sacrificing frame rate benefits.
2Reliability
If vector median filters or smaller block sizes are used to remove outliers, then motion smoothing is improved, but irregular motions are distorted and object structure is lost
Solution Approach 1:
The patent applies local quality by analyzing residual energy distribution to identify regions with irregular motions versus smooth motions. Only regions with high residual energy (indicating potential outliers) undergo filtering, while regions with low residual energy maintain their original motion vectors, preserving object structure and avoiding distortion of irregular motions.
Solution Approach 2:
The patent dynamically adjusts the motion vector processing strategy based on residual energy thresholds. The processing intensity and type vary adaptively across different regions, applying strong filtering only where needed while maintaining weak or no processing in smooth regions, thus balancing smoothness and structure preservation.
3Reliability
If object based motion estimation is used to maintain object structure, then interpolation quality is improved, but computational complexity increases
Solution Approach 1:
The patent segments the motion vector field into reliability levels based on residual energy analysis, which is computationally lightweight. This segmentation enables the system to apply complex object-based processing only to high reliability regions while using simpler methods for low reliability regions, reducing overall computational complexity while maintaining structure preservation.
Solution Approach 2:
The patent applies partial action by using object-based motion estimation only where necessary (in high reliability regions with low residual energy) rather than applying it globally. This selective application maintains object structure in critical regions while avoiding the high computational cost of applying object-based methods throughout the entire frame.
4Ease of manufacture
If macroblocks are processed independently, then processing simplicity is maintained, but edge continuity and object structure are not maintained
Solution Approach 1:
The patent merges adjacent macroblocks into larger units for processing, particularly in regions where motion vectors indicate continuous object structures. This merging allows edge continuity to be maintained across macroblock boundaries while preserving the simplicity of block-based processing by using straightforward merging criteria based on motion vector correlation and residual energy.
Data Source
AI summary
A preferred method for hierarchical motion vector processing determines reliability levels of blocks in image data according to residual energy levels. Macroblocks of an image frame are merged according to reliability levels of the motion vectors of blocks. Single motion vectors are selected for merged macroblocks. Motion vectors of blocks merged in the step of merging are iteratively assigned by minimizing the bi-directional prediction difference on successively smaller merged blocks. The reliability levels are preferably determined by measure residual energy of both chrominance and luminance components. In preferred embodiments, motion vector correlation is used to assist the MV reliability classification and the merging and iterative assignment. Refinement and smoothing can be conducted on successively finer block sizes. Additionally, preferred methods account for occlusions by choosing only one of forward or backward prediction for occlusion regions depending upon the class of the occlusion. Results of motion vector classification of the invention can be used in motion compensated frame interpolation and other techniques.