Motion Vector Correction for Video Frame Rate Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Frame rate conversion in video processing often results in undesirable judder artifacts, particularly around the edges of moving objects due to interpolation errors caused by incorrect motion vector estimation, leading to unnatural ringing artifacts or 'halo' effects.
Innovation Solution
A method and apparatus for correcting estimated motion vectors in frame rate conversion by estimating previous and following motion vector fields using three-frame motion estimation, determining reference motion vectors, and adapting weighted factors to improve consistency and accuracy, thereby reducing interpolation errors and artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If motion estimation is performed using two successive frames, then the process is simple and fast, but interpolation error occurs around edges of moving objects causing judder artifacts and halo effects
Solution Approach 1:
The patent applies preliminary action by estimating motion vectors using three frames (previous frame, current frame, and next frame) before performing the actual frame interpolation. This preliminary motion estimation with additional context allows the system to correct motion vectors proactively, preventing interpolation errors before they manifest as artifacts in the output frame.
Solution Approach 2:
The patent implements feedback by using the next frame information to correct motion vectors estimated from previous frames. The motion vector correction unit references motion vectors from the next frame to adjust and refine the motion vectors from the previous frame, creating a feedback loop that continuously improves accuracy and eliminates halo effects around moving objects.
2Ease of manufacture
If frame repetition methods are used for frame rate up-conversion, then the process is simple and efficient, but judder artifacts are generated when objects or backgrounds are moving
Solution Approach 1:
The patent introduces an intermediary approach by creating a motion vector correction unit that mediates between the simple frame repetition process and the complex motion estimation. This intermediary component processes motion vectors using three-frame estimation and correction algorithms, then feeds the corrected vectors back to guide the frame interpolation, effectively eliminating judder artifacts while maintaining efficiency.
3Loss of time
If motion vector correction is performed without considering temporal consistency, then the correction process is fast, but unnatural ringing artifacts and halo effects are formed around moving object edges
Solution Approach 1:
The patent applies continuity of useful action by maintaining temporal consistency in motion vector correction through the three-frame estimation process. The system continuously references motion vectors from both previous and next frames to ensure smooth, consistent corrections across time, preventing discontinuities that would cause ringing artifacts and halo effects around moving objects.
Data Source
AI summary
A motion vector correction apparatus for correcting an estimated motion vector for a pixel located at a position of an interpolated frame between a first and a second frame is provided, including a three-frame motion estimation unit estimating a previous and a following motion vector fields; a motion vector selection unit determining a set of reference motion vectors associated with the first and second frame for the position of the pixel, wherein the reference motion vectors include a first motion vector set as the estimated motion vector and a pair of corresponding motion vectors from the previous and following motion vector fields; and a motion vector correction unit obtaining a measure of consistency between the corresponding motion vectors, adapting a plurality of weighted factors according to the measure of consistency and correcting the estimated motion vector according to the adapted weighted factors and the set of reference motion vectors.