Motion Vector Field Refinement via Local Weighted Model Fitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital image processing techniques, such as frame interpolation, suffer from inaccurate and oscillating motion vectors, leading to artifacts like object/background breakage and ghosting, due to poor motion vector field accuracy.
Innovation Solution
A weighted motion model is applied within a local window to refine motion vectors, using average values and gradients to calculate weights for a refined motion model, which improves accuracy and smoothness by selecting the most suitable motion model for each neighborhood.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If true-motion estimation models (such as 3D recursive analysis) are used to generate motion vector fields, then the motion vectors can be obtained through iterative updating to seek the most accurate motion, but the motion vector field exhibits poor accuracy and oscillation in local neighborhoods
Solution Approach 1:
The patent applies local quality by dividing the image into local neighborhoods and applying motion model fitting independently to each neighborhood. This allows the motion vector field to be smooth and stable within each local region while maintaining accuracy. The weighted motion model is fitted to motion vectors within a local neighborhood window, ensuring local consistency and reducing oscillations that occur when using global iterative updating methods.
2Measurement precision
If motion vectors are iteratively updated to converge to the most accurate motion vector, then measurement precision improves, but this causes oscillation and convergence issues in the motion vector field
Solution Approach 1:
The patent changes the approach from iterative updating of individual motion vectors to a direct parameter fitting method. Instead of iteratively adjusting motion vectors to converge, the patent fits a weighted motion model (such as affine or perspective transform) directly to the motion vectors within a local neighborhood. This parameter-based fitting approach achieves accurate motion estimation without the oscillation and convergence issues associated with iterative updating.
3Ease of manufacture
If a single motion model is used for the entire frame, then processing is simpler, but accuracy and local smoothness of motion vectors deteriorate
Solution Approach 1:
The patent segments the image into multiple local neighborhoods and applies motion model fitting independently to each segment. This segmentation approach allows different motion characteristics to be captured in different regions of the image, improving overall accuracy and local smoothness. Each local neighborhood can have its own fitted motion model, which better represents the local motion patterns compared to a single global motion model.
Data Source
AI summary
A method of producing video data includes receiving, at a processor, a current frame of image data in a stream of frames of image data, dividing a current frame of image data into blocks, identifying a current block and defining a neighborhood of blocks for the current block, generating at least one initial motion vector for each block, using the initial motion vector for current block and an initial motion model to calculate a weight for each initial motion vector in the neighborhood based on a difference between initial motion vector for the current block and the initial motion vector for at least one other block from the current block in the neighborhood and differences in the image data between the current block and the other blocks in the neighborhood, using the weights for each initial motion vector to generate coefficients for a refined motion model, refining the initial motion vector for the current block according to the refined motion model to produce a refined motion vector, using the refined motion vector and the pixels in the stream of frames to produce at least one of adjusted pixels and new pixels, and displaying the at least one of adjusted pixels and new pixels on a display.