Gradient-Based Motion Vector Generation for Image Rate Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image rate conversion methods, such as sample and hold upconversions and motion compensation techniques like block matching, phase plane correlation, and optical flow, are expensive and inaccurate, leading to motion judder artifacts when upconverting image sequences to higher frame rates.
Innovation Solution
A single-pass, gradient-based image rate conversion method using a mipmap generator, a single-pass gradient-based motion vector generator, and an image interpolator to create an interpolated image by generating motion trajectories and interpolating pixel values, reducing the need for expensive recursive algorithms and improving motion accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sample and hold upconversion is used, then image rate conversion is achieved, but motion judder artifacts are created
Solution Approach 1:
The patent performs motion estimation in advance by analyzing motion vectors between source frames before generating the interpolated frame. This preliminary motion analysis allows the system to compensate for motion effects during the interpolation process, preventing motion judder artifacts from appearing in the final output.
Solution Approach 2:
The patent uses motion vectors derived from comparing source frames as feedback information to guide the interpolation process. The motion compensation mechanism adjusts the interpolation based on the detected motion patterns, creating a feedback loop that adapts the frame generation to actual motion conditions, thereby eliminating judder artifacts.
2Reliability
If block matching method is used, then motion compensation is achieved, but computational expense and recursive processing are required
Solution Approach 1:
Instead of performing exhaustive block matching across the entire image, the patent uses a simplified approach by sampling motion vectors at key locations and using them to guide the interpolation. This partial action approach achieves sufficient motion compensation without the computational burden of complete block matching, reducing complexity while maintaining reliability.
Solution Approach 2:
The patent replaces the complex mechanical block matching algorithm with a gradient-based motion estimation approach that uses mathematical gradients and motion vectors. This substitution eliminates the need for recursive processing and expensive integer pixel operations, reducing computational cost while maintaining motion compensation accuracy.
3Measurement precision
If phase plane correlation method is used, then motion detection is achieved, but Fourier transforms are required which are computationally expensive
Solution Approach 1:
The patent replaces the Fourier transform-based phase plane correlation method with a direct gradient-based motion estimation approach. This substitution eliminates the need for computationally intensive Fourier transforms while maintaining motion detection accuracy, significantly reducing the energy required for processing.
Solution Approach 2:
The patent changes the mathematical parameters used for motion detection from frequency-domain Fourier transforms to spatial-domain gradient calculations. This parameter change allows motion detection to be performed more efficiently using simpler arithmetic operations, reducing computational energy consumption while preserving measurement precision.
4Measurement precision
If optical flow method is used, then motion vector calculation is achieved, but recursive algorithms are required which are expensive
Solution Approach 1:
The patent replaces the recursive optical flow algorithms with a non-recursive gradient-based motion estimation method. This substitution eliminates the need for iterative refinement and complex recursive processing, reducing algorithmic complexity while maintaining motion vector calculation accuracy through direct mathematical computation.
Data Source
AI summary
A mipmap generator generates pairs of mipmaps that are each of a lower resolution that its respective source image. A single-pass, gradient-based motion vector generator generates an image motion vector map having values that represent the motion trajectories for pixels in the first and second source images. An image interpolator generates an interpolated image based on the source images and the image motion vector map. A motion detector generates a motion factor map based on a pair of mipmaps from those generated by the mipmap generator that represents a detected degree of motion between the first and second source images. The blending module generates a blended, upconverted new image using the motion factor map, the interpolated image and one of the first and second motion maps.