Frame Interpolation Using Six Affine Parameters for 3D Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing frame rate up-conversion techniques, such as frame duplication and motion compensation, fail to accurately interpolate frames, especially in complex scenes with 3D camera motion and zooming, leading to blurred images and computational complexity.
Innovation Solution
A method that estimates and interpolates six affine parameters to represent relative motion between frames, decomposing them into 2D motion parameters and using these to locate sub-pixel accurate positions and intensities in interpolated frames, effectively addressing the limitations of prior art by accounting for true 3D camera motion and relative object-camera motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If motion compensation based approaches are used to interpolate frames, then image quality is improved, but computational complexity and implementation complexity increase
Solution Approach 1:
The patent changes the parameter representation from full motion vectors to a compact parametric model with only 6 parameters (3 for camera motion, 3 for object motion). This parameter reduction maintains image quality by accurately modeling the essential motion components while significantly reducing computational complexity for frame interpolation
Solution Approach 2:
The patent applies different motion models to different parts of the scene: a global parametric model for camera-induced motion and local motion fields for object motion. This localized approach improves image quality in complex scenes while keeping the overall computational complexity manageable through selective application of complex models
2Device complexity
If linear interpolation of motion vectors is used, then computational complexity is reduced, but accuracy deteriorates in cases of large camera-induced motion
Solution Approach 1:
The patent replaces linear motion vector interpolation with a parametric model that explicitly accounts for camera-induced motion through 3 parameters (translation, rotation, zoom). This parameter transformation enables accurate motion estimation even with large camera movements while maintaining reasonable computational complexity through the compact parameter representation
Solution Approach 2:
The patent transitions from 2D motion vector interpolation to a 3D parametric model that includes camera position and orientation parameters. This dimensional expansion allows the model to capture the true nature of camera-induced motion in three-dimensional space, improving accuracy without proportionally increasing computational complexity
3Manufacturing precision
If affine transformation models are used to represent motion, then 2D motion is well represented, but 3D camera motion and zooming are not accurately captured
Solution Approach 1:
The patent extends the affine transformation by adding 3 parameters specifically for camera-induced motion (translation, rotation, zoom) to the traditional 2D affine model. This parameter expansion transforms the model into a 3D-aware parametric model that can accurately represent both 2D object motion and 3D camera motion, achieving versatile motion representation
Solution Approach 2:
The patent segments the motion representation into two independent components: camera-induced motion parameters (affecting the entire scene) and object motion parameters (affecting specific regions). This segmentation allows the model to independently handle 3D camera motion and 2D object motion, improving both accuracy and adaptability
Data Source
AI summary
A method and an apparatus are disclosed for increasing the frame rate of an input video signal by interpolating video frames between original video frames of the input video signal and inserting interpolated video frames between original video frames of the input video signal to produce an output video signal having a higher frame rate than the input signal.