Motion Compensated Frame Rate Conversion Using Adaptive Blending
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing frame rate conversion methods, such as simple linear processing, produce noticeable visual artifacts like motion judder and blur for video sources with moderate to fast motion, and motion compensated frame rate conversion (MC-FRC) faces challenges in accurate motion estimation and high computational complexity.
Innovation Solution
A novel motion estimation and motion vector processing stage that generates a motion field with motion vectors describing object movement, followed by a motion compensated interpolation stage using adaptively blended predictions, enhances the reliability of motion estimation and reduces complexity by utilizing spatial and temporal correlations and filtering aberrational vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If motion compensated frame rate conversion is used to improve visual quality, then visual quality is improved, but computational complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the frame rate conversion process into distinct stages: motion estimation, motion vector processing, and frame interpolation. This modular approach allows each stage to be optimized independently, reducing overall computational complexity while maintaining visual quality.
Solution Approach 2:
The patent implements a fallback scheme that uses simpler methods (such as temporal averaging) when motion estimation fails or is unavailable, rather than always attempting full motion compensated interpolation. This partial application of the complex method reduces computational load while maintaining acceptable visual quality.
2Manufacturing precision
If motion estimation is performed accurately to reduce visual artifacts, then visual quality is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary motion estimation between original frames before generating interpolated frames. By pre-computing motion vectors and identifying reliable motion regions in advance, the system avoids repeated complex calculations during frame interpolation, reducing overall computational complexity.
Solution Approach 2:
The patent replaces complex iterative motion estimation algorithms with a simplified motion vector processing approach that uses temporal and spatial correlation. This substitution maintains accurate motion tracking while significantly reducing computational requirements.
3Device complexity
If simple linear processing methods are used to reduce computational complexity, then computational complexity is reduced, but visual quality deteriorates due to motion artifacts
Solution Approach 1:
The patent dynamically adapts the frame generation method based on motion characteristics. For static or slow-moving regions, simple temporal averaging is used; for fast-moving regions with reliable motion estimation, motion compensated interpolation is applied. This dynamic adaptation maintains visual quality while reducing computational complexity compared to always using MC-FRC.
Solution Approach 2:
The patent changes the interpolation parameter (blend ratio between temporal average and motion compensated prediction) based on motion magnitude and reliability. By adjusting this parameter dynamically, the system achieves good visual quality with reduced computational complexity.
4Reliability
If motion vectors are processed to remove aberrations and improve reliability, then reliability is improved, but computational complexity increases
Solution Approach 1:
The patent implements a feedback mechanism where motion vectors are evaluated for consistency with neighboring vectors and temporal patterns. Inconsistent vectors are identified and corrected using neighboring vector information, improving reliability through a relatively simple consistency check rather than complex iterative optimization.
Data Source
AI summary
Systems and methods of motion compensated frame rate conversion are described herein. These systems and methods convert an input video sequence at a first frame rate to an output video sequence at a second frame rate through a novel motion estimation and motion vector processing stage that produces a motion field having a plurality of motion vectors that describe the movement of objects between input video frames from the perspective of an interpolated video frame. A subsequent motion compensated interpolation stage then constructs the interpolated video frame using an adaptively blended combination of a motion compensated prediction and a temporal average prediction of the pixel values from the input video frames. Motion estimation in these systems and methods is enhanced by utilizing spatial and temporal biasing on the predictions of moving objects between and within the input frames, and also by removing aberrational motion vectors from the motion field through a hierarchy of motion vector processing blocks.