Video Frame Interpolation Using Separate Motion Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The mismatch in frames per second (FPS) between image data and the apparatus reproducing it leads to deteriorated image quality, necessitating frame interpolation or synthesis, which existing methods are inefficient.
Innovation Solution
A method and apparatus using neural networks to obtain and process motion information from user control objects in video frames, generating interpolation frames through a mapping table based on user inputs and neural network analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If frame interpolation or synthesis is performed to increase FPS, then video quality is improved, but processing complexity and time consumption increase
Solution Approach 1:
The patent segments the video processing into distinct stages: motion estimation, motion compensation, and frame synthesis. Each stage handles specific aspects of the interpolation process independently, allowing for optimized processing of different video regions and reducing overall computational complexity while maintaining high video quality
Solution Approach 2:
The patent performs preliminary motion estimation and generates motion vectors before the actual frame interpolation process. By pre-processing and preparing motion information in advance, the system reduces the computational burden during the main interpolation stage, thereby decreasing processing time and complexity while preserving video quality
2Manufacturing precision
If frame interpolation is performed to increase FPS, then video quality is improved, but processing time increases
Solution Approach 1:
The patent implements periodic motion estimation and compensation operations at strategically selected frames rather than continuously processing every frame. This periodic approach maintains video quality by performing interpolation only when necessary, while significantly reducing overall processing time and computational resource consumption
Solution Approach 2:
The patent applies motion compensation selectively to specific regions of interest within video frames rather than processing the entire frame uniformly. By focusing computational resources on areas with significant motion or importance, the system achieves high video quality where needed while minimizing processing time for less critical regions
Data Source
AI summary
A method performed by at least one processor for improving video quality, includes: obtaining, from a first frame of a video comprising a plurality of pixels, first motion information regarding a user control object displayed in a display interface, the user control object comprising a first set of pixels included in the plurality of pixels; obtaining, from the first frame of the video by using a first neural network, second motion information from a second set of pixels included in the plurality of pixels, the second set of pixels excluding the first set of pixels; and generating, by using a second neural network with the first motion information and the second motion information as inputs for the second neural network, an interpolation frame between the first frame and a second frame included in the video.