Video Frame Interpolation Using Quantized Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video frame rate increasing technologies do not achieve ideal results in enhancing the playing effect of high frame rate videos.
Innovation Solution
A video frame adjustment method that inserts an intermediate frame between two adjacent video frames using a quantization neural network to determine optical flows and affine transformations, ensuring smooth playback by estimating motion and predicting a masked image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional video frame rate increasing technology is used, then the frame rate is increased, but the playing effect and viewing experience are not ideal
Solution Approach 1:
The patent performs preliminary actions by calculating optical flow between adjacent video frames before generating the intermediate frame. The quantization neural network pre-processes the optical flow information to capture motion characteristics, which are then used to guide the intermediate frame generation, ensuring high quality playback at the target frame rate.
Solution Approach 2:
The patent introduces an intermediate frame as a mediator between two adjacent video frames. This intermediate frame acts as a bridge that smooths the transition between frames, improving playback quality. The quantization neural network serves as another intermediary that processes optical flow information to enable the intermediate frame to be generated with accurate motion compensation.
2Speed
If frame insertion is performed to increase frame rate, then playback smoothness is improved, but processing complexity increases
Solution Approach 1:
The patent segments the frame rate conversion process into distinct stages: optical flow calculation between adjacent frames, quantization neural network processing of the optical flow, and intermediate frame generation. This segmentation allows each component to be optimized independently and facilitates real-time processing on mobile devices with limited computational resources.
Solution Approach 2:
The patent changes the parameter representation by using quantized optical flow values instead of full-precision optical flow data. This parameter transformation reduces the computational complexity and memory requirements while maintaining the essential motion information needed for accurate intermediate frame generation.
3Reliability
If complex scenarios are processed to maintain quality, then viewing experience is improved, but computational burden increases
Solution Approach 1:
The patent applies partial action by focusing computational resources on the most critical aspects of frame generation: optical flow calculation and quantization neural network processing. Rather than performing exhaustive processing on all possible frame details, the method concentrates on capturing the essential motion characteristics, which provides sufficient quality improvement for complex scenarios while controlling computational burden.
Data Source
AI summary
A video frame adjustment method, an electronic device and a non-transient computer-readable storage medium are provided. The method includes: determining, based on a first video frame and a second video frame in an initial video, a first optical flow from the first video frame to the second video frame and a second optical flow from the second video frame to the first video frame through a quantization neural network; determining a third optical flow from an intermediate frame to the first video frame and a fourth optical flow from the intermediate frame to the second video frame; determining the intermediate frame according to the first video frame, the second video frame, the third optical flow and the fourth optical flow; inserting, in the initial video, the intermediate frame between the first video frame and the second video frame to obtain a target video.