Video frame interpolation method, device and apparatus

Abstract

The embodiment of the invention discloses a video frame interpolation method, device and apparatus The scheme comprises the steps that a video frame training set is acquired, the video frame trainingset comprises an even number of continuous video frames and a first key frame, and the first key frame is an intermediate frame of the even number of continuous video frames; a pyramid deep learning model is constructed, wherein pyramid deep learning model comprises a plurality of convolutional neural network layers, and the convolutional neural network layers are used for generating intermediateframes with different resolutions; the even number of continuous video frames is input into a pyramid deep learning model to generate a second key frame; the pyramid deep learning model is corrected according to the second key frame and the first key frame; And then video frame interpolation is carried out according to the corrected pyramid deep learning model. According to the method, time-spacedomain information among multiple frames is fully mined, a pyramid refining strategy is adopted, motion information and a shielding area are effectively estimated, and the quality of an intermediate frame is greatly improved.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a video frame interpolation method, device and equipment. Background technique [0002] Video frame interpolation technology (super frame rate technology) and related applications are developing rapidly and have a wide range of application scenarios, such as virtual view synthesis, video frame rate up-conversion, 4K video conversion, and slow-motion video conversion. Since these applications all need to generate video intermediate frames that do not exist originally, how to make the intermediate frames more realistic and reasonable is a key technology in practical applications. [0003] In the prior art, most methods for generating intermediate frames of videos first calculate the optical flow between two adjacent frames, and then interpolate the intermediate frames according to the optical flow. This method relies heavily on the quality of optical flow, otherwise th...

AI Technical Summary

Problems solved by technology

This method relies heavily on the quality of the optical flow, otherwise the generated intermediate frames will produce obvious artifacts

Moreover, the calculation of optical flow requires a complex optimization process, which is very time-consuming.

Emerging methods based on deep learning mainly fall into two categories: the first method directly uses the convolutional neural network to generate the intermediate frame, and uses the real intermediate frame as the supervisory information to train the network, but this type of method usually leads to blurred results; the second type The method uses the convolutional neural network to unsupervisedly obtain the motion information between two frames, and then interpolates the intermediate frame according to the motion information, and also uses the real intermediate frame as the supervisory information to train the network. Although this type of method can effectively avoid blurred results, However, it often relies on accurate motion information estimation, occlusion area estimation, etc., but there is no method for accurately estimating motion information and occlusion area in the prior art

Images