Unlock instant, AI-driven research and patent intelligence for your innovation.

Model training method, video frame insertion method and corresponding device

A technology for model training and video frames, applied in the field of video processing, which can solve problems such as limiting image reconstruction quality, difficulty in training neural network models, and labeling.

Pending Publication Date: 2021-10-22
MEGVII BEIJINGTECH CO LTD
View PDF7 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, for actual videos, it is difficult to manually annotate optical flow data, which makes it difficult to train the neural network model that such methods rely on, and limits the further improvement of image reconstruction quality.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Model training method, video frame insertion method and corresponding device
  • Model training method, video frame insertion method and corresponding device
  • Model training method, video frame insertion method and corresponding device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

[0051]The term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements but also other elements not expressly listed elements, or also elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

[0052] ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of video processing, and provides a model training method, a video frame insertion method and corresponding devices. The model training method comprises the following steps: acquiring a first video frame, a second video frame and a reference intermediate frame; based on the first video frame and the second video frame, obtaining a first intermediate frame optical flow and a second intermediate frame optical flow by using a first neural network and a second neural network respectively; mapping the first video frame and the second video frame by using the first intermediate frame optical flow, fusing the first mapping video frame and the second mapping video frame obtained by mapping to obtain a first intermediate frame, and similarly obtaining a second intermediate frame; calculating first image loss and optical flow loss, and updating parameters of the first neural network according to the first image loss and the optical flow loss; and calculating second image loss, and updating parameters of the second neural network according to the second image loss. According to the method, a solution for realizing high-quality video frame insertion under the condition of lacking optical flow data annotation is provided.

Description

technical field [0001] The present invention relates to the technical field of video processing, in particular to a model training method, a video frame insertion method and a corresponding device. Background technique [0002] Video frame interpolation is a classic task in video processing, which aims to synthesize smooth intermediate frames from two frames in a video. The application scenarios of video frame insertion include: first, it is used to increase the video frame rate displayed on the device, so that users feel the video is clearer and smoother; second, in video production and editing, it is used to assist in realizing the slow motion effect of the video, or It is used to add intermediate frames between the key frames of animation to reduce the manpower expenditure of animation production; thirdly, it is used for intermediate frame compression of video, or provides auxiliary data for other computer vision tasks. [0003] Among the current video frame interpolatio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04N7/01H04N19/587G06N3/04G06N3/08
CPCH04N7/0135H04N19/587G06N3/084G06N3/045
Inventor 黄哲威胡晨周舒畅
Owner MEGVII BEIJINGTECH CO LTD