Lightweight video interpolation method based on a feature level optical flow

An interpolation method, a lightweight technology, applied in the direction of TV, color TV, color TV components, etc., can solve problems such as poor practicability, and achieve the effect of improving accuracy and speed

Active Publication Date: 2019-05-14
NORTHWESTERN POLYTECHNICAL UNIV
View PDF5 Cites 22 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In order to overcome the disadvantage of poor practicability of existing lightweight video interpolation me

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Lightweight video interpolation method based on a feature level optical flow
  • Lightweight video interpolation method based on a feature level optical flow
  • Lightweight video interpolation method based on a feature level optical flow

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] refer to Figure 1-2 . The specific steps of the light-weight video interpolation method based on feature-level optical flow of the present invention are as follows:

[0038] Step 1. Any two frames of images I in a given video 1 ,I 2 ∈R 3×H×W , for scale transformation of 1 / 2 and 1 / 4, use Represents the transformed image of the kth image at the sth scale, and s=0 represents the original image.

[0039] Step 2. For two frames of images at scale s Perform feature-level optical flow calculations. Firstly, two steps of convolution operation are performed separately,

[0040]

[0041]

[0042] Among them, C2D 1 (·),C2D 2 ( ) represents a two-dimensional convolution operation. Get three levels of feature maps (F s 11 , F s 12 ) and (F s 21 , F s 22 ). Merge the two feature maps of each level in two-way timing,

[0043]

[0044]

[0045]Among them, l=0,1,2, CAT(·,·) indicates the merging operation of two feature maps in sequence, H s ,W s ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a lightweight video interpolation method based on a feature level optical flow, which is used for solving the technical problem that the existing lightweight video interpolation method is poor in practicability. According to the technical scheme, firstly, multi-scale transformation is carried out on two continuous frames of images in a given video, and an optical flow estimation module at the feature level is adopted to calculate a forward optical flow and a reverse optical flow between the two frames under the current scale; warp transformation on the two images is performed in time sequence according to the forward optical flow and the reverse optical flow respectively to obtain two interpolation images; the interpolated images are combined to obtain a four-dimensional tensor, and the tensor is processed by utilizing three-dimensional convolution to obtain an interpolated image under the scale; And weighted average on the images are carried out with differentscales to obtain a final interpolation image. According to the invention, the video interpolation is carried out by using the optical flow of the feature level and the multi-scale fusion technology, and the precision and speed of the video interpolation are improved. A 1.03 MB network model is used for obtaining the average peak signal-to-noise ratio of 32.439 and the structural similarity of 0.886.

Description

technical field [0001] The invention relates to a lightweight video interpolation method, in particular to a lightweight video interpolation method based on feature level optical flow. Background technique [0002] As a classic video processing and computer vision task, video interpolation generates smooth transitions between two consecutive frames. Given two consecutive images, video interpolation synthesizes realistic intermediate frames to achieve the purpose of compensating motion information and enriching changing details. The generated pictures and the given frames form a continuous and consistent video segment in time. [0003] Video interpolation has been applied in several fields, such as video frame rate conversion, virtual view synthesis, video temporal super-resolution, etc. Different video playback standards require different video frame rates. Therefore, low frame rate video must be converted when played at a higher standard. Given images of adjacent viewpo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04N7/01H04N5/14G06T7/269
Inventor 袁媛王琦李昊鹏
Owner NORTHWESTERN POLYTECHNICAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap