Video frame synthesis method based on tensor

A synthesis method and video frame technology, applied in the field of computer vision, can solve the problems of complex network structure, affecting the effect of model training, etc., and achieve the effect of good synthesis effect, high recovery or prediction accuracy

Active Publication Date: 2019-08-16
北京三多堂传媒股份有限公司
View PDF4 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The existing mainstream video frame synthesis work is mainly based on machine learning algorithms or deep learning methods, such as convolutional neural network, generative confrontation network, long-term short-term memory network, etc. The network structure of the video frame synthesis method proposed based on the neural network model is uniform. It is more complicated, the model contains many parameters, the selection of parameters may affect the training effect of the model, and a large amount of data sets are required to train the model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Video frame synthesis method based on tensor
  • Video frame synthesis method based on tensor
  • Video frame synthesis method based on tensor

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0025] During the actual transmission of video, due to the influence of transmission conditions or other interference factors, frame loss often occurs. Finding effective video frame synthesis methods to restore lost frames can improve the quality of video. The video frame prediction can predict future frames through existing frames, predict the future state of the target, and learn the future actions of people or an object in the video. The synthesis of video frames has attracted more and more attention, but the existing neural network-based methods require a large amount of training data, and the existing tensor methods cannot obtain enough information due to the lack of the entire frame, and the recovery accuracy is low, so they cannot Effectively used for video frame compositing.

[0026] In view of the above-mentioned status quo, the present invention proposes a tensor-based video frame synthesis method through research and innovation, see figure 1 , including the follow...

Embodiment 2

[0033] The tensor-based video frame synthesis method is the same as in embodiment 1, and the low-rank tubal-rank tensor complement expression for constructing the video frame synthesis described in step 1 is specifically

[0034] 1.1 For a with n 3 full video frame Randomly zero the frames in the video for recovery, or zero the last few frames of the video for prediction, input the video data after randomly zeroing a few frames in the middle or zero the last few frames Ω represents the original video The set of sequence numbers of existing frames in is the projection tensor on Ω, Indicates the existing video frame data, namely

[0035]

[0036] in for tensor The i-th frontal slice of , i.e., the i-th frame of the full video, has size n 1 ×n 2 , for tensor The ith frontal slice of , 0 is n 1 ×n 2 A matrix of all 0s, indicating that the frame is missing; |Ω| indicates the video The number of existing frames in , then there are n 3 - |Ω| frames need...

Embodiment 3

[0043] The video frame synthesis method based on tensor is the same as embodiment 1-2, the target tensor described in step 2 decomposed into two sizes of with The third-order tensor of - product, specifically

[0044] 2.1 The target tensor in step 1 Decomposed into In the form of , where the third-order tensor third rank tensor Represents two third-rank tensors - product, for any two third-order tensors with defined as [n 1 ] means 1 to n 1 collection of Represents a rank three tensor The tube in row i and column j, Represents a linear transformation, namely the Fourier transform, yes The inverse transformation of , * indicates the multiplication between corresponding elements.

[0045] 2.2 The video frame synthesis is converted into the following form

[0046]

[0047] That is, according to the minimum Frobenius norm, solve the third-order tensor with

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a video frame synthesis method based on tensor, which solves the problems that the traditional low-rank completion video frame synthesis effect is worses and a neural network method needs a large number of training sets. The method comprises the following implementation steps of establishing a tensor-based video frame synthesis model, and synthesizing and converting a videoframe into a complementary tensor; decomposing the target tensor x; solving two decomposed tensors in a Fourier transform domain by adopting an alternating minimization method; and carrying out Fourier inversion on the two tensors, and multiplying the tensors to obtain a target tensor, i.e., recovering the video without the frame. According to the method, the video is regarded as the tensor, thevideo frame is regarded as a front slice of the tensor, the video frame is synthesized and converted into the complementary tensor, and a video synthesis result is obtained through solving in a transform domain. According to the method, more information of the missing frames is obtained, the detail effect is better, a large amount of data training is not needed, and the synthesis precision is higher. The method is used for recovering the frames lost by the video transmission, improving the video quality or predicting a future state of a target in the video.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and further relates to related video frame synthesis, specifically a tensor-based video frame synthesis method, which uses a transformed tensor model to complete video frame synthesis, and is used to restore lost frames in the video and improve Video quality, or used to predict the motion state of objects in the video. Background technique [0002] Video frame synthesis is a fundamental problem in computer vision. For example, in the actual situation of video transmission, due to the influence of transmission conditions or other interference factors, video transmission is often accompanied by frame loss. Finding an effective video frame synthesis method to recover lost frames can improve video quality. However, the synthesis of video frames is challenging due to the complex evolution of pixels across video frames. [0003] In recent years, the problem of synthesis of video frames has re...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04N5/265G06F17/50G06F17/14
CPCH04N5/265G06F17/141G06F30/20
Inventor 孙岳詹克羽刘小洋李颖
Owner 北京三多堂传媒股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap