Video generation method combining variational auto-encoder and generative adversarial network

An autoencoder and encoder technology, applied in the field of video generation combining variational autoencoders and generative confrontation networks, can solve problems such as image deformation, insufficient time continuity, and reduced video generation quality, so as to achieve easy training and improved Continuity between frames, the effect of overcoming poor continuity between frames

Active Publication Date: 2019-12-13
ZHEJIANG UNIV
View PDF6 Cites 26 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the existing video generation methods often have problems such as insufficient time continuity between frames in the generated video and image deformation when the input information is insufficient, thereby reducing the quality of video generation

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Video generation method combining variational auto-encoder and generative adversarial network
  • Video generation method combining variational auto-encoder and generative adversarial network
  • Video generation method combining variational auto-encoder and generative adversarial network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0054] Step 1. Take out the handwritten digital picture from the MNIST data set. If the type of the handwritten digital picture taken out is "0, 1, 4, 6, 9", a 16-frame 48×48 pixel video will be formed for the number. The number is in In the first frame, start at any position and move up and down in 16 frames; if the type of handwritten digital picture taken out is "2, 3, 5, 7, 8", a 48×48 pixel of 16 frames will be formed for the number The video, the number starts at any position in the first frame, and moves left and right in 16 frames; make a text description for the moving video of each handwritten number, such as "The digit 0is moving up anddown", "The digit 2is moving up and down", "The digit 2is moving left and right", so that 10 categories of handwritten digital moving videos are obtained, and each category of video has a corresponding text description;

[0055] Step 2, preprocessing the video data set and its text description obtained in step 1, to obtain the "video-...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a video generation method combining a variational auto-encoder and a generative adversarial network. Belonging to the technical field of video generation, the method comprisesthe following steps: a generator of the generative adversarial network does not directly generate a video but generate a series of associated hidden variables, the hidden variables pass through atrained decoder of the variational auto-encoder to generate a series of related images, a discriminator of the generative adversarial network does not directly discriminate the video, but enables the videoto pass through the encoder of the variational auto-encoder to obtain a series of low-dimensional hidden variables and discriminate the hidden variables. According to the method, the video can be generated according to the input description text; the method is advantaged in that a problem of poor inter-frame continuity in video generation is solved, inter-frame continuity of video generation is improved, the training step is divided into two parts of training the variation auto-encoder and training the generative adversarial network based on the trained variation auto-encoder, and training iseasier and more stable.

Description

technical field [0001] The invention belongs to the technical field of video generation, and in particular relates to a video generation method combining a variational autoencoder and a generative confrontation network. Background technique [0002] In recent years, with the wide application of artificial intelligence technology in various industries, the productivity of all walks of life has been greatly improved. For example, in the production of TV programs, video generation technology can greatly reduce human work. In the industry, companies such as NVIDIA have proposed video generation technology based on generative adversarial networks to meet video generation needs in various situations. However, the existing video generation methods often have problems such as insufficient temporal continuity between frames in the generated video and image deformation when the input information is insufficient, thereby reducing the quality of video generation. [0003] Diederik P Ki...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04N21/2343H04N21/4402
CPCH04N21/2343H04N21/4402
Inventor 吴萌李荣鹏赵志峰张宏纲
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap