Video generation method and device, equipment and storage medium

A video and video data technology, applied in character and pattern recognition, biological neural network models, instruments, etc., can solve problems such as time-consuming

Pending Publication Date: 2022-04-29
深圳市达旦数生科技有限公司
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] In order to solve the technical problem that the generation process is relat...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Video generation method and device, equipment and storage medium
  • Video generation method and device, equipment and storage medium
  • Video generation method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0083] It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. The terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or modules is not necessarily limited to the expressly listed Those steps or modules, but may include other steps or modules that are not clearly listed or...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of software image synthesis, and provides a video generation method and device, equipment and a storage medium. The method comprises the following steps: decomposing each piece of audio data into phoneme data through an automatic speech recognition system, and calculating a posterior probability of each piece of phoneme data to obtain a phoneme posterior probability; extracting a face expression parameter of each frame of image data in the corresponding video data through a 3D face reconstruction technology to obtain an expression feature vector; generating a target expression model from the expression feature vectors and the phoneme posterior probabilities of the video data corresponding to the multiple pieces of audio data through a recurrent neural network; obtaining a to-be-replaced target video; extracting a face three-dimensional reconstruction model of each frame of image data in the corresponding video data through a 3D face reconstruction technology to obtain virtual image data; and inputting the target video to be replaced and the virtual image data into the generative adversarial neural network to obtain a target generation model. And the face synthesis speed is improved.

Description

technical field [0001] The present invention relates to the field of image synthesis, in particular to a video generation method, device, equipment and storage medium. Background technique [0002] Virtual human generation technology is a technology driven by artificial intelligence and uses audio to generate corresponding actions, expressions, language, etc. of a specific image. Virtual humans can be used in various fields such as artificial intelligence (AI) assistants, AI anchors, and AI teachers. The general technical process is to first use Recurrent Neural Network (RNN) to learn the mapping relationship between audio and speaking expressions, and then use expression parameters to control the face synthesis process to synthesize the final natural speaking video. Since the AI ​​virtual human needs to do a lot of interaction with people, if these interactive contents are pre-generated with pre-fixed answers, the effect will be greatly reduced. The first half of the AI ​...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T13/20G06T13/40G06T17/00G06V40/16G06V10/762G06V10/764G06V10/82G06K9/62G06N3/04G10L15/02G10L15/16
CPCG06T13/205G06T13/40G06T17/00G10L15/02G10L15/16G10L2015/025G06N3/044G06N3/045G06F18/23213G06F18/24
Inventor 周艺超李坤胡景强刘鹏飞孙立发钟静华
Owner 深圳市达旦数生科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap