Video generation method and device, storage medium and electronic equipment

Abstract

The invention relates to a video generation method and device, a storage medium and electronic equipment. The method comprises the steps of obtaining a to-be-processed audio; converting the to-be-processed audio into a to-be-processed audio feature vector; inputting the to-be-processed audio feature vector into a video generation model to obtain a vocal organ video feature sequence; generating a vocal organ action video corresponding to the to-be-processed audio according to the vocal organ video feature sequence, wherein the video generation model is trained by: constructing model training data according to a sample audio feature vector converted from a sample audio and a sample vocal organ video feature sequence of the sample vocal organ action video corresponding to the sample audio, and training the video generation model according to the model training data. According to the invention, the generation efficiency of the vocal organ action video is improved, and the real vocal action process of the vocal organ is restored and visually displayed by using the vocal organ action video.

Description

technical field [0001] The present disclosure relates to the field of video, and in particular, to a method and device for generating video, a storage medium, and electronic equipment. Background technique [0002] In the scene of pronunciation learning, it is difficult for people to know the movement and exertion of other people's internal pronunciation organs. It is more difficult to judge and simulate how to pronounce based on the sound alone, and the learning effect is poor and the efficiency is low. [0003] At present, it is possible to simulate and display the human face during pronunciation by making demonstration animations, but this method cannot show the real action pictures of the internal pronunciation organs of the oral cavity, and the reference is not high, and artificially drawing animations is time-consuming, laborious and inefficient high. Contents of the invention [0004] This Summary is provided to introduce a simplified form of concepts that are desc...

AI Technical Summary

Problems solved by technology

[0002] In the scene of pronunciation learning, it is difficult for people to know the movement and force of the pronunciation organs in the oral cavity of others. It is more difficult to judge and simulate how to pronounce based on the sound alone, and the learning effect is poor and the efficiency is low.

[0003] At present, it is possible to simulate and display the human face during pronunciation by making demonstration animations, but this method cannot show the real action pictures of the internal vocal organs in the oral cavity, and the reference is not high, and artificially drawing animations is time-consuming, laborious and inefficient high

Images