Unlock instant, AI-driven research and patent intelligence for your innovation.

AI synthesis anchor generation method, electronic equipment and readable storage medium

A generator and video frame technology, applied in biological neural network models, voice analysis, instruments, etc., can solve the problems of not having wide practicability, being unable to promote multiple speakers, and the unnatural combination of anchor voice and expression, etc. Achieve the effect of small difference in details, high video quality, and easy to use

Active Publication Date: 2021-07-27
INNER MONGOLIA UNIVERSITY
View PDF14 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] At present, the performance of speech synthesis, mouth shape synthesis and expression synthesis technology has reached a relatively high level, but there are still many research difficulties, such as the inability to generate new anchor content with a small amount of speaker material, which is not widely applicable and cannot be extended to Multiple speakers can only target specific people, and the combination of the anchor's voice and expression is still unnatural

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • AI synthesis anchor generation method, electronic equipment and readable storage medium
  • AI synthesis anchor generation method, electronic equipment and readable storage medium
  • AI synthesis anchor generation method, electronic equipment and readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0034] The AI ​​synthetic anchor generation method specifically includes the following steps:

[0035] Step 1: Collect data and preprocess it. The specific process is as follows:

[0036] Step 1a, collect LipReading Sentences 2 (LRS2), an open-source data set of BBC (British Broadcasting Corporation), which consists of speech videos of hundreds of people, totaling about 29 hours, and renumbers the video expectations, starting from 0 a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an AI synthesis anchor generation method, electronic equipment and a readable storage medium, and the method comprises the following steps: collecting data, and preprocessing the data to obtain voice data and video frame data; training a mouth shape synchronization discriminator to judge the synchronization probability of the voice data and the video frame data; and training a video generator to generate a synthetic video frame. According the invention, difference between the generated AI synthesis anchor and the real video is small, the synchronism of the voice and the mouth shape in the video is good, and the video transition is smooth without jumping.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence, and relates to an AI synthetic anchor generation method, an electronic device and a readable storage medium. Background technique [0002] "AI (Artificial Intelligence) synthetic anchor" is a joint modeling training that combines multi-modal information such as voice and video frames through a number of cutting-edge technologies such as face key point detection, face feature extraction, and face reconstruction. It is an AI avatar model that is no different from a real person. This technology can automatically generate a video of the corresponding content from the input text, and ensure that the audio, expressions and lip movements in the video are naturally consistent, showing the same information transmission effect as a real anchor. [0003] In recent years, methods based on deep learning have gradually become the mainstream methods of speech synthesis, mouth shape synthesis, fa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G10L21/10G10L25/57G10L25/24G06K9/00G06N3/04
CPCG10L21/10G10L25/57G10L25/24G10L2021/105G06V40/172G06V40/161G06N3/045
Inventor 王炜华董林坤张晖飞龙高光来
Owner INNER MONGOLIA UNIVERSITY