Method and system for driving human face animation through real-time voice

A voice and animation technology, which is applied in the field of real-time voice-driven facial animation, can solve the problems that the voice animation method depends on the speaker, speaking style, and cannot be retargeted to any facial equipment, etc., and achieves easy editing and high fidelity Effect

Active Publication Date: 2020-02-04
北京中科深智科技有限公司
View PDF5 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the above-mentioned technical problems, the object of the present invention is to provide a method and system for real-time voice-driven facial animation that has nothing to do with the speaker and can be

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for driving human face animation through real-time voice
  • Method and system for driving human face animation through real-time voice
  • Method and system for driving human face animation through real-time voice

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is an embodiment of a part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0061] It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and a system for driving human face animation by real-time voice. The method comprises the following steps: acquiring a neutral voice audio-visual data set from a first speaker; tracking and parameterizing face video data by using an active appearance model; converting the voice data into a phoneme label sequence; training a deep convolutional neural network modelbased on a sliding window; relocating the target of the reference face model to the target role model; and inputting the target phoneme label sequence from the second speaker into the deep convolutional neural network model of the target role model for prediction. The system provided by the invention comprises an acquisition module, a face conversion module, a phoneme conversion module, a trainingmodule, a target redetermination module and a target prediction module. According to the method and the system provided by the invention, the problems that the existing voice animation method dependson a specific speaker and a specific speaking style and cannot retarget the generated animation to any facial equipment are solved.

Description

technical field [0001] The invention relates to the fields of virtual reality and animation, in particular to a method and system for real-time voice-driven facial animation. Background technique [0002] Voice animation is an important and time-consuming aspect of generating photorealistic animation. Broadly speaking, voice animation refers to moving the facial features of a graphic (or robotic) model so that lip movements are synchronized with speech and give the impression of speech generation. As humans, we're all experts on faces, and poor voice animation can be distracting, unpleasant, and confusing. For example, audio-visual language mismatches can sometimes change what viewers think they hear, and high-fidelity voice animation is critical to effective character animation. [0003] However, existing machine learning-based speech animation methods are usually evaluated based on test samples distributed in the same distribution as the training set, and the results are...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T13/20G06T13/40G06N3/04G06N3/08G10L13/08G10L15/02G10L15/25G10L15/26G10L25/30G10L25/45
CPCG06T13/205G06T13/40G06N3/08G10L15/02G10L15/25G10L25/30G10L25/45G10L13/08G06T2207/10016G06T2207/20081G06T2207/20084G06T2207/30201G10L2015/025G10L15/26G06N3/045
Inventor 不公告发明人
Owner 北京中科深智科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap