Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Voice synchronous-drive three-dimensional face mouth shape and face posture animation method

A voice-synchronized driving and facial gesture technology, which is applied in animation production, computer parts, image data processing, etc., can solve problems such as difficult facial gestures and weak correlations, and achieve the effect of reducing intelligibility and recognizability

Inactive Publication Date: 2013-07-24
SOUTHWEST JIAOTONG UNIV
View PDF5 Cites 70 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This is because, although there is an obvious correlation between mouth shape and speech, the correlation between facial posture and speech is relatively weak, so it is relatively difficult for speech to drive accurate facial posture

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Voice synchronous-drive three-dimensional face mouth shape and face posture animation method
  • Voice synchronous-drive three-dimensional face mouth shape and face posture animation method
  • Voice synchronous-drive three-dimensional face mouth shape and face posture animation method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0030] The present invention will be further described below in conjunction with accompanying drawing and specific embodiment:

[0031] The specific implementation method of the present invention roughly comprises the following steps:

[0032] 1. Viseme classification, because the corresponding mouth-frames of some consonants and finals are similar, in order to reduce the amount of calculation, the present invention carries out viseme classification of some consonants and finals according to their corresponding mouth shapes, which are divided into 16 categories, F 0 -F 15 . Specific categories such as figure 1 shown.

[0033] 2. Create an audio / video corpus and record it with a high-definition video camera. 20 people, 10 men and 10 women, read the classified consonants in step 1, and record audio and video at the same time. When the voice is recorded, facial video information synchronized with the voice is collected. In order to facilitate the retrieval and extraction of ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a voice synchronous-drive three-dimensional face mouth shape and face posture animation method. A user can input new voice information, and the new voice information can be preprocessed to combine mouth shape animations and face posture animations which are synchronous with voice on the face head of a virtual man. The method specifically comprises two stages. In a training stage, voice visualization modeling can be achieved through a k-nearest neighbor algorithm (KNN) and hidden Markov model (HMM) mixed model. In a combining stage, the user can input new voice information, characteristics of voice signals are extracted, face posture and mouth shape sequence parameters corresponding to the voice signals can be generated through the KNN and HMM mixed model and are processed in a transition mode, and X face open source software is used to combine delicate and abundant three-dimensional face animations. The method has significant theoretical study value and has wide application prospect in the fields of visual communication, virtual meetings, games, entertainments, teaching assistance and the like.

Description

technical field [0001] The invention relates to the technical field of voice-driven three-dimensional facial animation synthesis, in particular to a voice visualization collaborative pronunciation modeling based on a KNN and HMM hybrid model. Background technique [0002] The research on voice-driven 3D facial animation synthesis is an important content in the field of natural human-computer interaction. Voice-driven 3D facial animation synthesis is to preprocess a person's voice so that it can synthesize lip animation and facial expressions corresponding to the voice on a virtual 3D face head. At present, the research in this area is mainly focused on synchronous synthesis, precise lip animation, and the classification of facial expressions through speech analysis. There is no better way to realize voice to drive virtual human lip animation and facial expressions at the same time. Posture (facial gestures or visual prosody). The so-called facial posture refers to non-verb...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T13/40G06K9/62
Inventor 侯进米辉辉
Owner SOUTHWEST JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com