Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Statistics and rule combination based phonetic driving human face carton method

A technology that drives human face and face movement, which is applied in computing, special data processing applications, biological neural network models, etc.

Inactive Publication Date: 2004-01-07
INST OF COMPUTING TECH CHINESE ACAD OF SCI
View PDF0 Cites 25 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Like the problems encountered in speech synthesis, it is not difficult to obtain a large number of real face motion images and face synthesis primitives. The difficulty lies in how to edit and reuse these existing face animation data
At the same time, the above processing is based on statistical learning, which can deal with strong correlations such as speech and lip movement, but it is difficult to obtain weak correlations such as speech and blinking, speech and head gestures through learning.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Statistics and rule combination based phonetic driving human face carton method
  • Statistics and rule combination based phonetic driving human face carton method
  • Statistics and rule combination based phonetic driving human face carton method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] Firstly, the feature vector class of Face Motion Parameters (FAP) of the video can be obtained by unsupervised cluster analysis. Then count the face dynamic model (essentially the FAP category transition matrix) that occurs synchronously with the voice event, which we call the statistical visual model, and its principle is similar to the statistical language model in natural language processing. Finally, learn to train multiple neural networks (ANN) to complete the allusion from speech mode to facial animation mode. After machine learning, some face animation pattern sequences can be obtained by calculating the new voice data, and the best face motion parameter (FAP) sequence can be selected by using the statistical vision model, and then the FAP sequence can be corrected and supplemented by using the face motion rules , after smoothing is done, use these FAPs to directly drive the face mesh model. This strategy has the following unique features:

[0020] 1) The estab...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention is a method face cartoon driven by voice which is based on the combination of statistic and rules, it includes steps: gets the audio and video frequency data flow by using audio and video frequency synchronous cuttnig method; gets the corresponding eigenvector through audio and video frequency analyzing method; gets the radio and video frequency synchronous mapping relation model through statistic learning method; gets the face moving parameter which is corresponding to the vodei serial of the user by using the model and drives the face cartoon model. The invention uses video gathering, voice analyzing and photo processing, records the actual voice when speaking and the character points moving data.

Description

technical field [0001] The present invention relates to a voice-driven face animation method based on the combination of statistics and rules, especially a method of video collection, voice analysis and image processing to record the voice and facial feature point movement data when the real face speaks , establish an initial voice image database; the displacement of the voice data analysis window can be calculated through the video acquisition frame rate and the voice data sampling rate, and at the same time use these data to obtain the synchronous corresponding relationship between the voice and the video frame by using the statistical learning method Model. By using this model and adding rules, the face motion parameters corresponding to any voice can be obtained to drive the face animation model. Background technique [0002] After the technology of recovering a realistic three-dimensional face through one or several images or video sequences has become a reality, the c...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06N3/02
Inventor 陈益强高文王兆其
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com