Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method for voice-driven lip animation

A lip and voice technology, applied in the field of voice-driven lip animation, can solve the problems of visual matching error, audio matching error, large error, etc., and achieve the effect of strong operation, easy implementation and high motion efficiency.

Inactive Publication Date: 2010-06-23
SICHUAN UNIV
View PDF4 Cites 52 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method is used to deal with the synchronization of speech and lip animation in any language. Since the pronunciation rules and pronunciation characteristics of each language are different, it is difficult to have a uniform standard processing method, and in this method, each primitive The audio matching error and the visual matching error between the front and back primitives will cause such an error that it will be very large, and dynamically selecting an optimal path from the candidate primitives may not be the best path

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for voice-driven lip animation
  • Method for voice-driven lip animation
  • Method for voice-driven lip animation

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment approach

[0030] The sample lip shape data is obtained through SAM detection frame pictures. Since the width and height of each person's lips are different, they are normalized to a standard range according to a quantification rule, that is, although the height and width of people's lips are different, each The distance from the edge point to the center point of the individual lips is roughly the same as the width (height) ratio of the lips, so that during the lip shape clustering process, the clustering error caused by the lip shape sampling data due to the different lip sizes of each person can be eliminated , to normalize the lip shapes of different people. In addition, the two-dimensional coordinate points on a frame of lip pictures are converted into points in a high-dimensional space, so that the points on a frame of pictures are synthesized into a column of high-dimensional vectors, and each column vector represents a frame of lip picture information. For all These column vectors...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for voice-driven lip animation which is characterized by including the following steps: sorting a syllable structure in Chinese by adopting Consonant-Vowel, collecting the original video data and audio data of a plurality of persons, acquiring corresponding lips, acquiring voice characteristic data, training and modeling the acquired lip information and voice characteristic information, and inputting a lip motion sequence synthesized by voice in real time according to the trained model. The method overcomes the defects of the prior art, and has the characteristics of small calculated data volume, strong maneuverability, etc.

Description

technical field [0001] The invention relates to the technical field of three-dimensional human face animation, in particular to a voice-driven lip animation method. Background technique [0002] Lip animation is one of the important contents and one of the difficulties in the study of facial expression animation. Human language has multimodal characteristics. The speaker's voice and lip movement, that is, the state change of lip shape, jointly affect the perception of others. The speech comprehension of the speaker, which shows that there is a certain, natural connection between speech and lip shape. Lip synchronization is a time mapping relationship between audio stream and video stream, which refers to a synchronization relationship between the speaker's voice and lip movement state. Using the mapping model is a widely used and effective algorithm. [0003] The Chinese patent 200910003083.9 applied by Tencent Technology (Shenzhen) Co., Ltd. discloses a method of voice-dri...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T15/70G06T13/00G10L21/06G10L15/02
Inventor 章毅王平安周激流郑伯川吕建成张蕾彭德中于佳丽张海仙
Owner SICHUAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com