Method for voice-driven lip animation

A lip and voice technology, applied in the field of voice-driven lip animation, can solve the problems of visual matching error, audio matching error, large error, etc., and achieve the effects of easy implementation, operability, strong movement, and high motion efficiency.

Inactive Publication Date: 2012-05-30
SICHUAN UNIV
View PDF4 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method is used to deal with the synchronization of speech and lip animation in any language. Since the pronunciation rules and pronunciation characteristics of each language are different, it is difficult to have a uniform standard processing method, and in this method, each primitive The audio matching error and the visual matching error between the front and back primitives will cause such an error that it will be very large, and dynamically selecting an optimal path from the candidate primitives may not be the best path

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for voice-driven lip animation
  • Method for voice-driven lip animation
  • Method for voice-driven lip animation

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment approach

[0030] The sample lip shape data is obtained through the SAM detection frame picture. Since the width and height of each person’s lips are different, they are normalized to a standard range according to a quantification rule. That is, although the height and width of people’s lips are different, each The distance from the edge of the individual lips to the center point is roughly the same as the width (height) ratio of the lips, so that in the lip clustering process, the lip sampling data can be eliminated due to the difference in the size of each person’s lips. , To normalize the lip shape of different people. In addition, the two-dimensional coordinate points on a frame of lip-shaped pictures are converted into points on high-dimensional space, so that the points on a frame of pictures are synthesized into a column of high-dimensional vectors, and each column vector represents a frame of lip-shaped picture information. Clustering analysis of these column vectors divides them ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method for voice-driven lip animation which is characterized by including the following steps: sorting a syllable structure in Chinese by adopting Consonant-Vowel, collecting the original video data and audio data of a plurality of persons, acquiring corresponding lips, acquiring voice characteristic data, training and modeling the acquired lip information and voice characteristic information, and inputting a lip motion sequence synthesized by voice in real time according to the trained model. The method overcomes the defects of the prior art, and has the characteristics of small calculated data volume, strong maneuverability, etc.

Description

Technical field [0001] The invention relates to the technical field of three-dimensional face animation, in particular to a method for voice-driven lip animation. Background technique [0002] Lip animation is one of the important content of facial expression animation research, and it is also one of the difficulties. Human language has multi-modal characteristics. The speaker’s voice and lip movement, that is, the state change of the lip shape, affect other people’s The speaker's speech understanding shows that there is a certain and natural connection between speech and lip shape. Lip synchronization is a time mapping relationship between an audio stream and a video stream. It refers to a synchronization relationship between the speaker's voice and the lip motion state. Using a mapping model is a widely used and effective algorithm. [0003] The Chinese patent 200910003083.9 applied by Tencent Technology (Shenzhen) Co., Ltd. discloses a method for voice-driven lip shape change, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06T13/40G10L21/06G10L15/02
Inventor 章毅王平安周激流郑伯川吕建成张蕾彭德中于佳丽张海仙
Owner SICHUAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap