Collaborative filtering-based real-time voice-driven human face and lip synchronous animation system

A collaborative filtering algorithm and real-time voice technology, applied in animation production, voice analysis, voice recognition, etc., can solve the problems of slow speed and low recognition rate of voice recognition

Inactive Publication Date: 2010-12-29
INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF0 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Due to the relatively low recognition rate of the curren

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Collaborative filtering-based real-time voice-driven human face and lip synchronous animation system
  • Collaborative filtering-based real-time voice-driven human face and lip synchronous animation system
  • Collaborative filtering-based real-time voice-driven human face and lip synchronous animation system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] The present invention will be further described below with reference to the drawings and examples, and the steps and processes for realizing the present invention will be better described through the detailed description of each component of the system in conjunction with the drawings.

[0023] figure 1 It shows a schematic diagram of a real-time voice-driven human face lip synchronization animation system based on a collaborative filtering algorithm. The system is written in C language and can be compiled and run with visual studio under the windows platform, and can be compiled with the GNU compiler suite (GCC) under the linux platform run. exist figure 1 In the preferred embodiment of the present invention, the system of the present invention is divided into three parts: an audio and video encoding module 1, a collaborative filtering module 2, and a facial animation module 3. Among them, the multi-modal data acquisition equipment is used to collect and record the s...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a collaborative filtering-based real-time voice-driven human face and lip synchronous animation system. By inputting voice in real time, a human head model makes lip animation synchronous with the input voice. The system comprises an audio/video coding module, a collaborative filtering module, and an animation module; the module respectively performs Mel frequency cepstrum parameter coding and human face animation parameter coding in the standard of Moving Picture Experts Group (MPEG-4) on the acquired voice and human face three-dimensional characteristic point motion information to obtain a Mel frequency cepstrum parameter and human face animation parameter multimodal synchronous library; the collaborative filtering module solves a human face animation parameter synchronous with the voice by combining Mel frequency cepstrum parameter coding of the newly input voice and the Mel frequency cepstrum parameter and human face animation parameter multimodal synchronous library through collaborative filtering; and the animation module carries out animation by driving the human face model through the human face animation parameter. The system has the advantages of better sense of reality, real-time and wider application environment.

Description

technical field [0001] The invention relates to a voice-driven human face animation system, in particular to a real-time voice-driven human face and lip synchronous animation system based on a collaborative filtering algorithm. Background technique [0002] Harmonious human-computer interaction technology has always been the object of people's attention. The voice-driven face and lip animation technology is an important part of it. Synchronized face and lip animation, its research results are of great significance to the development of face animation, human-computer dialogue and other directions. At present, most of the face-to-speech synchronous lip animation technologies are generally based on speech recognition technology. This technology must first perform speech recognition, and form corresponding visemes from the obtained phonemes. Visemes are key mouth shapes that correspond to phonemes of speech. The continuous visemes can be stitched together to form the animation...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T13/00G10L19/00G10L15/28H04N7/26H04N19/20
Inventor 陶建华穆凯辉车建峰
Owner INST OF AUTOMATION CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap