Collaborative filtering-based real-time voice-driven human face and lip synchronous animation system

Abstract

The invention discloses a collaborative filtering-based real-time voice-driven human face and lip synchronous animation system. By inputting voice in real time, a human head model makes lip animation synchronous with the input voice. The system comprises an audio / video coding module, a collaborative filtering module, and an animation module; the module respectively performs Mel frequency cepstrum parameter coding and human face animation parameter coding in the standard of Moving Picture Experts Group (MPEG-4) on the acquired voice and human face three-dimensional characteristic point motion information to obtain a Mel frequency cepstrum parameter and human face animation parameter multimodal synchronous library; the collaborative filtering module solves a human face animation parameter synchronous with the voice by combining Mel frequency cepstrum parameter coding of the newly input voice and the Mel frequency cepstrum parameter and human face animation parameter multimodal synchronous library through collaborative filtering; and the animation module carries out animation by driving the human face model through the human face animation parameter. The system has the advantages of better sense of reality, real-time and wider application environment.

Description

technical field [0001] The invention relates to a voice-driven human face animation system, in particular to a real-time voice-driven human face and lip synchronous animation system based on a collaborative filtering algorithm. Background technique [0002] Harmonious human-computer interaction technology has always been the object of people's attention. The voice-driven face and lip animation technology is an important part of it. Synchronized face and lip animation, its research results are of great significance to the development of face animation, human-computer dialogue and other directions. At present, most of the face-to-speech synchronous lip animation technologies are generally based on speech recognition technology. This technology must first perform speech recognition, and form corresponding visemes from the obtained phonemes. Visemes are key mouth shapes that correspond to phonemes of speech. The continuous visemes can be stitched together to form the animation...

AI Technical Summary

Problems solved by technology

Due to the relatively low recognition rate of the current speech recognition itself, the speed is relatively slow

Images