Method for processing face and speech synchronous animation based on Chinese text drive

Abstract

The invention discloses a method for processing face and speech synchronous animation based on Chinese text drive, comprising the steps of classifying all Chinese phonemes into 16 groups of Chinese visual phonemes according to the size feature of the lip action when pronouncing according to Chinese pinyin, and using an input face image to synthesize a corresponding key frame; analyzing an input text to obtain a corresponding Chinese visual phoneme sequence and a key frame sequence of the animation; interposing a transition frame between every two neighbor key frames; aligning the key frame sequence and a speech stream; and finally, simultaneously playing a speech stream and an animation stream to achieve the face and speech synchronous animation. After inputting any face head portrait andany text content, the method can automatically finish generation and output of the face animation, is simple in operation and smooth in effect, and is suitable for kinds of such occasions as a visualhuman-computer interface, a computer game, the teaching of Chinese as a foreign language and the like.

Description

technical field [0001] The invention relates to the technical field of face-to-speech synchronous animation, in particular to a processing method for face-to-speech synchronous animation driven by Chinese text. Background technique [0002] Text information, audio information and visual information are the main manifestations of human information and knowledge today, and they are also important tools for human learning and communication. Nowadays, people pay more and more attention to the interaction between various information, integrating text, sound and image together to form a direct conversion from text to visual voice, that is, a voice-synchronized face animation system, Let people see a synchronously speaking human face while listening to the sound of the computer, making the human-computer interaction interface more friendly and harmonious. Face-to-voice synchronous animation technology In recent decades, from the initial sequential playback of stored static picture...

AI Technical Summary

Problems solved by technology

However, this system requires special multi-modal data acquisition equipment to simultaneously collect and record the speaker's speech and face three-dimensional feature point motion information, which will inevitably increase the difficulty of system implementation and limit the scope of use of the system. Based on the voice driver, it is necessary to record the sound information read aloud before the animation is generated, and the corresponding animation cannot be generated for any text that needs to be read aloud

Chinese patent application number: 200910263558.8, invention patent name: voice-driven lip animation method, the realization of this method needs to collect the original audio data and video data of several individuals: each person reads the initial and final words, and uses DV or camera to shoot at the same time, In order to obtain audio stream and video stream data, more content needs to be collected, and it is not completely automated

Images