Voice-driven animation method and device based on artificial intelligence

Abstract

An embodiment of the invention discloses a voice-driven animation method based on artificial intelligence. When a to-be-processed voice comprising a plurality of voice frames is obtained, linguistic information corresponding to the voice frames in the to-be-processed voice can be determined, wherein each piece of the linguistic information is used for identifying the probability of distribution ofphonemes to which the corresponding voice frame belongs, namely, reflecting which probability distribution of the phonemes contents in the voice frame belong to; information carried by the linguisticinformation is irrelevant to an actual speaker of the to-be-processed voice, so that the influence of pronunciation habits of different speakers on determination of subsequent expression parameters can be counteracted; and according to the expression parameters determined by the linguistic information, an animation image can be accurately driven to make an expression corresponding to the to-be-processed voice, such as a mouth shape, so that the to-be-processed voice corresponding to any speaker can be effectively supported and the interactive experience is improved.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a voice-driven animation method and device based on artificial intelligence. Background technique [0002] At present, the technology of voice-to-virtual facial animation generation is becoming a research hotspot in the field of industrial applications. For example, for a voice of any speaker, an animated image can be driven to make the mouth shape corresponding to the voice. In this scenario, the presence of animated images can greatly enhance the sense of reality, improve expressiveness, and bring users a more immersive experience. [0003] One way is to implement the above technology through the Speech2Face system. Generally speaking, for a speaker's voice, after the system extracts the acoustic features in the voice such as Mel Frequency Cepstral Coefficient (MFCC), the mapping model can be used to determine the corresponding one that can be adjusted based on the aco...

AI Technical Summary

Problems solved by technology

[0004] However, since the extracted acoustic features contain information related to the speaker, the mapping model established based on this can accurately determine the corresponding expression parameters for the speech of a specific speaker. If the speaker changes, the expression determined by the mapping model There will be a large deviation in the parameters, and the mouth shape of the animated image driven by this will be inconsistent with the voice, which will reduce the interactive experience

Images