The present invention discloses a method for speech-driven facial
animation simulation based on facial
muscle linkage, belonging to the field of
artificial intelligence, including the steps of: S1, constructing a PPMF
encoder; S2, constructing a decoder based on FDCP to decode the features F provided by the PPMF to obtain facial animations; S3, training the speech-driven 3D face
animation framework DCPTalk; S4, model optimization; S5, model quantitative evaluation. The present invention proposes the DCPTalk framework and, based on the linkage characteristics of facial
muscle groups, proposes Mouth2Face. The mouth movement has a strong correlation with the speech
signal and is easily synthesized with
vocal tract dynamics. In order to further enhance the details of facial movements, a Refine Decoder is used to simulate the
skin deformation on the surface to refine the facial animations. The inherent body characteristics and the body characteristics related to the facial
muscle group movements are embedded into Mouth2Face to construct a personalized facial muscle
control system, and at the same time, the external drive
signal is modulated by the
speaking style. Qualitative and quantitative experiments and
user studies show that DCPTalk is superior to the existing state-of-the-art methods. P The mouth movement has a strong correlation with the speech
signal and is easily synthesized with
vocal tract dynamics. In order to further enhance the details of facial movements, a Refine Decoder is used to simulate the
skin deformation on the surface to refine the facial animations. The inherent body characteristics and the body characteristics related to the facial
muscle group movements are embedded into Mouth2Face to construct a personalized facial muscle
control system, and at the same time, the external drive signal is modulated by the
speaking style. Qualitative and quantitative experiments and
user studies show that DCPTalk is superior to the existing state-of-the-art methods.