Method and device for driving virtual human in real time, electronic equipment and medium

Abstract

The embodiment of the invention discloses a method for driving a virtual human in real time, and the method comprises the steps: obtaining to-be-processed data used for driving the virtual human, wherein the to-be-processed data comprises at least one of text data and voice data; processing the to-be-processed data by using an end-to-end model, and determining an acoustic feature sequence, a facial feature sequence and a limb feature sequence corresponding to the to-be-processed data; and inputting the acoustic feature sequence, the facial feature sequence and the limb feature sequence into a trained muscle model, and driving a virtual human through the muscle model. In this way, the acoustic feature sequence, the facial feature sequence and the limb feature sequence can be obtained in a shorter time through the end-to-end model; and then the obtained sequence is input into the muscle model to directly drive the virtual human, so that the calculation amount and the data transmission amount are greatly reduced, the calculation efficiency is also improved, and the real-time performance of driving the virtual human is greatly improved.

Description

technical field [0001] The embodiments of this specification relate to the technical field of virtual human processing, and in particular to a method, device, electronic device and medium for real-time driving of a virtual human. Background technique [0002] Digital Human, referred to as Digital Human, is a comprehensive rendering technology that uses computers to simulate real humans. It is also known as virtual humans, super-realistic humans, and photorealistic humans. Because people are too familiar with real people, it takes a lot of time to obtain the 3D static model to make the 3D static model very real, but when driving the 3D static model to perform actions, even a subtle expression will be remodeled. As a result, modeling requires a large amount of data for calculation, and the calculation process is long. Usually, an action of the model may take an hour or several hours of calculation to be realized, resulting in very poor real-time performance of the driver. Co...

AI Technical Summary

Problems solved by technology

Because people are too familiar with real people, it takes a lot of time to obtain the 3D static model to make the 3D static model very real, but when driving the 3D static model to perform actions, even a subtle expression will be remodeled. As a result, modeling requires a large amount of data for calculation, and the calculation process is long. Usually, an action of the model may take an hour or several hours of calculation to achieve, resulting in very poor real-time performance of the driver.

Images