Method, system and device for converting voice into lip shape and storage medium

Abstract

The invention discloses a method, a system and a device for converting voice into a lip shape and a storage medium. According to the method, voice is processed by using the long-term and short-term memory network, so that a relatively high processing speed can be achieved, the process of outputting lip-shaped key points is completed with relatively low time consumption, and the delay caused by thewhole process is relatively low. In the training process of the long-term and short-term memory network, the long-term and short-term memory network can analyze received voice according to the language law of human beings, and a proper lip shape image can be output more accurately. By setting a reverse processing process for the lip-shaped key points output by the long-term and short-term memorynetwork, the lip-shaped key points are processed according to the opposite logic of the preprocessing process of the training set, so that the influence of the preprocessing process on the formation of the long-term and short-term memory network can be eliminated, the finally obtained lip-shaped key points have proper distribution, and the subsequent application of generative adversarial networksand the like for visual processing is facilitated. The method is widely applied to the technical field of voice data.

Description

technical field [0001] The invention relates to the technical field of voice data, in particular to a method, system, device and storage medium for converting voice into lip shapes. Background technique [0002] In fields such as virtual anchors, there is a wide demand for converting speech into lip shapes. Combining text-to-speech technology, then converting speech into lip shapes, and then displaying lip shapes on computer-generated character avatars, it can convert boring press releases into realistic lip movements and provide a good visual experience. However, the existing technology generally has the disadvantages of huge amount of calculation and long time consumption, and there is a large delay in the process of converting speech into lip shape. If the speech to be processed by the prior art is obtained from text conversion , the output lip shape needs to be applied to subsequent steps such as deformation or mapping, then the delay in the process of converting speech...

AI Technical Summary

Problems solved by technology

However, the existing technology generally has the disadvantages of huge amount of calculation and long time consumption, and there is a large delay in the process of converting speech into lip shape. If the speech to be processed by the prior art is obtained from text conversion , the output lip shape needs to be applied to subsequent steps such as deformation or mapping, then the delay in the process of converting speech into lip shape will be superimposed with the delay of other processes, resulting in an easily noticeable and unbearable delay, which seriously affects Visual experience

Images