Method for converting voice into virtual face image

Abstract

The invention relates to a method for converting voice into a virtual face image, which is executed by a computer system and comprises the following steps of (A) carrying out machine learning calculation on each conversation segment of each conversation and a corresponding phoneme thereof so as to establish a phoneme conversion model; (B) according to the voice feature data, filtering out voice except the voice of the user from the captured current voice so as to obtain filtered voice data; (C) judging whether the filtered sound data comprises a voice segment or not; (D) when the judgment result is positive, converting a voice segment included in the filtered sound data into a target phoneme by using the phoneme conversion model; (E) according to the target phonemes, target mouth shape images corresponding to the target phonemes being obtained from the stored multiple frames of mouth shape images, and each mouth shape image corresponding to one of the phonemes; and (F) obtaining at least one virtual face image related to the digital character according to the target mouth shape image.

Description

technical field [0001] The invention relates to a conversion method, in particular to a method for voice conversion of a virtual facial image. Background technique [0002] Industries such as animation, games, and movies often create virtual characters, and one of the most important technologies in creating virtual characters is to enable the virtual character to have natural, smooth mouth movements that are synchronized with the voice, and to achieve the harmony between voice and mouth movements. Simultaneously, the designer needs to adjust the mouth shape configuration on the time axis according to the audio content, which is very time-consuming and labor-intensive. Furthermore, the animation cannot be changed according to the atmosphere of the scene. Therefore, many manufacturers are committed to the development of real-time voice. A technology that drives the speaking of virtual characters. [0003] However, most of the existing real-time voice-driven avatar speaking te...

AI Technical Summary

Problems solved by technology

[0002] Industries such as animation, games, and movies often create virtual characters, and one of the most important technologies in creating virtual characters is to enable the virtual character to have natural, smooth mouth movements that are synchronized with the voice, and to achieve the harmony between voice and mouth movements. Simultaneously, the designer needs to adjust the mouth shape configuration on the time axis according to the audio content, which is very time-consuming and labor-intensive. Furthermore, the animation cannot be changed according to the atmosphere of the scene. Therefore, many manufacturers are committed to the development of real-time voice. Technology that drives virtual character talking

[0003] However, most of the existing real-time voice-driven avatar speaking technologies are to transfer the voice to text first, and then convert the text into the mouth shape image of the avatar. Effect

Images