Speech synthesis model training method and speech synthesis method

Abstract

The invention discloses a speech synthesis model training method. A speech synthesis model comprises an encoder, a speaker embedding prediction network, a duration expansion module and a decoder. The method comprises the following steps of: preprocessing training data to obtain a sample training data set and a target speaker data set; training the speech synthesis model based on the sample training data set; and performing adaptive training on the speaker embedding prediction network based on the target speaker data set so as to predict a speaker embedding prediction value based on a to-be-synthesized text. According to the embodiment of the invention, a speech synthesis model is integrally trained based on sample training data to obtain a general speech synthesis model, and adaptive training is further carried out on a speaker embedded prediction network in the general speech synthesis model obtained through training based on a target speaker data set, so that the speaker embedded prediction network can learn the timbre characteristics of a target speaker, and audio signals closer to the target speaker are synthesized during speech synthesis.

Description

technical field [0001] The invention relates to the technical field of speech synthesis, in particular to a speech synthesis model training method, a speech synthesis method, electronic equipment and a computer-readable storage medium. Background technique [0002] In recent years, with the popularization of mobile devices, human-computer interaction scenarios using voice have become more and more common. Speech, as the most important and natural way of communication for human beings, is considered to be the most natural entrance for human-computer interaction applications, and is currently widely used in different human-computer interaction scenarios. A complete voice-based human-computer interaction system includes user inquiries, machine recognition and understanding, and then generates text through natural language, and finally feedbacks to users through speech synthesis (Text-To-Speech, TTS) to reply. Therefore, synthesizing high-definition, high-naturalness and divers...

AI Technical Summary

Problems solved by technology

[0005] Global-level and sentence-level speaker embeddings, because they provide limited speaker timbre information, can only have a relatively good similarity in the test data set that is relatively similar to the data in the set. For some test data that cannot be fully fitted , it will make the similarity of the synthesized sound very poor

Specifically, because the training criterion of the pre-trained speaker embedding method does not require it to reconstruct the audio features, only the discriminative criterion is used, and the sentence-level timbre information it can provide is very limited. Similarity does not achieve a satisfactory result

Although the reference encoder method of joint training can provide more speaker information, since the number of speakers in TTS data will be much less than the number of speakers in the verification task, it will not be effective in the voice synthesis of unseen speakers. It will be better than the pre-trained speaker embedding method d-vector, x-vector, or even worse

[0006] Although the frame-level speaker embedding increases the speaker information in granularity, the similarity improvement is limited by the reference audio and the unstable attention mechanism calculation, so the similarity improvement of out-of-set speakers is very limited

[0007] The method of directly updating model parameters is prone to problems such as overfitting and very unstable synthetic sound quality due to inaccurate labeling of test data and small amount of data.

Using LHUC and other methods will reduce the amount of updated model parameters, which can alleviate the problem of overfitting to a certain extent, but if the distribution of the target data is very different from the original data, it will not be able to fit the new data well.

Images