Method and device for optimizing end-to-end speech synthesis model, and electronic equipment

Abstract

The invention provides a method and a device for optimizing an end-to-end speech synthesis model, electronic equipment and a storage medium. The method comprises the following steps: carrying out the first soft shielding of phonemes contained in a text inputted into an end-to-end speech synthesis model according to a first preset rule, and generating a second text; sequentially encoding the second text by using a phoneme encoder, and performing prediction processing on the encoded second text by using a variable information predictor to obtain a first output; performing second soft shielding on the first output according to a second preset rule; and inputting the first output subjected to the second soft shielding processing into a preset decoder, and decoding to obtain a Mel spectrum. According to the optimization method of the end-to-end speech synthesis model provided by the invention, soft shielding is added to the input of the end-to-end speech synthesis model and the input of the decoder respectively, so that data disturbance is increased, and the robustness of the end-to-end speech synthesis model can be improved.

Description

technical field [0001] The invention relates to the technical field of speech synthesis, in particular to an end-to-end speech synthesis model optimization method and device, and electronic equipment. Background technique [0002] Generally speaking as figure 1 As shown, TTS (text-to-speech, speech synthesis) is divided into several parts such as text analysis (for example, text regularization, polyphone disambiguation, etc.) module, prosody prediction module, duration model, acoustic model and vocoder . The processed text passes through the prosody prediction module, outputs the text with prosody symbols, and then performs word-to-sound conversion and other links. The current mainstream end-to-end model integrates the duration model and the acoustic model into one model. The text generates phoneme information through the front end, while the end-to-end model uses phoneme information as input to generate a mel spectrum, and then connects a vocoder. Convert acoustic featur...

AI Technical Summary

Problems solved by technology

However, there are also certain problems in end-to-end synthesis, which are related to the pure black-box structure of the end-to-end model

For example, for a model with an explicit duration module, its decoder is prone to overfitting to wrong information, thus affecting the quality of the final synthesized speech

It can be seen that the current end-to-end model has poor robustness, and it is urgent for those skilled in the art to provide a solution to improve the robustness of the end-to-end model

Images