Speech synthesis method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a speech synthesis method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring hidden layer representation for speech synthesis data, and inputting the hidden layer representation into a phoneme-level rhythm controller to obtain discrete phoneme-level rhythm prediction; inputting discrete phoneme-level rhythm prediction and hidden layer representation into an acoustic model in a mixed manner, predicting discrete acoustic features of each frame through a classifier in the acoustic model, and predicting frame-level rhythm features by using a convolutional neural network in the acoustic model; and inputting the discrete acoustic features and the frame-level rhythm features into a vocoder to generate various rhythm voices. According to the embodiment of the invention, the discretized voice representation is used for replacing the traditional Mel spectrum, so that the problem of wrong transmission is greatly reduced. Not only is the tone quality of the synthesized voice greatly improved, but also the rhythm diversity is reserved. Different rhythms can be generated through the rhythm controller, so that various voices are generated.

Description

technical field [0001] The present invention relates to the field of intelligent speech, in particular to a speech synthesis method, system, electronic device and storage medium. Background technique [0002] TTS (text-to-speech, text-to-speech) synthesis is the process of converting text into corresponding speech. Compared with traditional statistical parametric speech synthesis, the neural TTS model based on deep neural network has better performance. Mainstream neural text-to-speech synthesis systems are usually a cascaded system that converts input text to mel-spectrum and mel-spectrum to audio. Tacotron2, FastSpeech 2, GlowTTS, etc. can be used during conversion. Among them, Tacotron2 is a sequence-to-sequence model based on attention mechanism, FastSpeech2 is a parallel generation model based on Transformer network, GlowTTS uses a reversible network, inversely transforms the distribution of Mel spectrum into a simple distribution, through the maximum likelihood crite...

AI Technical Summary

Problems solved by technology

[0005] In order to at least solve the problem in the prior art that in the cascade system, the prediction error of the acoustic model will be passed down, and if the Mel spectrum prediction is inaccurate, the quality of the generated audio will be affected

Images