Sequence-to-sequence speech synthesis method and system for double-layer autoregressive decoding

Abstract

The invention provides a sequence-to-sequence speech synthesis method and system for double-layer autoregressive decoding. The system comprises an encoder and a decoder, and the decoder comprises a phoneme-level representation module, a phoneme-level prediction module and a frame-level prediction module; the encoder represents phoneme names, tones and rhythm phrase boundary information by using vectors, and then encodes and fuses the information by using a convolutional neural network and a bidirectional long-short-term memory network to obtain context unit representation of each phoneme in asentence; the phoneme-level representation module is used for obtaining acoustic unit representation of each phoneme unit through a frame-level long-short-term memory network (LSTM) and pooling processing; the phoneme level prediction module adopts a phoneme level autoregressive structure to predict acoustic unit representation of the current phoneme and establish a dependency relationship betweencontinuous phonemes; the frame level prediction module predicts acoustic characteristics of a frame level through a decoder LSTM.

Description

technical field [0001] The invention belongs to the field of speech signal processing, and in particular relates to a sequence-to-sequence speech synthesis method and system of double-layer autoregressive decoding. Background technique [0002] Speech synthesis, which aims to enable machines to speak fluently and naturally like humans, has benefited many voice-interactive applications, such as intelligent personal assistants and robots. Currently, statistical parametric speech synthesis (SPSS) is one of the mainstream methods. [0003] Statistical parametric speech synthesis utilizes an acoustic model to model the relationship between text features and acoustic features, and utilizes a vocoder to obtain speech waveforms given predicted acoustic features. Although this approach can produce clear voices, the quality of synthesized speech will always suffer due to the limitations of the acoustic model and vocoder. Recently, Wang and Shen et al. proposed a neural network-based...

AI Technical Summary

Problems solved by technology

[0005]The current neural network-based sequence-to-sequence speech synthesis methods are all designed based on the frame-level autoregressive decoding structure, and there is a lack of long-term correlation modeling ability. In addition, the robustness of the attention mechanism adopted by the model is not ideal. When synthesizing complex text, there are synthetic speech errors such as repetition, omission, and inability to stop.

Images