Speech synthesis device supporting styles of multiple speakers, language switching and controllable rhythm

Abstract

The invention discloses a speech synthesis device supporting styles of multiple speakers, language switching and controllable rhythm, and belongs to the field of speech synthesis. The device comprises a text acquisition unit and a text preprocessing unit which are used for acquiring and preprocessing different text data; a language switching unit used for storing and displaying speaker tags corresponding to the training data of different language types and automatically identifying the language type of the text to be synthesized; a style switching unit used for specifying a speech synthesis style according to the language type; a speaker switching unit for specifying a speaker; a coding-decoding unit for obtaining a predicted Mel spectrum; a training unit for training the encoding-decoding unit; and a voice synthesis unit which is used for generating the predicted Mel frequency spectrum and converting the predicted Mel frequency spectrum into a sound signal for voice playing. According to the invention, the speaker and the style of the speaker can be respectively controlled while the voice with richer rhythm change is generated.

Description

technical field [0001] The invention belongs to the field of speech synthesis, in particular to a speech synthesis device supporting multi-speaker styles, language switching and rhythm controllable. Background technique [0002] In recent years, with the development of deep learning, speech synthesis technology has also been greatly improved. Speech synthesis has moved from the traditional parametric method and concatenation method to an end-to-end method. They usually first generate the mel spectrum from the text features, and then use the vocoder image to synthesize the speech from the mel spectrum. According to the structure, these end-to-end methods can be divided into autoregressive models and non-autoregressive models. Autoregressive models usually use the Encoder-Attention-Decoder mechanism for autoregressive generation: to generate the current data point, all previous data points in the time series must be generated as model input, like Taoctron, Taoctron 2, Deep ...

AI Technical Summary

Problems solved by technology

[0003] Existing speech synthesis methods only control the synthesized speech, and cannot synthesize speech mixed with multiple languages, nor can they decouple the styles of multiple speakers and apply them to other speakers.

Images