Voice synthesis method and device based on rhythm boundary, medium and equipment

Abstract

The invention provides a speech synthesis method and device based on a rhythm boundary, a medium and equipment. The method comprises the following steps: acquiring rhythm boundary information of to-be-synthesized text information, and generating graph embedding information based on the rhythm boundary information; generating an implicit state vector of the graph embedding information and a sequence code of the text information to be synthesized; generating a speech spectrum based on the implicit state vector and the sequence code; and synthesizing voice information of the to-be-synthesized text information according to the voice spectrum. Based on the method provided by the invention, semantics and grammatical structures of sentences can be analyzed from a text side, and rhythm boundariesare represented through graph embedding, so that rhythm information in the text can fully participate in training and reasoning, and the rhythm feeling of the synthesized voice information is improved. The invention also relates to a blockchain technology, and data such as the hidden state vector and the sequence code of the text information to be synthesized are stored in a blockchain, so that the security of data storage is improved.

Description

technical field [0001] The present invention relates to the technical field of speech synthesis, in particular to a speech synthesis method, device, medium and equipment based on prosodic boundaries. Background technique [0002] In a speech synthesis system (TTS) based on deep learning, prosody is an important factor determining the naturalness and fluency of synthesized speech. Prosody can be subdivided into 3-dimensional features, fundamental frequency, loudness and duration. In end-to-end speech synthesis systems, academia and industry try to extract the hidden state of prosodic embedding from the MEL spectrum of the speech, and then a global style vector is introduced into the multi-head attention mechanism for training to control the synthesis of speech sentence prosodic effect; variational autoencoder is used as a prosodic classifier to learn the hidden state of prosodic embedding in a variety of prosodic data sets; in order to obtain more accurate local prosodic con...

AI Technical Summary

Problems solved by technology

[0003] These methods try to analyze prosodic information from the phonetic side, that is, extract prosodic features from spectral information in the frequency domain, because the prosody of a speech can be fully displayed in the frequency domain, but it cannot fully represent the semantics and semantics of the input text sequence. Grammatical information, but the text-side information largely determines the local prosody information of a sentence, so the prosody effect of synthesized speech often does not conform to the prosody rhythm of the text content

Images