Front-end text analysis method based on multi-task learning

Abstract

The invention discloses a front-end text analysis method based on multi-task learning, which uses the same corpus to label features and results, uses a CNN (Convolutional Neural Network) as a sharing layer to extract the features of the corpus, then respectively puts the features into two Bi-LSTMs (Bidirectional Long Short Term Memory) for parallel training, and outputs results for two tasks, and specifically comprises the following steps: S1, data labeling; s2, preparing features; s3, performing feature fusion; and S4, classifying. According to the method, polyphone prediction and rhythm prediction tasks are combined by using a multi-task learning method, and a unified end-to-end text processing model is realized, namely, a unified front-end structure is provided, so that a high-quality mandarin TTS system is constructed more quickly and more easily. The training of the unified model can use the same data as input, polyphones and rhythms can be directly predicted from an original text at the same time, two tasks can be trained in parallel, the workload of data labeling is reduced, the training cost is saved, two results are output at the same time, and the training process is simplified.

Description

technical field [0001] The invention relates to the technical field of speech synthesis, in particular to a front-end text analysis method based on multi-task learning. Background technique [0002] Text-to-Speech (TTS), also known as speech synthesis. Aiming to synthesize understandable natural speech from text, it has a wide range of applications in human communication and has long been a research topic in the fields of artificial intelligence, natural language processing, and speech processing. Developing a TTS system requires knowledge of language and human speech generation across multiple disciplines, including linguistics, acoustics, digital signal processing, and machine learning. With the development of deep learning, neural network-based TTS has flourished, and a lot of research work has focused on different aspects of neural TTS. As a result, the quality of synthesized speech has greatly improved in recent years. [0003] In Mandarin text-to-speech synthesis, t...

AI Technical Summary

Problems solved by technology

However, this serial structure also brings several problems

One is a complex feature engineering and data labeling effort, since each component requires different input and output labels

The other is that the front-end components need to be trained and optimized separately, which makes the training process very complicated

Images