Multi-to-multi speech conversion method based on text encoder under non-parallel text conditions

Abstract

The invention discloses a multi-to-multi speech conversion method based on a text encoder under non-parallel text conditions. The method includes a training part and a conversion part, a conditional variation self-encoder, the generative adversarial network (VAWGAN) and a text encoder (Text-Encoder) are used to implement a speech conversion system, sentence embedding representing semantics is added to the VAWGAN, the speech quality and personality similarity of the converted speech can be better improved, and high-quality speech conversion is realized. The method is advantaged in that dependence on parallel texts is relieved, speech conversion under the non-parallel text conditions is realized, not any alignment process is required in the training process, versatility and practicability ofthe speech conversion system are improved, the conversion system of multiple source / target speaker pairs can be integrated into one transformation model, that is, multi-talker-to-multi-talker conversion is realized, and the method has good application prospects in the fields of film dubbing, speech translation, speech synthesis and the like.

Description

technical field [0001] The invention relates to a voice conversion method, in particular to a text coder-based many-to-many voice conversion method under non-parallel text conditions. Background technique [0002] Voice conversion (Voice Conversion, VC) is a technique that converts source speaker features into target speaker features while preserving semantic information. In research in recent years, the VC model uses Deep Neural Networks (DNN) to convert source speech parameters into target speech parameters. Compared with the traditional Gaussian Mixture Model (GMM), DNN can be more effective transform voice features. [0003] Recently, Variational Auto-Encoder (VAE) has been used for non-parallel VC models because VAE is easier to train than restricted Boltzmann machines. In traditional VAE-based non-parallel VC, the encoder extracts speaker-independent latent variables representing semantic content from the input speech parameters, and then the decoder reconstructs the...

AI Technical Summary

Problems solved by technology

However, due to the over-regularization effect in the latent variables of VAE, which makes the distribution of latent variables too simplistic and difficult to represent the underlying structure of semantic content, the quality of this VAE-based non-parallel corpus-transformed speech is lower than that of parallel speech Quality of corpus-trained DNN conversions

If a more complex prior distribution of latent variables, such as GMM, is used to solve the problem, but because the semantic content varies greatly, it is not easy to determine the number of GMM clusters, so it is very difficult to implement

At present, the non-parallel corpus conversion speech method based on VAE has the disadvantages of poor speech quality and high noise after conversion.

Images