Multi-scale StarGAN voice conversion method based on shared training

Abstract

The invention discloses a multi-scale StarGAN voice conversion method based on shared training. According to the method, by using a multi-scale StarGAN structure, multi-scale features are representedat different levels, and the receptive field range of each layer of network is increased; by simultaneously using residual connections, gradient dissipation problem is relieved, so that the network can be propagated in a deep level, and the quality of the converted voice is obviously improved; and furthermore, a shared module Share-Block of a discriminator and a classifier is trained by using Share-Learning, so that model parameters can be reduced, the parameters of the shared module can be trained at the same time, the training process of the discriminator and the classifier can be accelerated, the performance of the discriminator and the classifier is improved, a high-quality voice conversion method is achieved, and the method has good application prospects in the fields of cross-language voice conversion, movie dubbing, voice translation, medical assistance and the like.

Description

technical field [0001] The present invention relates to a voice conversion method, in particular to a voice conversion method based on shared training multi-scale StarGAN Background technique [0002] Speech conversion is a research branch in the field of speech signal processing, which is developed and extended on the basis of speech analysis, recognition and synthesis. The goal of voice conversion is to change the voice personality of the source speaker so that it has the voice personality of the target speaker, that is, to make the voice spoken by one person sound like another person's voice after conversion, while preserving semantics . [0003] Speech conversion under non-parallel text refers to the speech content of the source speaker and the target speaker, and the speech duration is different. The existing speech conversion methods under non-parallel text conditions include methods based on Conditional Variational Auto-Encoder (C-VAE) and methods based on Cycle-Con...

AI Technical Summary

Problems solved by technology

In practical applications, the semantic features encoded by the encoder are all expressed on the semantic scale, and the receptive field of each layer of the network is the same, so it is not possible to focus on extracting features at different scale levels such as words and phonemes. Deepen, the gradient will disappear, making the network difficult to train

On the other hand, due to the limited training corpus, StarGAN has many module parameters, is easy to overfit, and has poor generalization ability, so the conversion performance in a small amount of corpus is very limited.

In addition, when training the discriminator and classifier separately, the focus of the model may be on a single task, ignoring other information that may be shared by multiple tasks that can help optimize metrics, so the converted speech has a high degree of similarity , the disadvantage of poor sound quality

Images