Multi-to-multi speaker conversion method based on STARGAN and x vector

A conversion method and speaker technology, applied in speech analysis, instrumentation, speech synthesis, etc., can solve problems such as the inability to fully express the individual characteristics of the speaker, and the lack of great improvement in the similarity of speech and speech.

Active Publication Date: 2019-04-23
NANJING UNIV OF POSTS & TELECOMM
View PDF6 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the identity label of the speaker in this method cannot fully express the individual characteristics of the speaker, so the converted speech has not been greatly improved in speech similarity

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-to-multi speaker conversion method based on STARGAN and x vector
  • Multi-to-multi speaker conversion method based on STARGAN and x vector
  • Multi-to-multi speaker conversion method based on STARGAN and x vector

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0055] Such as figure 1 As shown, the method of the present invention is divided into two parts: the training part is used to obtain the parameters and conversion functions required for voice conversion, and the conversion part is used to convert the source speaker's voice into the target speaker's voice.

[0056] The implementation steps of the training phase are:

[0057] 1.1) Obtain the training corpus of non-parallel text, the training corpus is the corpus of multiple speakers, including the source speaker and the target speaker. The training corpus is taken from the VCC2018 speech corpus. There are 6 male and 6 female speakers in the training set of this corpus, and each speaker has 81 sentences. This method can realize conversion under parallel text, and can also realize conversion under non-parallel text, so these training corpora can also be non-parallel text.

[0058] 1.2) The training corpus uses the WORLD speech analysis / synthesis model to extract the spectral en...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a multi-to-multi speaker conversion method based on STARGAN and an x vector, which comprises a training stage and a conversion stage, wherein a speech conversion system is achieved by combining the STARGAN and the x vector, the personality similarity and quality of the converted speech can be greatly improved, particularly, for the short-time utterance, the x vector has better characterization performance and better speech conversion quality can be achieved, meanwhile, the problem of over-smoothing in C-VAE can be overcome, and a high-quality speech conversion method isachieved. In addition, the method can achieve the speech conversion under the condition of non-parallel text, the training process does not need any alignment process, the universality and practicability of a speech conversion system are improved, and the method can also achieve that the conversion system with multiple source-target speaker pairs is integrated in a conversion model, namely, the multi-speaker-to-multi-speaker conversion is achieved, and the system has a better application prospect in the fields of cross-language speech conversion, film dubbing, speech translation and the like.

Description

technical field [0001] The invention relates to a many-to-many speaker conversion method, in particular to a many-to-many speaker conversion method based on STARGAN and x-vector. Background technique [0002] Speech conversion is a research branch in the field of speech signal processing, which is developed and extended on the basis of speech analysis, recognition and synthesis. The goal of voice conversion is to change the voice personality of the source speaker so that it has the voice personality of the target speaker, that is, to make the voice spoken by one person sound like another person's voice after conversion, while preserving semantics . [0003] After years of research on voice conversion technology, many classic conversion methods have emerged. These include most voice conversion methods such as Gaussian Mixed Model (GMM), Recurrent Neural Network (RNN), and Deep Neural Networks (DNN). However, most of these speech conversion methods require that the corpus u...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G10L21/013G10L25/18G10L25/30G10L13/02
CPCG10L13/02G10L21/013G10L25/18G10L25/30G10L2021/0135
Inventor 李燕萍曹盼张燕徐东祥
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap