Systems and methods for neural voice cloning with a few samples

Abstract

Voice cloning is a highly desired capability for personalized speech interfaces. Neural network-based speech synthesis has been shown to generate high quality speech for a large number of speakers. Neural voice cloning systems that take a few audio samples as input are presented herein. Two approaches, speaker adaptation and speaker encoding, are disclosed. Speaker adaptation embodiments are basedon fine-tuning a multi-speaker generative model with a few cloning samples. Speaker encoding embodiments are based on training a separate model to directly infer a new speaker embedding from cloningaudios, which is used in or with a multi-speaker generative model. Both approaches achieve good performance in terms of naturalness of the speech and its similarity to original speaker-even with veryfew cloning audios.

Description

technical field [0001] The present disclosure generally relates to systems and methods for computer learning that can provide improved computer performance, features and usage. More specifically, the present disclosure relates to systems and methods for text-to-speech via deep neural networks. Background technique [0002] Artificial speech synthesis systems, often referred to as text-to-speech (TTS) systems, convert written language into human speech. TTS systems are used in various applications such as human-machine interface, accessibility for the visually impaired, media and entertainment. Fundamentally, it allows human-computer interaction without a visual interface. Traditional TTS systems are based on complex multi-stage artificially engineered pipelines. Typically, these systems first convert text into a compact audio representation, and then convert this representation into audio using an audio waveform synthesis method called a vocoder. [0003] One goal of a T...

AI Technical Summary

Problems solved by technology

Traditional TTS systems are based on complex multi-stage artificially engineered pipelines

Images