System and method for training cloned tone and rhythm based on Bottleneck features

Abstract

The invention relates to the technical field of voice synthesis, voice recognition and voice cloning, and provides a voice cloning implementation scheme based on Bottleneck features (language featuresof audio) by combining a voice synthesis technology, a voice recognition technology and a transfer learning technology. A training system and a training method are included. The TTS service with highnaturalness and similarity is provided by using a small number of samples, so that the TTS service with target user characteristics is provided, and problems of large service sample size, long manufacturing period and high labor cost of a voice synthesis technology are solved. The training system comprises a data acquisition module, an acoustic feature extraction module, a voice recognition module, a rhythm module, a multi-person voice acoustic module and a voice synthesis module. The invention further provides a training method based on the system. The training method comprises the steps oftraining corpus preparation, acoustic feature extraction, training and fine adjustment of all modules and speech synthesis.

Description

technical field [0001] The invention relates to the fields of speech synthesis technology (TTS), speech recognition technology (ASR), and sound cloning technology, and belongs to the field of artificial intelligence-intelligent speech. Background technique [0002] With the rapid development of the telephone robot business market, the rapid increase in the volume of intelligent voice services has brought great difficulties to customized speech synthesis technology services (TTS). A set of customized speech synthesis technology services (TTS) requires nearly 10,000 For real recording samples, the production cycle from sample collection, data labeling, data preprocessing, model training to service provision is nearly one month, and requires a lot of labor costs. This delay cannot meet the market's response. Currently, TTS mainly includes two technical solutions: staged speech synthesis and end-to-end speech synthesis. The purpose of timbre and rhythm cloning is to synthesize ...

AI Technical Summary

Problems solved by technology

[0002] With the rapid development of the telephone robot business market, the rapid increase in the volume of intelligent voice services has brought great difficulties to customized speech synthesis technology services (TTS). A set of customized speech synthesis technology services (TTS) requires nearly 10,000 For real recording samples, the production cycle from sample collection, data labeling, data preprocessing, model training to service provision is nearly one month, and requires a lot of labor costs. This delay cannot meet the response of the market

Images