Voice recognition and voice synthesis model training method and device and computer equipment

Abstract

The invention relates to a voice recognition and voice synthesis model training method, which comprises the steps of obtaining a first high-dimensional vector output after an audio processing networkprocesses voice data of a first data pair in a training set, and obtaining a second high-dimensional vector output after a text processing network processes text data of the first data pair; trainingan audio processing network and a text processing network on the training set through a loss function until training convergence; after training convergence, fixing a first parameter set correspondingto the audio processing network and a second parameter set corresponding to the text processing network; under the first parameter set and the second parameter set, training a text recovery network and an audio recovery network to converge; sequentially combining and connecting the audio processing network and the text recovery network to obtain an acoustic pre-training model for speech recognition, and sequentially combining and connecting the text processing network and the audio recovery network to obtain an acoustic pre-training model for speech synthesis. The model construction and training cost is saved.

Description

technical field [0001] This application relates to the computer field, in particular to a model training method, device and computer equipment for speech recognition and speech synthesis. Background technique [0002] "Speech recognition" and "speech synthesis" are two "sequence-to-sequence" prediction tasks in a dual relationship, which can be modeled using the encoder-decoder framework. Since the training data of "speech recognition" and "speech synthesis" are not universal, the existing speech recognition system only achieves a one-way mapping of aligning speech information to text information, and speech synthesis only achieves aligning text information to speech information one-way mapping. Due to the diversity of sequences, the scale of each one-way mapping system is very large, and the amount of data required for training the system is also very large, so the construction and training costs of each one-way mapping system are very high, which is not conducive to "spee...

AI Technical Summary

Problems solved by technology

[0003] The main purpose of this application is to provide a model training method for speech recognition and speech synthesis, aiming to solve the problem that the construction and training costs of the existing one-way mapping system are very high, which is not conducive to the general promotion of "speech recognition" and "speech synthesis" systems Technical issues used

Images