Speech translation model training method and system based on model enhancement, and speech translation method and device

Abstract

The invention discloses a speech translation model training method and system based on model enhancement, and a speech translation method and device. The training method includes: collecting a speech translation data set, and the speech translation data set consists of multiple speech-translation-transcription triplets Composition; use the speech-transcription data in the speech translation data set to train the speech recognition model, use the transcription-translation data in the speech translation data set to train the machine translation model; use the speech recognition model to initialize the coding layer of the speech translation model, and use the machine translation The model initializes the decoding layer of the speech translation model; masks the output of the hidden layer of the speech translation model, uses the speech translation data set and combines the loss function to train the speech translation model; after the speech translation model is trained, remove the mask, and The trained speech translation model is fine-tuned. The invention improves the recognition performance of the speech translation model, and can effectively improve the speech translation efficiency and quality.

Description

technical field [0001] The invention belongs to the technical field of speech translation, and in particular relates to a speech translation model training method and system based on model enhancement, and a speech translation method and device. Background technique [0002] Speech translation is an end-to-end model that converts speech in one language into text in another language, that is, skips the step of converting the source language into text in the traditional model, and directly converts the speech in the source language into the target Language text is a hot research topic at present. In terms of model selection, the Transformer model proposed by Google (A.Vaswani, N.Shazeer, N.Parmar, J.Uszkoreit, L.Jones, A.N.Gomez, L.Kaiser, and I.Polosukhin, "Attention is all you need, "in Proc.NIPS, California, USA, 2017, pp.5998–6008.) Due to the effectiveness of its self-attention mechanism modeling, the efficiency of parallel processing and the simplicity of the model stru...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the problem that the existing speech translation model requires a large amount of data, the present invention proposes a speech translation model training method, system, and speech translation method and device based on ModAugment, by increasing the alignment difficulty of the encoding and decoding parts , transforming the overfitting problem into an underfitting problem, after a longer training period, an overall more robust model can be obtained, and better results can be achieved with a smaller hyperparameter adjustment burden

Images