Block-based self-attention real-time end-to-end speech translation method

Abstract

The invention discloses a block-based self-attention real-time end-to-end speech translation method, which comprises the following steps of: preprocessing recorded audio file training data, mapping an ID (Identity) of each speech, a storage path of each speech and a corresponding target language text, and constructing two mapping files; respectively extracting two acoustic features of a Mel filter bank and a Mel frequency cepstrum coefficient of the audio; constructing a target language dictionary by using the training data, wherein the target language dictionary is used for generating a target language text sequence during decoding; cleaning the training data, and converting the training data into a format file required by an end-to-end speech translation model; initializing an end-to-end speech translation model, and training by using a data file in a specific format; in the inference stage, the size of the block is set, and the trained end-to-end speech translation model is used to dynamically encode the source speech, so that a target statement is generated in real time. According to the method, the model has the capability of real-time speech translation, and the decoding speed of the model is improved under the condition that the performance of the model is not reduced.

Description

technical field [0001] The invention relates to an end-to-end real-time speech translation method, in particular to a block-based self-attention real-time end-to-end speech translation method. Background technique [0002] Speech translation (Speech Translation) broadly refers to the process of translating the speech of a language into the speech or text corresponding to the target language. Speech translation usually refers to the process of translating speech into corresponding target language text, while Speech-to-speech Translation specifically refers to the process of translating speech into corresponding target language speech. Speech translation has a very wide range of application scenarios, such as subtitle generation, conference simultaneous interpretation, etc., and plays an important role in cross-language communication. [0003] In the past, speech translation was usually carried out in a cascading manner, that is, a speech recognition system was used to recogn...

AI Technical Summary

Problems solved by technology

However, since there is a relatively strong constraint in the speech recognition task, that is, the input and output are monotonically aligned, there is no such constraint in the speech translation task

Therefore, direct transfer of block-based real-time speech recognition methods to real-time speech translation tasks is less effective.

Images