Block-based self-attention real-time end-to-end speech translation method

Abstract

The invention discloses a block-based self-attention real-time end-to-end speech translation method, which comprises the following steps of: preprocessing recorded audio file training data, mapping an ID (Identity) of each speech, a storage path of each speech and a corresponding target language text, and constructing two mapping files; respectively extracting two acoustic features of a Mel filter bank and a Mel frequency cepstrum coefficient of the audio; constructing a target language dictionary by using the training data, wherein the target language dictionary is used for generating a target language text sequence during decoding; cleaning the training data, and converting the training data into a format file required by an end-to-end speech translation model; initializing an end-to-end speech translation model, and training by using a data file in a specific format; in the inference stage, the size of the block is set, and the trained end-to-end speech translation model is used to dynamically encode the source speech, so that a target statement is generated in real time. According to the method, the model has the capability of real-time speech translation, and the decoding speed of the model is improved under the condition that the performance of the model is not reduced.

Description

technical field [0001] The invention relates to an end-to-end real-time speech translation method, in particular to a block-based self-attention real-time end-to-end speech translation method. Background technique [0002] Speech translation (Speech Translation) broadly refers to the process of translating the speech of a language into the speech or text corresponding to the target language. Speech translation usually refers to the process of translating speech into corresponding target language text, while Speech-to-speech Translation specifically refers to the process of translating speech into corresponding target language speech. Speech translation has a very wide range of application scenarios, such as subtitle generation, conference simultaneous interpretation, etc., and plays an important role in cross-language communication. [0003] In the past, speech translation was usually carried out in a cascading manner, that is, a speech recognition system was used to recogn...

AI Technical Summary

Problems solved by technology

However, since there is a relatively strong constraint in the speech recognition task, that is, the input and output are monotonically aligned, there is no such constra

Images