Direct speech-to-speech translation via machine learning

Abstract

The present disclosure provides systems and methods that train and use machine-learned models such as, for example, sequence-to-sequence models, to perform direct and text-free speech-to-speech translation. In particular, aspects of the present disclosure provide an attention-based sequence-to-sequence neural network which can directly translate speech from one language into speech in another language, without relying on an intermediate text representation.

Description

[0001] related application [0002] This application claims priority and benefit to US Provisional Patent Application No. 62 / 826,258, which is hereby incorporated by reference in its entirety. technical field [0003] This disclosure relates generally to machine learning. More specifically, the present disclosure relates to direct and text-free speech-to-speech translation by machine learning models, such as sequence-to-sequence models. Background technique [0004] Speech-to-speech translation (S2ST) refers to the process of translating speech in one language (eg, represented by a first speech waveform) into speech in a different language (eg, represented by a different second speech waveform). Conventional S2ST systems rely on a cascaded approach that combines multiple disparate systems to perform translation. In particular, conventional S2ST systems are often divided into three components that operate separately and sequentially: automatic speech recognition (ASR), text...

AI Technical Summary

Problems solved by technology

Finally, cascaded systems cannot learn to produce fluent pronunciations...

Images