System and method for end-to-end speech recognition with triggered attention

Abstract

A speech recognition system includes an encoder to convert an input acoustic signal into a sequence of encoder states, an alignment decoder to identify locations of encoder states in the sequence of encoder states that encode transcription outputs, a partition module to partition the sequence of encoder states into a set of partitions based on the locations of the identified encoder states, and an attention-based decoder to determine the transcription outputs for each partition of encoder states submitted to the attention-based decoder as an input. Upon receiving the acoustic signal, the system uses the encoder to produce the sequence of encoder states, partitions the sequence of encoder states into the set of partitions based on the locations of the encoder states identified by the alignment decoder, and submits the set of partitions sequentially into the attention-based decoder to produce a transcription output for each of the submitted partitions.

Description

technical field [0001] The present invention relates generally to a system and method for speech recognition, and more particularly, to a method and system for end-to-end speech recognition. Background technique [0002] Automatic speech recognition (ASR) systems are widely used in various interface applications, such as voice search. However, it is challenging to make speech recognition systems that achieve high recognition accuracy. This is because such production requires in-depth linguistic knowledge of the target language accepted by the ASR system. For example, phoneme sets, vocabularies, and pronunciation dictionaries are essential to making such an ASR system. Phoneme sets need to be carefully defined by the language's linguists. Pronunciation dictionaries need to be created manually by assigning one or more phoneme sequences to each word in a vocabulary comprising more than 100,000 words. Also, some languages ​​do not have clear lexical boundaries, so we may nee...

AI Technical Summary

Problems solved by technology

However, attention-based neural networks have output latency and are less suitable for online / streaming ASR where low latency is required

Images