End-to-end speech recognition method based on rotation position coding

Abstract

The invention discloses an end-to-end speech recognition method based on rotation position coding. According to the method, the modeling capability of a convolutional self-attention network (Conformer) on acoustic features is enhanced by using the rotation position coding. Firstly, absolute position information of elements in an input sequence is coded through a rotation matrix, then relative position information is added into an inner product of an input vector of a multi-head self-attention module, an end-to-end speech recognition model based on a convolutional self-attention network is constructed, and then input speech is converted into text information through the speech recognition model. According to the invention, experiments are carried out on AISHELL-1 and LibriSpeech corpora, and the experiment result shows that the Conformer enhanced by the rotation position coding is better than the original Conformer in the aspect of speech recognition task. The word error rate of 4.69% is achieved on a test set of an AISHELL-1 data set, and the word error rate of 2.1% and the word error rate of 5.1% are achieved on a test-clean set and a test-over set of a LibriSpeech data set respectively.

Description

technical field [0001] The invention belongs to the technical field of pattern recognition, and in particular relates to a speech recognition method. Background technique [0002] The temporal information of the input sequence plays a crucial role in many sequence learning tasks, especially in speech recognition. Recurrent neural network based models can learn the timing information of sequences by recursively computing their hidden states along the time dimension. Convolutional neural network-based models can implicitly learn the position information of input sequences through padding operators. In recent years, Transformer-based models have shown great superiority in various sequence learning tasks such as machine translation, language modeling, and speech recognition. The Transformer-based model uses self-attention mechanism to model the dependencies between different elements in the input sequence, which provides more efficient parallel computing than recurrent neural ...

AI Technical Summary

Problems solved by technology

However, the relative position encoding increases the parameter amount of the model, and the matrix operation of the relative position encoding is more cumbersome to implement

Images