End-to-end speech recognition system based on deep learning

Abstract

The invention discloses an end-to-end speech recognition system based on deep learning. The system includes: an acoustic model, which sequentially comprises a VGG-Net layer, a first full connection layer, a bidirectional RNN layer, a second full connection layer, a Softmax layer and a CTC layer, and is used for extracting two-dimensional FBank features of an audio, performing network processing toobtain probability distribution of each time step, and outputting a candidate pinyin sequence according to an entropy result of the probability distribution of time steps; a language model, which isconnected to the acoustic model and comprises a Transformer encoder and an n-gram model; wherein the Transformer encoder is used for outputting a Chinese character sequence with a same length according to the input candidate pinyin sequence, and the n-gram model is used for processing the output Chinese character sequence and selecting a target Chinese character text for output. According to the method, a final recognition result most conforming to a current context and human expression habits can be obtained.

Description

technical field [0001] The invention relates to the technical field of speech recognition, in particular to an end-to-end speech recognition system based on deep learning. Background technique [0002] Speech recognition is used to convert speech into corresponding text, and generally includes two basic modules: an acoustic module and a language module. For the input speech signal, the acoustic module is responsible for extracting the characteristics of the signal and calculating the probability of speech to syllable (or other smallest unit), while the language module uses the language model to convert the smallest unit into a complete human or computer understandable natural language. [0003] There are currently two types of speech recognition, one is the probabilistic model method, and the other is the deep learning method. For the former, the most typical is the speech recognition model (HMM-GMM) based on hidden Markov model (HMM) and mixed Gaussian distribution (GMM)....

AI Technical Summary

Problems solved by technology

[0005] The above two solutions have certain shortcomings. For the former, the model cannot use the context information of each frame, that is, it cannot use historical information to assist the current task.

In addition, the model assumes a Gaussian distribution between the frame and the state. Although the model is simplified, it has great limitations.

For the latter, although the model can achieve a better convergence effect, due to the cyclic structure of the RNN itself, more RNN units make the training time longer and difficult to parallelize

Images