A training method for neural network of speech recognition device

Abstract

The invention discloses a cyclic neural network, comprising: a baseline model, which is formed by connecting two layers of LSTM network layers; an extended model, which includes a multi-layer residual network layer, and the residual network layer of each layer is composed of a layer of LSTM network layer It is formed by connecting with an additive function layer, the input of the residual network layer is connected to the output of the previous network layer, and the two inputs of the additive function layer are respectively connected to the output of the LSTM network layer of the residual network layer and the previous network layer The output of the layer, the output of the additive function layer is used as the output of the residual network layer. The invention also discloses a training method of the cyclic neural network. The invention can deepen the depth of the LSTM network layer recurrent neural network, and can improve the training effect and performance.

Description

technical field [0001] The present invention relates to speech recognition, in particular to a recurrent neural network. The invention also relates to a training method for the cyclic neural network. Background technique [0002] Such as figure 1 Shown is a model structure diagram of an existing speech recognition device; an existing recurrent neural network (RNN) is formed by connecting two layers of long short-term memory (LSTM) network layers 102 . [0003] figure 1 In , the recurrent neural network is used in a speech recognition device. [0004] The speech recognition device includes: a convolution layer (convolution layer) 101 , the cyclic neural network, a fully connected layer (Fully connected Layer) 103 and a layer 104 based on a Connectionist Temporal Classification (CTC) layer. [0005] The convolutional layer 101 receives a sound frequency spectrum signal, the output of the convolutional layer 101 is connected to the cyclic neural network, and the cyclic deep...

AI Technical Summary

Problems solved by technology

[0009] The disadvantage of the existing cyclic neural network composed of LSMT network layers 102 is that only about 2 layers of cyclic neural network can be used; when the number of layers is increased, the training will fail to converge, or the training effect will be significantly worse than that of the shallow network, thus Cannot further improve the performance of recurrent networks

Images