System and method for accelerating RNN network and storage medium

Abstract

The invention discloses a system for accelerating an RNN network. The system comprises a first cache, a second cache, a vector multiplication circuit, a summing circuit, an activation circuit, a stateupdating circuit, a bias data cache, a vector cache and a cell state cache, wherein the first cache is used for outputting Wx1 to WxN or Wh1 to WhN in parallel in N paths in a mode of cyclic switching, and the degree of parallelism of the Wx1 to WxN or the Wh1 to WhN is k; the second cache is used for outputting xt or h<t-1> in a cyclic switching mode; the vector multiplication circuit is used for respectively calculating Wx1xt to WxNxt or respectively calculating Wh1h<t-1> to WhNh<t-1> by utilizing the N groups of multiplication arrays; the summing circuit is used for realizing calculation from Wx1xt+Wh1h<t-1>+b1 to WxNxt+WhNh<t-1>+bN; the activation circuit is used for performing activation operation according to the output of the summing circuit; and the state updating circuit is usedfor acquiring c<t-1>, calculating ct and ht, updating c<t-1> and sending ht to the second cache. By applying the scheme of the invention, the acceleration of the RNN network is effectively realized, and the flexibility and expansibility are very high. The invention also provides a method for accelerating the RNN network and a storage medium, which have corresponding effects.

Description

technical field [0001] The invention relates to the technical field of neural networks, in particular to a system, method and storage medium for accelerating RNN networks. Background technique [0002] RNN (Recurrent Neural Network, cyclic neural network) is a neural network for processing sequence data. It is one of the most promising tools in deep learning and is widely used in speech recognition, machine translation, text generation and other fields. It solves the problem that traditional neural networks cannot share positional features from data. In traditional neural network models such as CNN and DNN, from the input layer to the hidden layer to the output layer, the layers are fully connected, and the nodes between each layer are unconnected. This ordinary neural network is powerless for many problems. For example, if you need to predict what the next word of a sentence is, you generally need to use the previous words, because the preceding and following words in a s...

AI Technical Summary

Problems solved by technology

It solves the problem that traditional neural networks cannot share positional features from data

Images