Accelerated calculation method and system of recurrent neural network and related device

A technology of cyclic neural network and calculation method, applied in the field of systems and related devices, and the field of accelerated calculation method of cyclic neural network, can solve problems such as unfavorable deep learning algorithm operation, reduce deep learning efficiency, occupy system hardware resources, etc., to avoid computing Effects of difficulty, shortening calculation time, and simplifying the calculation process

CN111723906APending Publication Date: 2020-09-29LANGCHAO ELECTRONIC INFORMATION IND CO LTD
2 Cites 2 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Publication Date
2020-09-29

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention provides an accelerated calculation method of a recurrent neural network. The accelerated calculation method comprises the steps of obtaining a parameter matrix multiplier of the recurrent neural network; dividing a multiplier and a multiplicand in the parameter matrix multiplier to obtain a two-dimensional matrix; inputting the two-dimensional matrix into a three-dimensional pulsation array according to a preset sequence; and determining an output result of the parameter matrix multiplication according to the three-dimensional pulsation array. According to the method and the device, the calculation time of large-size matrix multiplication is greatly shortened, and hardware resources required by calculation are reduced. The invention further provides an accelerated computingsystem of the recurrent neural network, a computer readable storage medium and a terminal, which have the above beneficial effects.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The present application relates to the field of deep learning, and in particular to an accelerated computing method, system and related devices of a recurrent neural network. Background technique

[0002] Recurrent Neural Network (RNN for short) is a type of recurrent neural network that takes sequence data as input, recurses in the evolution direction of the sequence, and connects all nodes (recurrent units) in a chain. Research on recurrent neural networks began in the 1980s and 1990s, and developed into one of the deep learning algorithms in the early 21st century, among which bidirectional recurrent neural network (Bidirectional RNN, Bi-RNN), gated recurrent unit network (Gated RecurrentUnit networks, GRU) and Long Short-Term Memory networks (LSTM) are common recurrent neural networks.

[0003] In the calculation process of the recurrent neural network, its network parameters usually exist in the form of a matrix, that is, the calculation process ...

Examples

Embodiment Construction

[0037] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0038] RNN is a sequence-to-sequence model that can be defined as follows:

[0039] Xt: represents the input at time t, ot: represents the output at time t, and St: represents the memory at time t.

[0040] The basis of RNN is:

[0041] St=f(U*Xt+W*St-1)

[0042] The f function here is the activation function in the neural network, the more common ...