Hardware structure of LSTM accelerator computing unit based on tensor processing
By designing accelerator calculation unit based on tensor processing in the LSTM neural network, and using new pulsating arrays and time division multiplexing technology, the problem of high delay in matrix vector multiplication in the LSTM model is solved, and more efficient calculation and resource utilization are achieved. Meet the computing power needs of deep networks.
Patent Information
- Application Number
- CN202310664369.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
A large number of matrix vector multiplication calculations in the LSTM neural network model lead to high computational delay and power consumption, and existing hardware accelerators are difficult to meet the computing power requirements of deep networks, especially in embedded devices.
The LSTM accelerator calculation unit based on tensor processing is designed, and a new pulsating array structure is used to separate the multiplier and adder. The Element-wise calculation module realizes the activation function through time division multiplexing and segmented linear function approximation, reducing the computational complexity and memory consumption.
It effectively reduces the delay and hardware overhead of matrix multiplication in LSTM, improves the inference speed and efficiency of the LSTM neural network model, and meets the computing power requirements of deep networks.
Smart Images

Figure CN119067183A_ABST
Abstract
Description
Technical Field
[0001] The present invention discloses a hardware structure implementation of an LSTM accelerator computing unit based on tensor processing, which has lower delay and power consumption in the reasoning process of an LSTM neural network model with a large number of matrix-vector multiplication calculations. The calculation process of the LSTM network model is mainly divided into a matrix-vector multiplication module, an element-wise calculation module and an activation function module, wherein the activation function module is divided into a sigmoid function module and a Tanh function module. The present invention performs targeted hardware design according to the calculation characteristics of different modules. There is a large demand for matrix-vector multiplication in the LSTM neural network, so the matrix-vector multiplication module adopts a new systolic array structure, improves the multiplication and accumulation unit (MAC) in the general systolic array, and separates the multiplier from the adder. Different from the way in which the multiplier and the adder are combined in a unified MAC array in the traditional systolic array, the new systolic array structure connects the multipliers in a tree topology. The element-wise calculation module adopts time division multiplexing to improve performance. By dividing the Element-wise module into three different states, the computing units in the Element-wise module can be reused in different states, so the Element-wise computing module only needs one Sigmoid, one Tanh, one adder and one multiplier, which effectively saves resources. The activation function modules are the Sigmoid activation function and the Tanh activation function. The Sigmoid and Tanh activation function modules can be implemented by the piecewise linear function approximation method, that is, using a lookup table after linear segmentation. Two RAMs are used to store a i and b i The value of , and then the activation function is implemented through multipliers and adders. Sigmoid activation function and Tanh activation function are only in a i and b i The piecewise linear function approximation method effectively solves the problem of consuming a large amount of memory based on the lookup table calculation and the disadvantage of high computational complexity based on curve fitting calculation. Background Art
[0002] In recent years, neural networks have become a hot research field and an important driving factor for the development of new architectures in new big data and artificial intelligence applications. With the increasing requirements for accuracy, power consumption, and computing time, more and more scholars are keen to design and implement accelerators suitable for neural networks.
[0003] LSTM, a representative of recurrent neural networks, has made significant achievements in speech recognition. As the depth of the network model increases, the demand for hardware computing power is also increasing, and embedded devices cannot meet such a large demand for computing power. Therefore, it is necessary to compress the parameters of the network model without affecting the accuracy, and design accelerator hardware for the compressed model to accelerate the reasoning of the LSTM neural network model.
[0004] The LSTM network model controls the information flow between nodes by adding gate units. Although it avoids the gradient vanishing and explosion problems of the traditional RNN network, each gate structure has a different weight matrix. A large number of weight matrices are related to the input vector x. t and the output h at the previous moment t-1 Multiplication results in a large number of complex matrix-vector multiplication operations.
[0005] Therefore, the key to the LSTM network model hardware accelerator lies in designing a suitable tensor computing unit.
[0006] In view of the fact that the LSTM neural network model has a large number of matrix-vector multiplication calculations, the present invention innovatively proposes an accelerator computing unit hardware circuit based on tensor processing. Summary of the invention
[0007] The technical objectives of the present invention are:
[0008] In order to accelerate the reasoning process of the LSTM neural network model, an accelerator computing unit hardware circuit based on tensor processing is provided to process a large number of matrix-vector multiplication calculations in the LSTM neural network model, thereby improving the calculation speed and accelerating the reasoning process.
[0009] The technical solution implemented by the present invention is as follows:
[0010] The optimization of the hardware structure of the LSTM accelerator computing unit based on tensor processing designed by the present invention mainly includes the following three parts. The first part is the optimization of the matrix-vector multiplication module. The new systolic array structure is adopted to further improve the multiplication and accumulation unit (MAC) in the traditional systolic array, separate the multiplier from the adder, and connect the multipliers in a tree topology instead of combining them in a unified MAC array. The second part is the optimization of the Element-wise computing module. By dividing the Element-wise computing module into three different states, the computing units in the module are reused in different states, thereby improving performance. The third part is the optimization of the activation function module. Both Sigmoid and Tanh activation functions use piecewise linear function approximation methods to effectively reduce computational complexity and memory consumption.
[0011] The matrix vector multiplication calculation module of the present invention is as followsFigure 1 As shown. The matrix-vector multiplication module adopts a systolic array structure, which consists of a group of interconnected processing elements (PEs). Each PE contains two input registers R1 and R2, an output register R3, and a multiplier, and each row is connected to an adder tree. All multipliers are active at each time step. R1 with streaming data is connected in a row in the array, and the data is shifted to the right by one column every time it is cycled. Data can flow directly between cells in a pipelined manner, and communication between the systolic array and the outside world only occurs at the boundary cells.
[0012] The structure of the Element-wise calculation module of the present invention is as follows Figure 2 As shown. The element-wise module is divided into three different states, and different computing units are reused in different states. In the first state S1, the two input data are processed by Sigmoid and Tanh computing units to obtain outputs i and g, and then i and g are multiplied. The computing units required for the first state S1 are Sigmoid, Tanh and a multiplier. In the second state S2, the third input is processed by Sigmoid to obtain output f, and then f and the value c of the previous cell state are multiplied. t-1 Multiply, and finally f*c t-1 and i*g. In the S2 state, the computing units needed are Sigmoid, an adder and a multiplier. In the third state S3, the fourth input is outputted by Sigmoid operation o, c t The output Tanh(c t ), then Tanh(c t ) is multiplied by g, and the output result is h t The computing units needed in this state are Sigmoid, Tanh and a multiplier. Due to the time division multiplexing method, the element-wise computing module only needs a Sigmoid, a Tanh, an adder and a multiplier.
[0013] The structure of the activation function module of the present invention is as follows Figure 3 As shown in the figure, the Sigmoid and Tanh activation function modules are implemented by using a lookup table after linear segmentation. A total of 22 piecewise linear functions are used to approximate the approximation. The formula of the piecewise function is f(x) = a i x+b i , x∈[x i , x i+1 ]. The hardware uses two RAMs to store a i and b i Different input x values correspond to different a iand b i The value of x is the same as the corresponding a in RAM a i The values are multiplied by the multiplier to get a i *x, then a i *x corresponds to b in RAM b i The values are added by the adder to get f(x)=a i *x+b i , and finally the activation function is approximated.
[0014] The overall architecture of the LSTM accelerator computing unit of the present invention is as follows: Figure 4 There is a strict data dependency between different operators in the LSTM model. Taking the input gate as an example, the input vector x i With the previous output y t-1 Composed of vector groups, and weight matrix W ix Common input matrix-vector multiplication operator, old cell state c t-1 With the weight matrix W ic The output of the matrix-vector multiplication operator and the element-wise operator are then combined with the bias b. i The adder is input together, and the adder output is obtained by the Sigmoid activation function operator. t Similarly, the forget gate and output gate go through a similar process to obtain f t and t . g t The input vector group and the weight matrix are multiplied by the matrix-vector multiplication operator and the bias b c The sum is then obtained through the Sigmoid activation function operator. t and g t Common input Element-wise operator, old cell state c t-1 With f t The Element-wise operator is input together, and the output of the two Element-wise operators is then added to obtain the new cell state c. t .c t After the Tanh activation function operator and o t Input the Element-wise operator together to get h t .h t With the weight matrix W yh After the matrix-vector multiplication operator, the output y is obtained t .
[0015] Compared with the traditional LSTM network model accelerator computing unit hardware circuit, the present invention has the following advantages:
[0016] 1. Reduced computational latency of matrix multiplication operations in LSTM. A new systolic array structure is used to improve the multiplication and accumulation unit (MAC) in the traditional systolic array. The multipliers are separated from the adders, and the multipliers are connected in a tree topology instead of being combined in a unified MAC array, which effectively reduces latency.
[0017] 2. Reduce the hardware overhead of the Element-wise calculation module and activation function module in LSTM. The Element-wise calculation module uses time-division multiplexing to divide the calculation module into three different states. The calculation units in the module can be reused in different states, so the Element-wise calculation module only needs one Sigmoid, one Tanh, one adder and one multiplier. In addition, both the Sigmoid and Tanh activation function modules are implemented using piecewise linear function approximation. Compared with curve fitting or direct table lookup, it effectively reduces the computational complexity and memory consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 Matrix-vector multiplication module structure diagram
[0019] Figure 2 Element-wise calculation module structure diagram
[0020] Figure 3 Activation function module structure diagram
[0021] Figure 4 Overall structure diagram of LSTM accelerator computing unit DETAILED DESCRIPTION
[0022] The hardware structure of the LSTM accelerator computing unit based on tensor processing provided by the present invention is described in detail below in conjunction with the accompanying drawings, but does not constitute a limitation to the present invention.
[0023] The present invention is a hardware structure of an LSTM accelerator computing unit based on tensor processing. The hardware circuit of the matrix-vector multiplication computing unit based on the novel systolic array adopts the following Figure 1 The architecture shown in the figure, the Element-wise computing unit hardware circuit adopts Figure 2 The architecture shown in the figure, the hardware circuit of the activation function calculation unit adopts Figure 3 The overall hardware circuit of the LSTM accelerator computing unit adopts Figure 4 The LSTM accelerator hardware unit calculation process proposed by the present invention is as follows:
[0024] Step 1: Matrix-vector multiplication calculation in each gate structure. In the gate structures such as input gate, forget gate and output gate, the input vector x iWith the previous output y t-1 Composed of vector groups, respectively with the weight matrix W ixy , W cxy , W fxy , W oxy Matrix-vector multiplication is performed in the Matrix-Vector Multiplication operator.
[0025] Step 2: Element-wise calculation in each gate structure. In the gate structures such as input gate, forget gate and output gate, the old cell state c t-1 Respectively with the weight matrix W ic , W fx , W oc Perform element-wise calculation in the element-wise operator, and output and bias b i Added by adder.
[0026] Step 3: Calculation of activation functions in each gate structure. In the gate structures such as the input gate, forget gate, and output gate, the output of the Element-wise operator is obtained by the Sigmoid activation function. t , g t 、f t , o t . After i t With g t 、f t With c t-1 After adding the Element-wise operator, the new cell state c is obtained t , c t After the Tanh activation function operator, it is finally combined with o t The output is sent to the Element-wise operator to form a loop.
[0027] The present invention implements the hardware of the LSTM accelerator computing unit based on the above steps, optimizes the matrix-vector multiplication module, and adopts a new systolic array structure; optimizes the Element-wise computing module, divides the module into three different states, and reuses it in different states, thereby improving performance; optimizes the activation function module, and both the Sigmoid and Tanh activation functions adopt the piecewise linear function approximation method, effectively reducing the computational complexity and memory consumption. The present invention can effectively reduce the delay of the LSTM accelerator and accelerate the reasoning of the LSTM neural network model at a relatively low hardware overhead cost.
[0028] The above embodiments are only used to illustrate the method of the present invention rather than to limit it. A person skilled in the art may modify or make equivalent substitutions to the method of the present invention without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be subject to the claims.
Claims
1. A hardware structure of an LSTM accelerator computing unit based on tensor processing, characterized in that: According to the reasoning process of the LSTM network model, the LSTM accelerator computing unit is decomposed into different modules, and different hardware structure designs are carried out according to the computing characteristics of the matrix-vector multiplication module, Element-wise computing module and activation function module in the LSTM network model.
2. A matrix-vector multiplication module according to claim 1, characterized in that: It adopts a systolic array structure, which consists of a group of interconnected processing elements (PEs). Each PE contains two input registers R1 and R2, an output register R3, and a multiplier, and each row is connected to an adder tree. R1 is connected in a row in the array, and the data moves one column to the right every time it cycles; data flows directly between PEs in a pipelined manner, and communication between the systolic array and the outside world only occurs at the boundary units.
3. An Element-wise calculation module according to claim 1, characterized in that: The Element-wise module is divided into three different states, and different computing units are reused in different states. In the first state S1, two input data are passed through the Sigmoid and Tanh operation units to obtain outputs i and g, and then i is multiplied by g. The computing units required in the first state S1 are Sigmoid, Tanh and a multiplier; in the second state S2, the third input is operated by Sigmoid to obtain the output f, which is then multiplied by the value c of the previous cell state. t-1 Multiply, and finally f*c t-1 Add i*g; In the S2 state, the computing units needed are Sigmoid, an adder and a multiplier; in the third state S3, the fourth input is output through the Sigmoid operation o, c t The output Tanh(c t ), then Tanh(c t ) is multiplied by g, and the output result is h t ; The computing units needed in the third state are Sigmoid, Tanh and a multiplier; Due to the use of time division multiplexing, the Element-wise computing module only requires a Sigmoid, a Tanh, an adder and a multiplier.
4. An activation function module according to claim 1, characterized in that: A 22-segment piecewise linear function is used to approximate the activation function, and two RAMs are used on the hardware to store a i and b i Different input x values correspond to different a i and b i The value of x is the same as the corresponding a in RAM a i The value is multiplied by the multiplier to get a i *x, corresponding to b in RAM b i Value and a i *x is added through the adder to get f(x)=a i *x+b i , and finally the activation function is approximated.
Citation Information
Patent Citations
LSTM (Long Short-Term Memory) forward direction operation accelerator based on FPGA (Field Programmable Gate Array)
CN108763159A
Efficient LSTM accelerator based on FPGA
CN113191494A
Compression LSTM accelerator and acceleration method based on FPGA
CN113222133A
Systolic array including fused multiply-accumulate with efficient pre-normalization and extended dynamic range
CN115039067A
