An acceleration system for long short-term memory neural networks

Abstract

The present application provides an acceleration system for a long short-term memory neural network, the acceleration system comprising a host and an acceleration device for realizing data interaction with the host. The acceleration device comprises: a channel group containing multiple computing channels, a weight buffer, a bias buffer and an activation buffer connected to the channel group, in addition, the acceleration device also comprises a controller and a decoding component. In the present application, the host is used to pre-process the weight data to obtain the first input data, and the first input data is a collection of multiple groups of weight data with balanced number of parameters. The weight buffer is used to receive the first input data, and the first input data can be used for calculation after being decoded by the decoding component. Multiple groups of weight data with balanced number of parameters are respectively allocated to different processing units for calculation, so that the processing time of different processing units is basically the same, thereby solving the problem of hardware load imbalance caused by random sparsity.

Description

Technical Field

[0001] The present application relates to a method for accelerating a recurrent neural network (RNN), and more specifically to an acceleration system for a long short-term memory neural network. Background Art

[0002] RNN has been widely used in various sequence tasks, such as natural language processing, audio and video processing, etc. LSTM (Long Short-Term Memory) is one of the most widely used and complex networks. By introducing storage units and gates to store serialized inputs, LSTM can capture and analyze relevant input data in a large range, thereby obtaining relatively more accurate prediction results.

[0003] Although the prediction accuracy of LSTM is higher than other types of neural networks, the size of LSTM models is also expanding. A typical LSTM network generally has hundreds of megabytes of parameters, which leads to high computational complexity, large memory usage, and high power consumption, limiting the application of LSTM in edge devices.

[0004] Weight pruning, architecture search, circulant matrix and other schemes have been applied to reduce the amount of LSTM computation, among which weight pruning is the most widely used. Existing weight pruning schemes are divided into two categories: random sparsity and structured sparsity. Among them, random sparsity refers to random pruning that does not follow a fixed structure. For example, SCNN and STICKER achieve the acceleration effect of random sparsity on CNN (Convolutional Neural Networks). Structured sparsity refers to pruning weights according to a certain established structural rule. For example, the ESE (Efficient Speech Recognition Engine) method proposed by Han requires that the number of non-zero weights in each row of the matrix is ​​equal to ensure computational efficiency. Cao et al. proposed BBS (Bank-Balanced-Sparsity, block balanced sparsity), which ensures hardware load balancing based on the consistency of sparsity between different blocks, that is, pre-designing the sparsity rules and designing the corresponding hardware according to this rule to achieve the actual acceleration effect.

[0005] Although structured sparsity can effectively accelerate LSTM, it also introduces great limitations, namely, a hardware architecture can only support a single pruning scheme, and the hardware can only calculate weight matrices with fixed sparsity. This limits the flexibility of the hardware, and different tasks often use different pruning schemes. Different layers of the neural network in a single task may also use different sparsities. Once the pruning scheme changes, the hardware often must also be adjusted, which greatly increases the development cost. Although random sparsity is more flexible in terms of algorithm, the uneven distribution of weights caused by random pruning can easily lead to unbalanced hardware load. Therefore, how to deal with randomly sparse LSTM is a challenging problem. Summary of the invention

[0006] The present application provides an acceleration system for a long short-term memory neural network to solve the problem of hardware load imbalance caused by random sparsity.

[0007] In a first aspect of the present application, an acceleration system for a long short-term memory neural network is provided, comprising: a host and an acceleration device for realizing data interaction with the host, the host being used to provide data to be calculated to the acceleration device, the data to be calculated comprising weight data obtained after random sparseness in the long short-term memory neural network, and first input data obtained by preprocessing the weight data, the first input data being a set of multiple groups of weight data with balanced number of parameters, the number of groups in the set being no greater than the number of rows of a matrix corresponding to the weight data; wherein the weight data is derived from model compression of the long short-term memory neural network, and the objects of model compression include words and sentences in machine translation or speech recognition, or data in price prediction;

[0008] The acceleration device comprises: a channel group including a plurality of computing channels, a weight buffer, a bias buffer and an activation buffer connected to the channel group; wherein,

[0009] The weight buffer is used to receive and store the first input data;

[0010] The bias buffer is used to receive and store second input data after processing the weight data;

[0011] The activation buffer is used to receive the weight data and the output result of the channel group, and store the output result as the third input data for the next calculation;

[0012] Each of the calculation channels comprises a plurality of processing units, and the processing units are used to receive various types of input data transmitted by the weight buffer, the bias buffer, and the activation buffer, so as to perform calculations using the various types of input data;

[0013] The acceleration device also includes a controller and a decoding component; the controller is connected to the weight buffer, bias buffer, activation buffer and channel group, and is used to update the input data and control the channel group to perform calculations before and after each calculation; the decoding component is arranged between the weight buffer and the channel group, and is used to decode the first input data to obtain multiple groups of weight data with balanced parameter numbers, so that each of the processing units can process the weight data respectively.

[0014] In a second aspect of the present application, a preprocessing method for a long short-term memory neural network is provided, the preprocessing method being applied to a host in an acceleration system provided in any possible implementation of the first aspect, the preprocessing method comprising:

[0015] The weight data are grouped according to the following mathematical model:

[0016]

[0017] st:

[0018]

[0019]

[0020] Among them, N represents N groups, R represents the matrix set corresponding to the weight data, G represents each individual group, L represents the number of non-zero elements in each group, E represents the average number of non-zero elements, and the constraint represents that each group belongs to R and there is no intersection between groups;

[0021] The grouped weight data is encoded to obtain first input data.

[0022] Optionally, encoding the grouped weight data to obtain first input data includes:

[0023] Encode each non-zero parameter in each row of data to obtain a weight storage block, wherein the weight storage block includes a weight value of the non-zero parameter and a relative position index, wherein the relative position index indicates an index of the non-zero parameter relative to a previous non-zero parameter after grouping;

[0024] A position information block is added at the end of each row of data, which indicates the actual position of the row.

[0025] Optionally, after encoding each non-zero parameter in each row of data to obtain a weight storage block, the following steps are further included:

[0026] Determine whether the size of the relative position index exceeds a preset threshold;

[0027] If exceeded, an index supplement block is added after the weight storage block.

[0028] Optionally, a prompt block is added before the position information block, and the prompt block is used to prompt the position of the position information block.

[0029] In a third aspect of the present application, a preprocessing device for a long short-term memory neural network is provided, the preprocessing device comprising:

[0030] The grouping module is used to group weight data according to the following mathematical model:

[0031]

[0032] st:

[0033]

[0034]

[0035] Among them, N represents N groups, R represents the matrix set corresponding to the weight data, G represents each individual group, L represents the number of non-zero elements in each group, E represents the average number of non-zero elements, and the constraint represents that each group belongs to R and there is no intersection between groups;

[0036] The encoding module is used to encode the grouped weight data to obtain first input data.

[0037] Optionally, the encoding module includes:

[0038] An encoding unit, configured to encode each non-zero parameter in each row of data to obtain a weight storage block, wherein the weight storage block includes a weight value of the non-zero parameter and a relative position index, wherein the relative position index indicates an index of the non-zero parameter relative to a previous non-zero parameter after grouping;

[0039] The first adding unit is used to add a position information block at the end of each line of data, where the position information block indicates the actual position of the line.

[0040] Optionally, the device further comprises:

[0041] A judging unit, configured to judge whether the size of the relative position index exceeds a preset threshold value after the encoding unit encodes each non-zero parameter in each row of data to obtain a weight storage block;

[0042] A second adding unit is configured to add an index supplement block after the weight storage block when the judging unit determines that the size of the relative position index exceeds a preset threshold.

[0043] Optionally, the device further comprises:

[0044] The third adding unit is used to add a prompt block before the position information block, and the prompt block is used to prompt the position of the position information block.

[0045] In a fourth aspect of the present application, a decoding method for a long short-term memory neural network is provided, the decoding method being applied to a decoding component in an acceleration system provided in any possible implementation of the first aspect, the decoding method comprising:

[0046] Acquire a data block in the first input data, wherein the data block includes a weight storage block, a position information block, an index supplement block, and a prompt block;

[0047] Decoding is performed according to the type of the data block; wherein, the weight storage block and index supplement block that are adjacent and have a sequence are decoded as the same data block to be decoded, and the absolute index position of the data block to be decoded is equal to the sum of the absolute position of the previous parameter of the data block to be decoded and the relative position index between the two data blocks; if the data block is a position information block, it is output as is; if the data block is a prompt block, the absolute position index of the prompt block is equal to a preset value.

[0048] It can be seen from the above technical solutions that the present application provides an acceleration system for long short-term memory neural networks, the acceleration system comprising a host and an acceleration device for realizing data interaction with the host. The acceleration device comprises: a channel group containing multiple computing channels, a weight buffer, a bias buffer and an activation buffer connected to the channel group. In addition, the acceleration device also includes a controller and a decoding component. In the present application, the host is used to pre-process the weight data to obtain the first input data, and the first input data is a collection of multiple groups of weight data with balanced number of parameters. The weight buffer is used to receive the first input data, and the first input data can be used for calculation after being decoded by the decoding component. Multiple groups of weight data with balanced number of parameters are respectively allocated to different processing units for calculation, so that the processing time of different processing units is basically the same, thereby solving the problem of hardware load imbalance caused by random sparsity. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 A schematic diagram of the structure of an acceleration system for a long short-term memory neural network provided in an embodiment of the present application;

[0051] Figure 2A schematic diagram of the structure of a single computing channel provided in an embodiment of the present application;

[0052] FIG3( a ) is a schematic diagram before task reallocation provided in an embodiment of the present application;

[0053] FIG3( b ) is a schematic diagram of a task after reallocation provided in an embodiment of the present application;

[0054] FIG4( a ) is a schematic diagram before weight encoding provided in an embodiment of the present application;

[0055] FIG4( b ) is a schematic diagram after weight encoding provided in an embodiment of the present application;

[0056] Figure 5 A schematic diagram of the total number of modules in each group in an example provided in an embodiment of the present application;

[0057] Figure 6 A decoding circuit diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] LSTM is used in areas such as text generation, machine translation, speech recognition, image description generation, and video tagging. It should be noted that the input and output of the neural network are both vectors. In order for various models to be processed by the neural network, we must express the words or data in the form of vectors so that the neural network can process it.

[0060] For example, if the input of the neural network is words, we can vectorize the input using the following steps: Create a dictionary containing all words, and each word has a unique number in the dictionary. Any word can be represented by an N-dimensional one-hot vector. Among them, N is the number of words contained in the dictionary. Using this vectorization method, we get a high-dimensional, sparse vector (sparse means that the value of most elements is 0).

[0061] In order to solve the problem of hardware load imbalance caused by random sparsity, the present application provides an acceleration system for long short-term memory neural networks.

[0062] See also Figure 1As shown in the structural diagram, an embodiment of the present application provides an acceleration system for a long short-term memory neural network, including: a host and an acceleration device that implements data interaction with the host. In the embodiment of the present application, the host is the software end, which can be used for inputting data, model compression, etc.; the acceleration device is the hardware end, which is stored in RAM (such as Intel Arria10SX660 FPGA RAM). A bus design is used between the host and the acceleration device, and data interaction is performed through the data bus.

[0063] The host is used to provide the acceleration device with data to be calculated, and the data to be calculated includes weight data obtained after random sparsification in the long short-term memory neural network, and first input data obtained by preprocessing the weight data, and the first input data is a collection of multiple groups of weight data with balanced number of parameters, and the number of groups in the collection is not greater than the number of rows of the matrix corresponding to the weight data; wherein the weight data comes from model compression of the long short-term memory neural network, and the objects of model compression include words and sentences in machine translation or speech recognition, or data in price prediction.

[0064] In the embodiment of the present application, the weight data is stored in the form of a matrix, and the matrix contains multiple rows of data, each row of data is of different lengths, that is, each row contains different numbers of parameters. The purpose of preprocessing the weight data in the embodiment of the present application is to regroup the rows of data of different lengths to obtain multiple groups of data with a balanced number of parameters. Then, when the multiple groups of data are respectively assigned to PE (Processing Element) for calculation, the calculation time used by each PE is basically the same, thereby solving the problem of imbalance between different PEs.

[0065] The acceleration device includes: a channel group containing multiple computing channels, a weight buffer, a bias buffer and an activation buffer connected to the channel group; wherein the weight buffer is used to receive and store the first input data; the bias buffer is used to receive and store the second input data after processing the weight data; the activation buffer is used to receive the weight data and the output result of the channel group, and store the output result as the third input data for the next calculation; each of the computing channels includes multiple processing units, and the processing units are used to receive various types of input data transmitted by the weight buffer, the bias buffer and the activation buffer, so as to use the various types of input data for calculation.

[0066] In order to cooperate with the preprocessing process of the host, the embodiment of the present application also provides a controller and a decoding component in the acceleration device; the controller is connected to the weight buffer, bias buffer, activation buffer and channel group, and is used to update the input data and control the channel group to perform calculations before the end and start of each calculation; the decoding component is arranged between the weight buffer and the channel group, and is used to decode the first input data to obtain multiple groups of weight data with a balanced number of parameters, so that each of the processing units can process the weight data respectively.

[0067] Although the compressed network is smaller, it still cannot be completely stored in RAM (for example, the Intel Arria10SX660 FPGA RAM size is only 5.2MB), so the weights and other parameters are stored in DDR3. This solution deploys 32 computing channels in the public decoding component. Each computing channel has the same internal structure and can independently process the sequence in the input weight data. In order to adapt to the data processing rate of DDR3, each computing channel deploys 32 PEs (Processing Elements). The design of a single computing channel is as follows: Figure 2 shown.

[0068] like Figure 2 As shown in the figure, after the calculation of a row is completed, its psum (partial sum) result is stored in the first-in-first-out queue. Since the task allocation mentioned above leads to disordered rows, position blocks are additionally stored in the first-in-first-out queue. In order to ensure the overall calculation efficiency, the first-in-first-out queue provides two signals: fifo_empty (empty) and fifo_half_full (half full). If the queue reaches a half-full state, the "combination" will give priority to reading the queue. Before calculating the activation function, psum should also be added with the offset in the offset buffer. Since the bit width of psum (22bit-64bit) is wider than other signals, the result after the activation function calculation will be stored in buffer i, buffer f, buffer o, and buffer c. After completing the matrix-vector multiplication of a unit, the dot multiplication unit starts to perform dot multiplication. The final output result is stored in the activation buffer of the calculation unit. After the entire calculation process is completed, the output is transmitted back to the data bus.

[0069] It can be seen from the above technical solutions that the present application provides an acceleration system for long short-term memory neural networks, the acceleration system comprising a host and an acceleration device for realizing data interaction with the host. The acceleration device comprises: a channel group containing multiple computing channels, a weight buffer, a bias buffer and an activation buffer connected to the channel group. In addition, the acceleration device also includes a controller and a decoding component. In the present application, the host is used to pre-process the weight data to obtain the first input data, and the first input data is a collection of multiple groups of weight data with balanced number of parameters. The weight buffer is used to receive the first input data, and the first input data can be used for calculation after being decoded by the decoding component. Multiple groups of weight data with balanced number of parameters are respectively allocated to different processing units for calculation, so that the processing time of different processing units is basically the same, thereby solving the problem of hardware load imbalance caused by random sparsity.

[0070] In an embodiment of the present application, preprocessing includes task allocation and weight encoding. In an achievable manner, the preprocessing method for the long short-term memory neural network includes the following steps:

[0071] Step 301, grouping weight data according to the following mathematical model:

[0072]

[0073] st:

[0074]

[0075]

[0076] Among them, N represents N groups, R represents the matrix set corresponding to the weight data, G represents each individual group, L represents the number of non-zero elements in each group, E represents the average number of non-zero elements, and the constraint represents that each group belongs to R and there is no intersection between groups.

[0077] For example, FIG3(a) shows a sparse matrix. After random pruning, row 0 has 3 non-zero elements, row 1 may have 6, row 2 has 5, and row 3 has 2. The 4 rows of data are assigned to a processing unit respectively, and each processing unit processes in parallel. After the second calculation cycle, only processing unit 1 is still working, and the sparsity rate of the neural network cannot be fully utilized. After task redistribution, referring to the schematic diagram shown in FIG3(b), the 4 rows are divided into two calculation groups, group 0 contains row 0 and row 2, and group 1 contains row 1 and row 3. As shown in FIG3(b), the number of non-zero elements in the two groups is equal. When 4 processing units are used to calculate these two groups, there is no extra waiting for clock cycles.

[0078] In the embodiment of the present application, the “task” of task reallocation is a task relative to a processing unit or a computing channel.

[0079] Step 302: Encode the grouped weight data to obtain first input data.

[0080] Optionally, encoding the grouped weight data to obtain first input data includes the following steps:

[0081] Step 3021, encode each non-zero parameter in each row of data to obtain a weight storage block, wherein the weight storage block includes the weight value of the non-zero parameter and a relative position index, wherein the relative position index indicates the index of the non-zero parameter relative to the previous non-zero parameter after grouping.

[0082] In this step, since task redistribution causes disorder among rows, the present application adds a position information block to each row to identify the actual position of the row, and also serves as a separator between rows.

[0083] Step 3022: Add a position information block at the end of each row of data, where the position information block indicates the actual position of the row.

[0084] Optionally, after encoding each non-zero parameter in each row of data to obtain a weight storage block, the following steps are further included:

[0085] Determine whether the size of the relative position index exceeds a preset threshold;

[0086] If exceeded, an index supplement block is added after the weight storage block.

[0087] Optionally, in order to distinguish the location information block from other information, a prompt block is added before the location information block, and the prompt block is used to prompt the location of the location information block.

[0088] Exemplarily, Figure 4(a) and Figure 4(b) respectively show schematic diagrams before and after encoding. The stored weight data contains 4 types of information blocks: the position information block (Weight block) contains the values ​​of all non-zero weights (8 bits) and their relative position indexes (8 bits), the index supplement block (Pad block) is used to supplement the relative position index overflow, the position information block (Position block) represents the actual position of the row, and the prompt block (Row-pointer block) is used to distinguish the Positionblock. After weight encoding, the groups need to be aligned, the alignment width is 64 bits, and the insufficient ones are padded with 0.

[0089] Take a random matrix with a sparsity of 95% as an example. Assuming that the DRAM width is 512 bits, task redistribution divides the matrix into 8 groups. After weight encoding, the total number of modules in each group is shown in the figure below. Figure 5 As shown. Figure 5 It can be seen that the weight data among the groups are very close.

[0090] The present application embodiment provides a preprocessing device for a long short-term memory neural network, the preprocessing device comprising:

[0091] The grouping module is used to group weight data according to the following mathematical model:

[0092]

[0093] st:

[0094]

[0095]

[0096] Among them, N represents N groups, R represents the matrix set corresponding to the weight data, G represents each individual group, L represents the number of non-zero elements in each group, E represents the average number of non-zero elements, and the constraint represents that each group belongs to R and there is no intersection between groups;

[0097] The encoding module is used to encode the grouped weight data to obtain first input data.

[0098] Optionally, the encoding module includes:

[0099] An encoding unit, configured to encode each non-zero parameter in each row of data to obtain a weight storage block, wherein the weight storage block includes a weight value of the non-zero parameter and a relative position index, wherein the relative position index indicates an index of the non-zero parameter relative to a previous non-zero parameter after grouping;

[0100] The first adding unit is used to add a position information block at the end of each line of data, where the position information block indicates the actual position of the line.

[0101] Optionally, the device further comprises:

[0102] A judging unit, configured to judge whether the size of the relative position index exceeds a preset threshold value after the encoding unit encodes each non-zero parameter in each row of data to obtain a weight storage block;

[0103] A second adding unit is configured to add an index supplement block after the weight storage block when the judging unit determines that the size of the relative position index exceeds a preset threshold.

[0104] Optionally, the device further comprises:

[0105] The third adding unit is used to add a prompt block before the position information block, and the prompt block is used to prompt the position of the position information block.

[0106] The present invention provides a decoding method for a long short-term memory neural network. Figure 1 A decoding component in an acceleration system provided in any possible implementation manner, wherein the decoding method comprises the following steps:

[0107] Acquire a data block in the first input data, wherein the data block includes a weight storage block, a position information block, an index supplement block, and a prompt block;

[0108] Decoding is performed according to the type of the data block; wherein, the weight storage block and index supplement block that are adjacent and have a sequence are decoded as the same data block to be decoded, and the absolute index position of the data block to be decoded is equal to the sum of the absolute position of the previous parameter of the data block to be decoded and the relative position index between the two data blocks; if the data block is a position information block, it is output as is; if the data block is a prompt block, the absolute position index of the prompt block is equal to a preset value.

[0109] Figure 6 As an example, a decoding circuit diagram is shown, in which the decoding circuit is responsible for processing four types of encoding: Weight Block, Pad Block, Position Block and Row-pointer Block. Before calculating the multiplication, it is first necessary to determine the encoding type to which the data belongs. The Weight Block and Pad Block are divided into 8-bit weights and 8-bit relative position indexes, the Row-pointer Block is divided into two 8-bit zeros, and the Position Block is output as is. The main purpose of the entire circuit is to convert the relative position index into an absolute position index so that the weight and its corresponding input can be matched during calculation.

[0110] Index calculation module such as Figure 6 As shown in , the absolute index position of each non-zero parameter is calculated by the absolute position of its previous parameter and the relative index between them. The circuit has 4 inputs and outputs. If a module is a Row-pointer Block, its absolute index is 0. Since the relative position index in the first input module is regarded as an absolute position index, an additional accumulation operation is required: by judging whether the four input modules belong to the same row, if so, the circuit will accumulate the absolute index value of the last module in the previous row for the first module in the next row, otherwise it will accumulate 0.

[0111] In order to verify the effect of the acceleration system provided by the present application in solving the hardware balancing problem, the embodiment of the present application is deployed on the Arria 10 SX660 FPGA, and the resource utilization, clock frequency and energy consumption of the proposed architecture are shown in Table 1. The reported results are based on Quartus Pro 18.1. Our architecture has 32 parallel computing channels, each containing 32 PE units, so the architecture contains a total of 1024 PE units, consuming 512 DSPs of Intel Arria 10FPGA. In addition, there are still other modules that consume DSPs, so the total DSP consumption is 536.

[0112] Table 1 Structure Energy Consumption Table

[0113]

[0114] This scheme can accelerate LSTMs of any sparse form. Table 2 shows the performance of our architecture under various sparse forms, including ESE, BBS, and random sparse with different sparsity. The architecture under this scheme only loses about 3% of PE utilization when the sparsity of random sparse increases to more than 90%.

[0115] Table 2 PE utilization comparison table

[0116]

[0117]

[0118] Table 3 Comparison of this solution with ESE and BBS

[0119]

[0120] In summary, this scheme can accelerate any sparse form of LSTM, and the PE utilization rate can reach more than 96%. Compared with previous work, the architecture under this scheme can improve the computing efficiency by 1 to 2.3 times.

[0121] The present application is described in detail above in conjunction with specific implementation methods and exemplary examples, but these descriptions cannot be understood as limiting the present application. Those skilled in the art understand that, without departing from the spirit and scope of the present application, a variety of equivalent replacements, modifications or improvements can be made to the technical solution of the present application and its implementation methods, all of which fall within the scope of the present application. The scope of protection of the present application shall be subject to the attached claims.

Claims

1. An acceleration system for long short-term memory neural networks, characterized in that: include: A host and an acceleration device for realizing data interaction with the host, wherein the host is used to provide data to be calculated to the acceleration device, wherein the data to be calculated includes weight data obtained after random sparseness in a long short-term memory neural network, and first input data obtained by preprocessing the weight data, wherein the first input data is a set of multiple groups of weight data with balanced number of parameters, and the number of groups in the set is not greater than the number of rows of a matrix corresponding to the weight data; wherein the weight data is derived from model compression of the long short-term memory neural network, and the objects of model compression include words and sentences in machine translation or speech recognition, or data in price prediction; The acceleration device comprises: a channel group including a plurality of computing channels, a weight buffer, a bias buffer and an activation buffer connected to the channel group; wherein, The weight buffer is used to receive and store the first input data; The bias buffer is used to receive and store second input data after processing the weight data; The activation buffer is used to receive the weight data and the output result of the channel group, and store the output result as the third input data for the next calculation; Each of the calculation channels comprises a plurality of processing units, and the processing units are used to receive various types of input data transmitted by the weight buffer, the bias buffer, and the activation buffer, so as to perform calculations using the various types of input data; The acceleration device also includes a controller and a decoding component; the controller is connected to the weight buffer, bias buffer, activation buffer and channel group, and is used to update the input data and control the channel group to perform calculations before and after each calculation; the decoding component is arranged between the weight buffer and the channel group, and is used to decode the first input data to obtain multiple groups of weight data with balanced parameter numbers, so that each of the processing units can process the weight data respectively.

2. The acceleration system for long short-term memory neural network according to claim 1, characterized in that: The pretreatment method comprises: The weight data are grouped according to the following mathematical model: Among them, N represents N groups, R represents the matrix set corresponding to the weight data, G represents each individual group, L represents the number of non-zero elements in each group, E represents the average number of non-zero elements, and the constraint represents that each group belongs to R and there is no intersection between groups; The grouped weight data is encoded to obtain first input data.

3. The acceleration system for long short-term memory neural network according to claim 2, characterized in that: The grouped weight data is encoded to obtain first input data, including: Encode each non-zero parameter in each row of data to obtain a weight storage block, wherein the weight storage block includes a weight value of the non-zero parameter and a relative position index, wherein the relative position index indicates an index of the non-zero parameter relative to a previous non-zero parameter after grouping; A position information block is added at the end of each row of data, which indicates the actual position of the row.

4. The acceleration system for long short-term memory neural network according to claim 3, characterized in that: After encoding each non-zero parameter in each row of data and obtaining the weight storage block, it also includes: Determine whether the size of the relative position index exceeds a preset threshold; If exceeded, an index supplement block is added after the weight storage block.

5. The acceleration system for long short-term memory neural network according to claim 3, characterized in that: The pre-processing method further comprises: A prompt block is added before the position information block, and the prompt block is used to prompt the position of the position information block.

6. An acceleration system for a long short-term memory neural network, characterized in that: include: A host and an acceleration device for realizing data interaction with the host, wherein the host is used to provide data to be calculated to the acceleration device, wherein the data to be calculated includes weight data obtained after random sparseness in a long short-term memory neural network, and first input data obtained by preprocessing the weight data, wherein the first input data is a set of multiple groups of weight data with balanced parameter numbers, and the number of groups in the set is not greater than the number of rows of a matrix corresponding to the weight data; wherein the weight data is derived from a model compression of the long short-term memory neural network, and the object of the model compression includes words and sentences in machine translation or speech recognition, or data in price prediction; the host includes a preprocessing device, and the preprocessing device includes: The grouping module is used to group weight data according to the following mathematical model: Among them, N represents N groups, R represents the matrix set corresponding to the weight data, G represents each individual group, L represents the number of non-zero elements in each group, E represents the average number of non-zero elements, and the constraint represents that each group belongs to R and there is no intersection between groups; An encoding module, used for encoding the grouped weight data to obtain first input data; The acceleration device comprises: a channel group including a plurality of computing channels, a weight buffer, a bias buffer and an activation buffer connected to the channel group; wherein, The weight buffer is used to receive and store the first input data; The bias buffer is used to receive and store second input data after processing the weight data; The activation buffer is used to receive the weight data and the output result of the channel group, and store the output result as the third input data for the next calculation; Each of the calculation channels comprises a plurality of processing units, and the processing units are used to receive various types of input data transmitted by the weight buffer, the bias buffer, and the activation buffer, so as to perform calculations using the various types of input data; The acceleration device also includes a controller and a decoding component; the controller is connected to the weight buffer, bias buffer, activation buffer and channel group, and is used to update the input data and control the channel group to perform calculations before and after each calculation; the decoding component is arranged between the weight buffer and the channel group, and is used to decode the first input data to obtain multiple groups of weight data with balanced parameter numbers, so that each of the processing units can process the weight data respectively.

7. The acceleration system for long short-term memory neural network according to claim 6, characterized in that: The encoding module comprises: An encoding unit, used to encode each non-zero parameter in each row of data to obtain a weight storage block, wherein the weight storage block includes a weight value of the non-zero parameter and a relative position index, wherein the relative position index indicates an index of the non-zero parameter relative to a previous non-zero parameter after grouping; The first adding unit is used to add a position information block at the end of each line of data, where the position information block indicates the actual position of the line.

8. The acceleration system for long short-term memory neural network according to claim 7, characterized in that: The pre-processing device also includes: A judging unit, configured to judge whether the size of the relative position index exceeds a preset threshold value after the encoding unit encodes each non-zero parameter in each row of data to obtain a weight storage block; A second adding unit is configured to add an index supplement block after the weight storage block when the judging unit determines that the size of the relative position index exceeds a preset threshold.

9. The acceleration system for long short-term memory neural network according to claim 7, characterized in that: The pre-processing device also includes: The third adding unit is used to add a prompt block before the position information block, and the prompt block is used to prompt the position of the position information block.

10. A method for accelerating a long short-term memory neural network, characterized in that: include: A host and an acceleration device for realizing data interaction with the host, wherein the host is used to provide data to be calculated to the acceleration device, wherein the data to be calculated includes weight data obtained after random sparseness in a long short-term memory neural network, and first input data obtained by preprocessing the weight data, wherein the first input data is a set of multiple groups of weight data with balanced number of parameters, and the number of groups in the set is not greater than the number of rows of a matrix corresponding to the weight data; wherein the weight data is derived from model compression of the long short-term memory neural network, and the objects of model compression include words and sentences in machine translation or speech recognition, or data in price prediction; The acceleration device comprises: a channel group including a plurality of computing channels, a weight buffer, a bias buffer and an activation buffer connected to the channel group; wherein, The weight buffer is used to receive and store the first input data; The bias buffer is used to receive and store second input data after processing the weight data; The activation buffer is used to receive the weight data and the output result of the channel group, and store the output result as the third input data for the next calculation; Each of the calculation channels comprises a plurality of processing units, and the processing units are used to receive various types of input data transmitted by the weight buffer, the bias buffer, and the activation buffer, so as to perform calculations using the various types of input data; The acceleration device also includes a controller and a decoding component; the controller is connected to the weight buffer, the bias buffer, the activation buffer and the channel group, and is used to update the input data and control the channel group to perform calculations before and after each calculation ends; the decoding component is arranged between the weight buffer and the channel group, and is used to decode the first input data to obtain multiple groups of weight data with balanced parameter quantities, so that each of the processing units can process the weight data respectively; wherein the decoding method includes: Acquire a data block in the first input data, wherein the data block includes a weight storage block, a position information block, an index supplement block, and a prompt block; Decoding is performed according to the type of the data block; wherein, the weight storage block and index supplement block that are adjacent and have a sequence are decoded as the same data block to be decoded, and the absolute index position of the data block to be decoded is equal to the sum of the absolute position of the previous parameter of the data block to be decoded and the relative position index between the two data blocks; if the data block is a position information block, it is output as is; if the data block is a prompt block, the absolute position index of the prompt block is equal to a preset value.

Images