LSTM implementation method based on SIMD instruction

Abstract

The invention provides an SIMD (Single Instruction Multiple Data) instruction-based LSTM (Long Short Term Memory) implementation method, which comprises the following steps: S1, a first-layer cycle is a cycle of a time sequence, the value of one time sequence is calculated each time to obtain a result, and the cycle is counted for times; s2, the second layer of circulation is circulation of output channels, the number of the output channels is output channels, and the output channels are output channels / 16 times of circulation in total; calculating data pointers of four weights in each cycle; the sum I, the sum J, the sum F and the sum O need to be reset to 0 in each cycle; and S3, the third layer of circulation is divided into two circulation, and the first circulation is an input data data circulation. The second cycle is a cycle of input data; the second cycle is a cycle of input data; and S4, after the third layer of circulation is finished, continuously calculating the sum I, the sum J, the sum F and the sum O in the second layer of circulation. And when the second-layer cycle is ended, information of all output channels of the current time sequence is obtained, and then the next time sequence cycle, namely the first-layer cycle, is carried out until the first-layer cycle is ended. According to the method, SIMD instructions are combined and used to complete the same function as the LSTM realized by Python.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural network, and in particular relates to an LSTM implementation method based on SIMD instructions. Background Art

[0002] In traditional neural networks, the model does not pay attention to what information can be used for the next moment from the previous moment's processing. It only pays attention to the processing of the current moment each time. For example, if we want to classify the events that appear at each moment in a movie, if we know the event information before the movie, then it will be very easy to classify the events at the current moment. In fact, traditional neural networks have no memory function, so when they classify the events that appear at each moment, they will not use the information that has already appeared in the movie. Recurrent Neural Networks (RNNs) recursive neural networks allow neural networks to remember this information. The results of recursive neural networks are somewhat different from traditional neural networks. It has a loop pointing to itself, which is used to indicate that it can pass the information processed at the current moment to the next moment. Because recursive neural networks have a certain memory function, they can be used to solve many problems, such as speech recognition, language models, machine translation, etc. However, they cannot handle long-term dependency problems well.

[0003] The long-term dependency problem refers to the difficulty in learning relevant information when the prediction point is far away from the dependent related information. For example, in the sentence "I was born in France, ..., I speak French", to predict the last "French", we need to use the context "France". In theory, recursive neural networks can handle such problems, but in practice, the problems with RNN are gradient vanishing and gradient explosion. Since the gradient vanishing only has short-term memory, it cannot solve the long-term dependency problem. The problem of gradient vanishing can be improved by selecting a better activation function or changing the network structure (such as LSTM). LSTM brings addition operations into the network through sophisticated gate control, which solves the problem of gradient vanishing to a certain extent.

[0004] The core of LSTM is as follows Figure 1 As shown in the figure below, it is called a memory block, which mainly contains three gates (forget gate, input gate, output gate) and a memory cell. t That is, cell, input from below (h t-1 , x t ) to output h t A line is the cell state, which is like a conveyor belt that controls the transmission of information to the next moment. t It is ct The candidate input is the candidate value generated by the tanh layer. t ,i t , o t They are the forget gate, input gate, and output gate, which are represented by sigmoid layers. The output of the gate is a number between 0 and 1, indicating how much information is allowed to pass through. 0 means that it is not allowed to pass through at all, and 1 means that it is allowed to pass through completely. LSTM can add and delete information to cells through gating units, and the gate can selectively decide whether information passes through.

[0005] The first step of LSTM is controlled by the "forget gate" layer through sigmoid, which will be based on the output h of the previous moment t-1 and the current input x t To generate an f between 0 and 1 t value, to decide whether to let the information C learned at the last moment t-1 Pass or partially pass. For example, a lot of things have been learned from the previous sentences, and some things are useless at present, so they can be selectively filtered.

[0006] The second step of LSTM consists of two parts. The first is that the "input gate" layer uses sigmoid to decide which values ​​to update. The second is a tanh layer to generate new candidate values ​​c_int, which may be added to the cell state as the candidate values ​​generated by the current layer.

[0007] Steps 1 and 2 are combined to update the old cell state. First, multiply the old cell state by f t to forget the unnecessary information, and then communicate with it*c_in t The results are added together to obtain the new cell state c t . This is the process of discarding unnecessary information and adding new information.

[0008] The last step of LSTM is to determine the output of the model. First, a sigmoid layer is used to get an initial output, and then tanh is used to transform c t The values ​​are scaled to between -1 and 1, and then multiplied with the initial output pair by pair to get the output of the model. Figure 1 shown.

[0009] Currently, there are interfaces for building LSTM in the deep learning frameworks pytorch and tensorflow.

[0010] The LSTM in the existing deep learning framework is implemented based on Python language and can run on the PC side. However, for some chip manufacturers' chips used to run deep learning neural networks, such as the chips of Beijing Ingenic Integrated Circuit Co., Ltd. (hereinafter referred to as Beijing Ingenic), the algorithms implemented in Python language cannot run. In addition, Python's underlying language is C language. Running C language on a platform or chip with limited resources will cause a lot of cache waste, resulting in slow computing speed. For some applications that require computing speed, the algorithms implemented in Python are not the best choice.

[0011] In addition, commonly used technical terms include:

[0012] 1. SIMD stands for Single Instruction Multiple Data, which is a set of instructions that can process multiple operands at a time and pack them into large registers.

[0013] 2.RNN stands for Recurrent Neural Network. RNN is a network that specializes in processing sequence data. It is the most natural neural network architecture for sequence data and has been applied in speech recognition, language translation, and image description.

[0014] 3. LSTM stands for Long Short Term Memory network. It is an improved recursive neural network that can solve the problem that RNN cannot handle long-distance dependencies. It is currently quite popular. Summary of the invention

[0015] In order to solve the above problems, the purpose of this application is to propose a method for implementing LSTM based on the SIMD instruction set, which can efficiently run the LSTM algorithm on specific chips (such as the chip used by Beijing Junzheng to run deep learning neural networks) by combining SIMD instructions to achieve the same function as the LSTM implemented in Python.

[0016] Specifically, the present invention provides a LSTM implementation method based on SIMD instructions, the method comprising the following steps:

[0017] S1, the first layer of loop is the time series loop, each time a time series value is calculated to get the result, a total of time_sequence loops, expressed as:

[0018] for(int t=0;t <time_sequence;t++);

[0019] S2, executes the second layer loop, which is the loop of the output channel. There are output_channel output channels. Since the data type is int8, the Ingenic_simd128 instruction set processes 128-bit data at a time, so it can process 16 int8 data at a time, a total of output_channel / 16 loops; expressed as: for(int j=0; j<=output_channel-16; j+=16);

[0020] Each loop first calculates the data pointers of the four weights, expressed as:

[0021] int8_t*weight_I_ptr=weight_i+j*ic;weight_i is the starting pointer of weight i, each output channel corresponds to ic weights, when the output channel loops to j, it corresponds to j*ic weights, so the weight pointer of this loop is the starting pointer weight_i plus j*ic; similarly, int8_t*weight_J_ptr=weight_j+j*ic;

[0022] int8_t*weight_F_ptr=weight_f+j*ic;

[0023] int8_t*weight_O_ptr=weight_o+j*ic;

[0024] Each time the loop is completed, sum_I, sum_J, sum_F, and sum_O need to be reset to 0, expressed as: sum_I = 0;

[0025] sum_J=0;

[0026] sum_F = 0;

[0027] sum_O = 0;

[0028] S3, the third layer loop is divided into two loops because there are two input data input_data and input_hidden;

[0029] The first loop is to loop over the input_data data. Similarly, 16 int8 data are processed at a time, with input_channel / 16 loops, expressed as:

[0030] for(int k=0; k<=input_channel-16; k+=16);

[0031] Use the Ingenic_simd128_load data loading instruction to load 16 input_data data into the register in at a time, expressed as: in = Ingenic_simd128_load (input_data);

[0032] Use the Ingenic_simd128_fwsum multiplication and addition instruction to load 16 weight data from weight_I_ptr, multiply them with the 16 input data in in, and then add them with the 16 data in sum_I. sum_I is the accumulated value after each loop, so after the first loop, sum_I contains the sum value calculated by the 16 output channels for input_data, expressed as: sum_I = Ingenic_simd128_fwsum (sum_I, in, weight_I_ptr);

[0033] Similarly, the sum_J, sum_F, sum_O values ​​corresponding to the 16 output channels are obtained, expressed as: sum_J = Ingenic_simd128_fwsum(sum_J, in, weight_J_ptr);

[0034] sum_F=Ingenic_simd128_fwsum(sum_F,in,weight_F_ptr);

[0035] sum_O=Ingenic_simd128_fwsum(sum_O,in,weight_O_ptr);

[0036] The second loop is for the input_hidden data loop. Similarly, 16 int8 data are processed at a time, with hidden_channel / 16 loops, expressed as:

[0037] for(int k=0; k<=hidden_channel-16; k+=16);

[0038] Use the Ingenic_simd128_load instruction to load the input_hidden data into the in_H register, expressed as:

[0039] in_H=Ingenic_simd128_load(input_hidden);

[0040] Use the Ingenic_simd128_fwsum instruction to multiply weight and input_hidden data and then add it to the sum value. The sum value here is the sum value calculated by the previous input_data loop. After the second loop, sum_I is the cumulative value calculated by the 16 output channels for input_data and input_hidden, expressed as:

[0041] sum_I=Ingenic_simd128_fwsum(sum_I,in_H,weight_I_ptr);

[0042] Similarly, the sum_J, sum_F, sum_O values ​​corresponding to the 16 output channels can be obtained, expressed as: sum_J = Ingenic_simd128_fwsum(sum_J, in_H, weight_J_ptr);

[0043] sum_F=Ingenic_simd128_fwsum(sum_F,in_H,weight_F_ptr);

[0044] sum_O=Ingenic_simd128_fwsum(sum_O,in_H,weight_O_ptr);

[0045] S4, after the third loop is finished, sum_I, sum_J, sum_F, sum_O are continued to be calculated in the second loop;

[0046] Use the Ingenic_simd128_sigmoid instruction, which is a singmoid instruction, to perform sigmoid calculations on the 16 sum values ​​in the sum_I register, and store the results in the quant_I register, expressed as:

[0047] quant_I=Ingenic_simd128_sigmoid(sum_I);

[0048] Similarly, quant_J, quant_F, and quant_O are expressed as:

[0049] quant_J=Ingenic_simd128_sigmoid(sum_J);

[0050] quant_F=Ingenic_simd128_sigmoid(sum_F);

[0051] quant_O=Ingenic_simd128_sigmoid(sum_O);

[0052] Load 16 input_cell data into register in_C, expressed as:

[0053] in_C=Ingenic_simd128_load(input_cell);

[0054] Use the Ingenic_simd128_mul multiplication instruction to multiply the 16 numbers in quant_F with the 16 numbers in in_C, and save the result in register fc, expressed as: fc = Ingenic_simd128_mul (quant_F, in_C);

[0055] Use the Ingenic_simd128_tanh instruction, which is a tanh instruction, to perform tanh calculation on the 16 values ​​in quant_J, and the result is saved in register act_J, which is expressed as: act_J = Ingenic_simd128_tanh (quant_J);

[0056] The corresponding multiplication result of the values ​​of quant_I and quant_J is stored in register ij, which is expressed as: ij = Ingenic_simd128_mul (quant_I, quant_J);

[0057] Use the Ingenic_simd128_add addition instruction to add the 16 numbers in fc to the 16 numbers in ij, and save the result in register new_c, which is expressed as:

[0058] new_c=Ingenic_simd128_add(fc,ij);

[0059] The tanh calculation is performed on the 16 values ​​in new_c, and the result is saved in the register act_new_c, which is expressed as:

[0060] act_new_c=Ingenic_simd128_tanh(new_c);

[0061] The corresponding multiplication result of the values ​​of quant_O and act_new_c is stored in register new_h, expressed as:

[0062] new_h=Ingenic_simd128_mul(quant_O,act_new_c);

[0063] Use the Ingenic_simd128_store data storage instruction to save the 16 data in register new_h to the output data pointer out_ptr, expressed as:

[0064] out_ptr=Ingenic_simd128_store(new_h);

[0065] At this point, the second layer loop ends, and the information of all output output_channels of the current time series has been obtained. Then the next time series loop, i.e., the first layer loop of step S1, is performed until the first layer loop ends.

[0066] The SIMD instruction set used in the method is the Ingenic_simd128 instruction set, which can load 128-bit data in a register at a time and perform the same operation on 128-bit data at a time during calculation. The instruction set includes data loading and data storage instructions, arithmetic operation addition, subtraction, and multiplication instructions, logical operation and, or, not, and exclusive or instructions, and shift operation logical left shift, logical right shift, and arithmetic right shift instructions.

[0067] In the method, time_sequence indicates that there are a total of time_sequence time series, that is, feature information of time_sequence moments needs to be calculated;

[0068] The number of output channels for each sequence is output_channel, that is, each sequence needs to output output_channel feature information;

[0069] The number of input channels for each sequence is input_channel, that is, each sequence inputs input_channel feature information, and the input data input_data type is signed 8 bits, that is, int8_t;

[0070] The h value of the previous sequence is also used as the input of this sequence and is called the hidden layer. The channel of the hidden layer is hidden_channel, which means there are hidden_channel feature information. Therefore, the input of each sequence consists of two parts, including input data input_data and hidden layer input data input_hidden. The total number of input channels ic = input_channel + hidden_channel;

[0071] Each output channel corresponds to ic weights; three gates f are required t ,i t , o t and a candidate input c_in tParticipate in the calculation to get the output result; weight_f, weight_i, weight_o correspond to the weights of the three gates respectively, and weight_j corresponds to c_in t weight, weight_I_ptr, weight_J_ptr, weight_F_ptr, weight_O_ptr are the data pointers corresponding to the four weights;

[0072] sum_I, sum_J, sum_F, sum_O are the values ​​after multiplying the input data by the corresponding weights;

[0073] quant_I, quant_J, quant_F, and quant_O are the values ​​obtained by performing sigmoid calculation on the four sum values. The values ​​obtained by sigmoid calculation range from 0 to 1, indicating how much information is retained.

[0074] The cell value of the previous time series is also used as another input of this sequence, namely input_cell; fc represents the information retained by the forget gate for input_cell; quant_J is tanhed and then multiplied by quant_I to obtain a new weight ij; weight ij is multiplied by the retained information fc to select the information again to obtain a new cell, namely new_c; tanh operation is performed on the new cell and then multiplied by quant_O to obtain new h data, namely new_h, where weight ij is a variable name, which is the weight ij obtained by multiplying quant_J by quant_I.

[0075] Therefore, the advantages of this application are:

[0076] 1. The LSTM algorithm is implemented using the SIMD instruction set, especially the Ingenic_simd128 instruction set in the chip developed by Beijing Ingenic. The LSTM algorithm can be run on the Beijing Ingenic T31 chip using the Ingenic_simd128 instruction set, and the results are consistent with the LSTM algorithm implemented in Python.

[0077] 2. The instruction set, especially the Ingenic_simd128 instruction set, can load 128-bit data at a time. One instruction can calculate 128-bit data at the same time, which greatly improves the calculation speed. It can run efficiently on chips with limited hardware resources and high calculation speed requirements. By comparison, under the same chip conditions, the LSTM algorithm implemented by SIMD is 5 times faster than the LSTM algorithm implemented by Python. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0079] Figure 1 This is a schematic diagram of the core memory block of LSTM.

[0080] Figure 2 This is the core code diagram of this method.

[0081] Figure 3 is a schematic diagram of the process of this method. DETAILED DESCRIPTION

[0082] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0083] The present application relates to a LSTM implementation method based on SIMD instructions. The SIMD instruction set used in this method is the Ingenic_simd128 instruction set, which can load 128 bits of data in a register at a time, and can perform the same operation on 128 bits of data at a time during calculation. The instruction set includes data loading and data storage instructions; arithmetic operations such as addition, subtraction, and multiplication instructions; logical operations such as AND, OR, NOT, and XOR instructions; and shift operations such as logical left shift, logical right shift, and arithmetic right shift. The core code implemented in this method is as follows: Figure 2 shown.

[0084] In this method, time_sequence indicates that there are a total of time_sequence time series, that is, the feature information of time_sequence moments needs to be calculated; the number of output channels of each sequence is output_channel, that is, each sequence needs to output output_channel feature information; the number of input channels of each sequence is input_channel, that is, each sequence inputs input_channel feature information, and the input data input_data type is signed 8bit (int8_t); the h value of the previous sequence is also called the hidden layer as the input of this sequence, and the channel of the hidden layer is hidden_channel, indicating that there are hidden_channel feature information, so the input of each sequence consists of two parts, including input data input_data and hidden layer input input_hidden, and the total number of input channels ic = input_channel + hidden_channel; each output channel has ic weights, which are given by Figure 1 It can be seen that three gates f are required to obtain the final output t ,i t , o tand a candidate input c_in t Participate in the calculation, so there are 4 weights, namely weight_f, weight_i, weight_o correspond to the weights of the three gates respectively, and weight_j is c_in t weights, weight_I_ptr, weight_J_ptr, weight_F_ptr, weight_O_ptr are the data pointers corresponding to the four weights; sum_I, sum_J, sum_F, sum_O are the values ​​after the input data is multiplied by the corresponding weights; quant_I, quant_J, quant_F, quant_O are the values ​​after the sigmoid calculation of the four sum values, which are 0 to 1 after the sigmoid calculation, indicating the weight of how much information is retained; the cell value of the previous time series is also used as another input of this sequence, namely input_cell; fc represents the information retained by the forget gate for the input_cell data; quant_J is tanhed and then multiplied by quant_I to obtain the new control weight ij; the weight ij is multiplied by the retained information fc to reselect the information and retain the new cell, namely new_c; the new cell is tanhed and then multiplied by quant_O to obtain the new h data, namely new_h.

[0085] like Figure 3 As shown, the specific implementation steps are as follows:

[0086] S1, the first layer of loop is the time series loop, each time a time series value is calculated to get the result, so the total loop is time_sequence times.

[0087] S2, the second loop is the loop of the output channel. There are output_channel output channels. Since the data type is int8, the Ingenic_simd128 instruction set can process 128-bit data at a time, so it can process 16 int8 data at a time, a total of output_channel / 16 loops; each loop first calculates the data pointers of the four weights, such as int8_t*weight_I_ptr=weight_i+j*ic; weight_i is the starting pointer, each output channel corresponds to ic weights, when the output channel loops to j, it corresponds to j*ic weights, so the weight pointer of this loop is the starting pointer weight_i plus j*ic; each loop needs to reset sum_I, sum_J, sum_F, sum_O to 0;

[0088] S3, the third layer loop is divided into two loops, because there are two input data input_data and input_hidden. The first loop is the loop of input_data data, and similarly, 16 int8 data are processed at a time, with input_channel / 16 loops;

[0089] In in=Ingenic_simd128_load(input_data), Ingenic_simd128_load is a data loading instruction, which loads 16 input_data data into register in at a time.

[0090] In sum_I=Ingenic_simd128_fwsum(sum_I,in,weight_I_ptr), Ingenic_simd128_fwsum is a multiplication and addition instruction, which loads 16 weight data from weight_I_ptr, multiplies them with the 16 input data in in, and then adds them with the 16 data in sum_I. sum_I is the accumulated value after each loop, so after the first loop, sum_I contains the sum value calculated by the 16 output channels for input_data. Similarly, the sum_J, sum_F, and sum_O values ​​corresponding to these 16 output channels can be obtained.

[0091] The second loop is for the input_hidden data loop. Similarly, 16 int8 data are processed at a time, with hidden_channel / 16 loops. The input_hidden data is loaded into the in_H register using the Ingenic_simd128_load instruction. The weight is multiplied by the input_hidden data using the Ingenic_simd128_fwsum instruction and then added to the sum value. The sum value here is the sum value calculated by the previous input_data loop. After the second loop ends, sum_I contains the accumulated value calculated by the 16 output channels for input_data and input_hidden. Similarly, the sum_J, sum_F, and sum_O values ​​corresponding to these 16 output channels can be obtained.

[0092] S4, after the third loop ends, sum_I, sum_J, sum_F, sum_O are continued to be calculated in the second loop.

[0093] In quant_I=Ingenic_simd128_sigmoid(sum_I), Ingenic_simd128_sigmoid is a singmoid instruction, which performs sigmoid calculation on the 16 sum values ​​in the register sum_I and stores the result in the quant_I register. Similarly, quant_J, quant_F, and quant_O can be obtained.

[0094] in_C=Ingenic_simd128_load(input_cell) means loading 16 input_cell data into register in_C.

[0095] In fc=Ingenic_simd128_mul(quant_F,in_C), Ingenic_simd128_mul is a multiplication instruction, which multiplies the 16 numbers in quant_F with the 16 numbers in in_C respectively, and saves the result in register fc.

[0096] In act_J=Ingenic_simd128_tanh(quant_J), Ingenic_simd128_tanh is a tanh instruction, which performs tanh calculation on the 16 values ​​in quant_J, and the result is saved in register act_J.

[0097] ij=Ingenic_simd128_mul(quant_I,quant_J) means that the corresponding multiplication result of the values ​​of quant_I and quant_J is saved in register ij.

[0098] In new_c=Ingenic_simd128_add(fc,ij), Ingenic_simd128_add is an addition instruction, which adds the 16 numbers in fc to the 16 numbers in ij correspondingly, and saves the result in register new_c.

[0099] act_new_c=Ingenic_simd128_tanh(new_c) means that tanh calculation is performed on the 16 values ​​in new_c, and the result is saved in register act_new_c.

[0100] new_h=Ingenic_simd128_mul(quant_O,act_new_c) means that the corresponding multiplication result of the values ​​of quant_O and act_new_c is saved in register new_h.

[0101] In out_ptr=Ingenic_simd128_store(new_h), Ingenic_simd128_store is a data storage instruction, which saves the 16 data in the register new_h to the output data pointer out_ptr.

[0102] At this point, the second layer loop ends, and the information of all output output_channels of the current time series has been obtained. Then the next time series loop, that is, the first layer loop, is performed until the first layer loop ends.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A LSTM implementation method based on SIMD instructions, characterized in that: The method comprises the following steps: S1, the first layer of loop is the time series loop, each time a time series value is calculated to get the result, a total of time_sequence loops, expressed as: for(int t=0;t <time_sequence;t++); S2, executes the second layer loop, which is the loop of the output channel. There are output_channel output channels. Since the data type is int8, the Ingenic_simd128 instruction set processes 128-bit data at a time, so it can process 16 int8 data at a time, a total of output_channel / 16 loops; expressed as: for(int j=0; j<=output_channel-16; j+=16); Each loop first calculates the data pointers of the four weights, expressed as: int8_t*weight_I_ptr=weight_i+j*ic;weight_i is the starting pointer of weight i, each output channel corresponds to ic weights, when the output channel loops to j, it corresponds to j*ic weights, so the weight pointer of this loop is the starting pointer weight_i plus j*ic; similarly, int8_t*weight_J_ptr=weight_j+j*ic; int8_t*weight_F_ptr=weight_f+j*ic; int8_t*weight_O_ptr=weight_o+j*ic; Each time in the loop, sum_I, sum_J, sum_F, sum_O need to be reset to 0, expressed as: sum_I = 0; sum_J=0; sum_F = 0; sum_O = 0; S3, the third layer loop is divided into two loops because there are two input data input_data and input_hidden; The first loop is to loop over input_data. Similarly, 16 int8 data are processed at a time, with input_channel / 16 loops, expressed as: for(int k=0; k<=input_channel-16; k+=16); Use the Ingenic_simd128_load data loading instruction to load 16 input_data data into the register in at a time, expressed as: in = Ingenic_simd128_load (input_data); Use the Ingenic_simd128_fwsum multiplication and addition instruction to load 16 weight data from weight_I_ptr, multiply them with the 16 input data in in, and then add them with the 16 data in sum_I. sum_I is the accumulated value after each loop. So after the first loop, sum_I contains the sum value calculated by the 16 output channels for input_data, expressed as: sum_I=Ingenic_simd128_fwsum(sum_I,in,weight_I_ptr); Similarly, the sum_J, sum_F, sum_O values ​​corresponding to the 16 output channels are obtained, expressed as: sum_J = Ingenic_simd128_fwsum(sum_J, in, weight_J_ptr); sum_F=Ingenic_simd128_fwsum(sum_F,in,weight_F_ptr); sum_O=Ingenic_simd128_fwsum(sum_O,in,weight_O_ptr); The second loop is for the input_hidden data loop. Similarly, 16 int8 data are processed at a time, with hidden_channel / 16 loops, expressed as: for(int k=0; k<=hidden_channel-16; k+=16); Use the Ingenic_simd128_load instruction to load the input_hidden data into the in_H register, expressed as: in_H=Ingenic_simd128_load(input_hidden); Use the Ingenic_simd128_fwsum instruction to multiply weight and input_hidden data and then add it to the sum value. The sum value here is the sum value calculated by the previous input_data loop. After the second loop, sum_I is the cumulative value calculated by the 16 output channels for input_data and input_hidden, expressed as: sum_I=Ingenic_simd128_fwsum(sum_I,in_H,weight_I_ptr); Similarly, the sum_J, sum_F, sum_O values ​​corresponding to the 16 output channels can be obtained, expressed as: sum_J = Ingenic_simd128_fwsum(sum_J, in_H, weight_J_ptr); sum_F=Ingenic_simd128_fwsum(sum_F,in_H,weight_F_ptr); sum_O=Ingenic_simd128_fwsum(sum_O,in_H,weight_O_ptr); S4, after the third loop is finished, sum_I, sum_J, sum_F, sum_O are continued to be calculated in the second loop; Use the Ingenic_simd128_sigmoid instruction, which is a singmoid instruction, to perform sigmoid calculations on the 16 sum values ​​in the sum_I register, and store the results in the quant_I register, expressed as: quant_I=Ingenic_simd128_sigmoid(sum_I); Similarly, quant_J, quant_F, and quant_O are expressed as: quant_J=Ingenic_simd128_sigmoid(sum_J); quant_F=Ingenic_simd128_sigmoid(sum_F); quant_O=Ingenic_simd128_sigmoid(sum_O); Load 16 input_cell data into register in_C, expressed as: in_C=Ingenic_simd128_load(input_cell); Use the Ingenic_simd128_mul multiplication instruction to multiply the 16 numbers in quant_F with the 16 numbers in in_C. The result is stored in register fc, which is expressed as: fc=Ingenic_simd128_mul(quant_F,in_C); Use the Ingenic_simd128_tanh instruction, which is a tanh instruction, to perform tanh calculations on the 16 values ​​in quant_J. The results are stored in register act_J, expressed as: act_J=Ingenic_simd128_tanh(quant_J); The corresponding multiplication result of the values ​​of quant_I and quant_J is stored in register ij, expressed as: ij=Ingenic_simd128_mul(quant_I,quant_J); Use the Ingenic_simd128_add addition instruction to add the 16 numbers in fc to the 16 numbers in ij, and save the result in register new_c, which is expressed as: new_c=Ingenic_simd128_add(fc,ij); The tanh calculation is performed on the 16 values ​​in new_c, and the result is saved in the register act_new_c, which is expressed as: act_new_c=Ingenic_simd128_tanh(new_c); The corresponding multiplication result of the values ​​of quant_O and act_new_c is stored in register new_h, expressed as: new_h=Ingenic_simd128_mul(quant_O,act_new_c); Use the Ingenic_simd128_store data storage instruction to save the 16 data in register new_h to the output data pointer out_ptr, expressed as: out_ptr=Ingenic_simd128_store(new_h); At this point, the second layer loop ends, and the information of all output output_channels of the current time series has been obtained. Then the next time series loop, i.e., the first layer loop of step S1, is performed until the first layer loop ends.

2. The LSTM implementation method based on SIMD instructions according to claim 1, characterized in that: The SIMD instruction set used in the method is the Ingenic_simd128 instruction set, which can load 128-bit data in a register at a time and perform the same operation on 128-bit data at a time during calculation. The instruction set includes data loading and data storage instructions, arithmetic operation addition, subtraction, and multiplication instructions, logical operation and, or, not, and exclusive or instructions, and shift operation logical left shift, logical right shift, and arithmetic right shift instructions.

3. The LSTM implementation method based on SIMD instructions according to claim 1, characterized in that: In the method, time_sequence indicates that there are a total of time_sequence time series, that is, feature information of time_sequence moments needs to be calculated; the number of output channels of each sequence is output_channel, that is, each sequence needs to output output_channel feature information; The number of input channels for each sequence is input_channel, that is, each sequence inputs input_channel feature information, and the input data input_data type is signed 8 bits, that is, int8_t; The h value of the previous sequence is also used as the input of this sequence and is called the hidden layer. The channel of the hidden layer is hidden_channel, which means there are hidden_channel feature information. Therefore, the input of each sequence consists of two parts, including input data input_data and hidden layer input data input_hidden. The total number of input channels ic = input_channel + hidden_channel; Each output channel corresponds to ic weights; three gates f are required t ,i t , o t and a candidate input c_in t Participate in the calculation to get the output result; weight_f, weight_i, weight_o correspond to the weights of the three gates respectively, and weight_j corresponds to c_in t weight, weight_I_ptr, weight_J_ptr, weight_F_ptr, weight_O_ptr are the data pointers corresponding to the four weights; sum_I, sum_J, sum_F, sum_O are the values ​​after multiplying the input data with the corresponding weights; quant_I, quant_J, quant_F, quant_O are the values ​​after sigmoid calculation of the four sum values. The values ​​obtained by sigmoid calculation are 0 to 1, indicating how much information is retained; the cell value of the previous time series is also used as another input of this series, namely input_cell; fc represents the information retained by the forget gate for input_cell; quant_J is tanhed and then multiplied by quant_I to obtain a new weight ij; weight ij is multiplied by the retained information fc to select the information again to obtain a new cell, namely new_c; the new cell is tanhed and multiplied by quant_O to obtain a new h data, namely new_h.

Images