NPU, equipment and storage media

Abstract

The present application relates to the field of chip technology, and provides an NPU, a device and a storage medium, wherein the NPU includes: obtaining the ith initial data, processing the ith initial data based on the first FC layer, and obtaining the ith intermediate data; the ith initial data includes the ith weight data and the ith image data; obtaining the ith-1st target hidden result, processing the ith-1st target hidden result based on the second FC layer, and obtaining the ith intermediate hidden data; processing the ith intermediate hidden data and the ith intermediate data based on the first floating-point algorithm, and obtaining the ith hidden floating-point result; processing the ith hidden floating-point result based on the nonlinear function and the second floating-point algorithm, and obtaining the ith nonlinear result; processing the ith nonlinear result based on the third floating-point algorithm, and obtaining the ith target hidden result. The technical solution of the present application improves the speed of neural network processing data.

Description

Technical Field

[0001] The present application relates to the field of chip technology, and in particular to an NPU, a device and a storage medium. Background Art

[0002] RNN (Recurrent Neural Network) is a deep learning model that is mainly used to process and analyze sequence data. The network structure of RNN mainly includes input layer, hidden layer and output layer. Among them, the hidden layer is the core of RNN, which is responsible for storing the historical information of the sequence and updating this information through recurrent units (such as simple RNN units, LSTM (Long Short-Term Memory) units, GRU (Gated Recurrent Unit), etc.). At each time step, the recurrent unit will combine the current input and the hidden state of the previous moment to update the current hidden state, and then calculate the output of the current time step.

[0003] However, the current deep learning model requires the CPU (Central Processing Unit) to participate in network calculations, which causes the neural network to process data relatively slowly. Summary of the invention

[0004] The embodiment of the present application provides an NPU, a device and a storage medium, which does not require the CPU to participate in network computing, thereby increasing the speed of neural network processing data. The technical solution is as follows:

[0005] According to a first aspect of an embodiment of the present application, an NPU is provided, the NPU comprising:

[0006] A first matrix calculation module, a second matrix calculation module and a floating point module; the first matrix calculation module and the second matrix calculation module are electrically connected to the floating point module respectively;

[0007] The first matrix calculation module is used to obtain the i-th initial data, process the i-th initial data based on the first FC layer, and obtain the i-th intermediate data; the i-th initial data includes the i-th weight data and the i-th image data; i is an integer greater than or equal to 2;

[0008] The second matrix calculation module is used to obtain the i-1th target hidden result, and process the i-1th target hidden result based on the second FC layer to obtain the i-th intermediate hidden data; the i-1th target hidden result includes the i-1th target hidden data and the i-1th target hidden weight data;

[0009] A floating point module, used for processing the i-th intermediate hidden data and the i-th intermediate data based on a first floating point algorithm to obtain an i-th hidden floating point result;

[0010] The floating point module is further used to process the i-th hidden floating point result based on the nonlinear function and the second floating point algorithm to obtain the i-th nonlinear result;

[0011] The floating-point module is also used to process the i-th nonlinear result based on a third floating-point algorithm to obtain the i-th target hidden result.

[0012] In a possible implementation, the processing of the i-th intermediate hidden data and the i-th intermediate data based on the first floating-point algorithm to obtain the i-th hidden floating-point result includes:

[0013] Processing the i-th intermediate hidden data and the i-th intermediate data based on the subtraction to obtain the i-th hidden subtraction result;

[0014] Perform data format conversion on the i-th hidden division result to obtain the i-th hidden floating-point data;

[0015] Processing the i-th hidden floating-point result based on floating-point multiplication to obtain the i-th hidden multiplication result;

[0016] The i-th hidden multiplication result is processed based on floating-point addition to obtain the i-th hidden floating-point result.

[0017] In a possible implementation, the i-th hidden floating-point result is processed based on the nonlinear function and the second floating-point algorithm to obtain the i-th nonlinear result, including:

[0018] Processing the i-th hidden floating-point result based on the nonlinear function to obtain an i-th first intermediate result;

[0019] The i-th first intermediate result is processed based on the second floating-point algorithm to obtain the i-th nonlinear result.

[0020] In a possible implementation, processing the i-th hidden floating-point result based on the nonlinear function to obtain the i-th first intermediate result includes:

[0021] Processing the i-th hidden floating point result based on the first nonlinear function to obtain the i-th hidden forgotten result; or

[0022] Processing the i-th hidden floating-point result based on the first nonlinear function to obtain the i-th hidden input result; or

[0023] Processing the i-th hidden floating-point result based on the first nonlinear function to obtain the i-th hidden output result; or

[0024] The i-th hidden floating-point result is processed based on the second nonlinear function to obtain the i-th hidden processing result.

[0025] In a possible implementation, processing the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result includes:

[0026] Processing the i-th first intermediate result based on floating-point multiplication to obtain an i-th multiplication result;

[0027] Processing the i-th multiplication result based on floating-point addition to obtain the i-th first addition result;

[0028] The i-th first addition result is processed based on the floating-point multiplication to obtain an i-th first nonlinear result.

[0029] In a possible implementation, the processing the i-th first intermediate result based on floating-point multiplication to obtain the i-th multiplication result includes:

[0030] Processing the i-th hidden-forgetting result and the i-1-th first addition result based on the floating-point multiplication to obtain an i-th first multiplication result; and

[0031] Based on the floating-point multiplication processing, the i-th hidden input result and the i-th hidden processing result are processed to obtain the i-th second multiplication result.

[0032] In a possible implementation, the i-th multiplication result is processed based on floating-point addition to obtain the i-th first addition result, including:

[0033] The i-th first multiplication result and the i-1-th second multiplication result are processed based on the floating-point addition to obtain the i-th first addition result.

[0034] In a possible implementation, before the i-th first addition result is processed based on the floating-point multiplication to obtain the i-th first nonlinear result, the method further includes:

[0035] The i-th first addition result is processed based on the second nonlinear function to obtain an i-th intermediate nonlinear result.

[0036] In a possible implementation, the processing the i-th first addition result based on the floating-point multiplication to obtain the i-th first nonlinear result includes:

[0037] The i-th intermediate nonlinear result and the i-th hidden output result are processed based on the floating-point multiplication to obtain the i-th first nonlinear result.

[0038] In a possible implementation, processing the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result includes:

[0039] The i-th first intermediate result is processed based on the floating-point multiplication to obtain the i-th second nonlinear result; the first intermediate result is the hidden forgotten result.

[0040] In a possible implementation, processing the i-th first intermediate result based on the floating-point multiplication to obtain the i-th second nonlinear result includes:

[0041] The i-th hidden forgetting result and the i-1-th third nonlinear result are processed based on the floating-point multiplication to obtain the i-th second nonlinear result.

[0042] In a possible implementation, processing the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result includes:

[0043] Processing the (i-1)th second nonlinear result based on the floating-point addition to obtain an (i)th second addition result;

[0044] Processing the i-1th second nonlinear result and the i-1th third nonlinear result based on the floating-point multiplication to obtain an i-th third multiplication result; and processing the i-th second addition result and the i-th first intermediate result based on the floating-point multiplication to obtain an i-th fourth multiplication result;

[0045] The i-th fourth multiplication result and the i-th second addition result are processed based on the floating-point addition to obtain the i-th third nonlinear result.

[0046] In a possible implementation, the processing the i-th second addition result and the i-th first intermediate result based on the floating-point multiplication to obtain the i-th fourth multiplication result includes:

[0047] Based on the floating-point multiplication processing, the i-th second addition result and the i-th hidden processing result are processed to obtain the i-th fourth multiplication result; the first intermediate result is the hidden processing result.

[0048] In a possible implementation, the processing of the i-th nonlinear result based on the third floating-point algorithm to obtain the i-th target hidden result includes:

[0049] Processing the i-th nonlinear result based on floating-point division to obtain the i-th hidden division result;

[0050] Perform data format conversion on the i-th hidden division result to obtain the i-th hidden integer result;

[0051] Based on integer addition, i hidden integer results are processed to obtain the i-th target hidden result.

[0052] According to a second aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executed by the NPU as described above.

[0053] According to a third aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program is stored. The at least one program is loaded and executed by a processor to implement the above NPU.

[0054] In an embodiment of the present application, an NPU is provided. Through multiple reuse of the first matrix calculation module, the second matrix calculation module and the floating-point module by the NPU, the functions of LSTM or GRU can be realized without increasing the chip area. And the acceleration of matrix operations by the first matrix calculation module and the second matrix calculation module and the acceleration of nonlinear functions based on the floating-point module, the data processing involved can all be completed by the NPU without the participation of the CPU, which greatly improves the speed of the neural network processing data. In addition, based on the floating-point module to accelerate nonlinear functions, the sensitivity of the data to precision will not lead to large differences, thereby improving the accuracy of the calculation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 is a schematic diagram of an implementation environment provided according to an embodiment of the present application;

[0057] Figure 2 It is a schematic diagram of the structure of an LSTM unit in the related art;

[0058] Figure 3 It is a structural diagram of a GRU unit in the related art;

[0059] Figure 4 It is a schematic diagram of the structure after a LSTM unit is layered according to an embodiment of the present application;

[0060] Figure 5 is a schematic diagram of a structure after a GRU unit is layered according to an embodiment of the present application;

[0061] Figure 6 is a simplified structural diagram of an NPU provided according to an embodiment of the present application;

[0062] Figure 7 is a schematic diagram of the structure of a first NPU provided according to an embodiment of the present application;

[0063] Figure 8 It is a schematic diagram of a process of performing neural network acceleration by a first NPU provided in an embodiment of the present application;

[0064] Fig. 9 is a flowchart of step 803 provided according to an embodiment of the present application;

[0065] Fig.10 is a flowchart of step 804 provided according to an embodiment of the present application;

[0066] Fig.11 is a flowchart of step 8042 provided according to an embodiment of the present application;

[0067] Fig.12 is a flowchart of step 805 provided according to an embodiment of the present application;

[0068] Fig.13 is a schematic diagram of the structure of a second NPU provided according to an embodiment of the present application;

[0069] Fig.14 It is a schematic diagram of a process of performing neural network acceleration by a second NPU provided in an embodiment of the present application;

[0070] Fig.15 is a schematic diagram of the structure of a third NPU provided according to an embodiment of the present application;

[0071] Fig.16 It is a schematic diagram of a process of performing neural network acceleration by a third NPU provided in an embodiment of the present application;

[0072] Fig.17 is a flowchart of step 1608 provided according to an embodiment of the present application;

[0073] Fig.18 It is a schematic diagram of a process of neural network acceleration provided according to an embodiment of the present application;

[0074] Fig.19 is a schematic diagram of the structure of a terminal provided according to an embodiment of the present application;

[0075] Fig. 20 It is a structural diagram of a server provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0077] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0078] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0079] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.

[0080] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.

[0081] Figure 1 It is a schematic diagram of an implementation environment provided according to an embodiment of the present application, and the implementation environment may include a terminal 101 and a server 102.

[0082] In the terminal 101, an NPU is provided, such as an NPU provided on a computer or a camera NPU provided on a smart phone. As a hardware accelerator, NPU (Neural Processing Unit) can efficiently perform calculations of various neural network models including RNN.

[0083] The terminal 101 may be a smart phone with an NPU, a wearable device, a personal computer, a laptop, a tablet computer, a smart TV, a car terminal, etc.

[0084] The server 102 may be a single server, a server cluster consisting of multiple servers, or a cloud processing center.

[0085] The terminal 101 is connected to the server 102 via a wired or wireless network.

[0086] In some embodiments, the wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to any combination of local area network (LAN), metropolitan area network (MAN), wide area network (WAN), mobile, wired or wireless network, private network or virtual private network. In some embodiments, the data exchanged through the network is represented by technology and / or format including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technology can also be used to replace or supplement the above data communication technology.

[0087] Figure 2 It is a structural diagram of an LSTM unit in the related art.

[0088] In one LSTM unit, there are 8 FCs, Sigmoid, Tanh, vector addition, and vector multiplication. In actual scenarios, the LSTM function can be realized by calling the LSTM unit multiple times according to actual needs.

[0089] Among them, FC (Fully Connected Layer) is used for matrix calculation. "+" indicates vector addition. "-" indicates vector subtraction. Sigmoid and Tanh are two commonly used activation functions in neural networks. Both Sigmoid and Tanh are nonlinear functions. C refers to the state data of the memory unit. t indicates the tth clock cycle. t is an integer greater than or equal to 1. H is the hidden layer input data; X is the input data. H(t) is the tth hidden layer input data. X(t) is the tth input data.

[0090] It should be noted that nonlinear functions can also be expressed as nonlinear activation functions.

[0091] Figure 3 It is a structural diagram of a GRU unit in the related art.

[0092] A GRU unit includes 4 FCs, Sigmoid, Tanh, vector addition, and vector multiplication. In actual scenarios, the GRU function can be realized by calling the GRU unit multiple times according to actual needs.

[0093] Among them, R refers to the reset gate. Z refers to the update gate. The reset gate and update gate are implemented based on Sigmoid.

[0094] Combination Figure 2 and Figure 3 , we can know that both LSTM and GRU include FC, Sigmoid, Tanh, vector addition and vector multiplication. Therefore, the functions of LSTM and GRU can be realized by reusing FC, Sigmoid, Tanh, vector addition and vector multiplication.

[0095] Figure 4 It is a schematic diagram of the structure after a LSTM unit is layered according to an embodiment of the present application.

[0096] Combination Figure 4 In some embodiments, an LSTM unit is divided into two FC layers and an LSTM layer, and the two FC layers are respectively a first FC layer and a second FC layer. The first FC layer and the second FC layer are respectively electrically connected to the LSTM layer.

[0097] The LSTM layer includes a gate unit, a second storage unit, a memory unit, a fourth storage unit, a third storage unit and an output unit.

[0098] The first FC layer and the second FC layer each include 4 FCs.

[0099] In an example, the second storage unit, the third storage unit, and the fourth storage unit are all single-port SRAMs (Static Random Access Memory) to save hardware area and reduce data interaction with external memories, thereby reducing latency.

[0100] In one example, the gate unit is any one of a forget gate, an input gate, an output gate, and a Tanh unit. The forget gate, the input gate, and the output gate perform nonlinear processing on data based on Sigmoid. The Tanh unit performs nonlinear processing on data based on Tanh.

[0101] In an example, the functionality of LSTM can be implemented by repeatedly calling two FC layers and one LSTM layer.

[0102] In an example, the external memory may be DDR (Double Data Rate Synchronous Dynamic Random Access Memory).

[0103] Figure 5 It is a schematic diagram of the structure after a GRU unit is layered according to an embodiment of the present application.

[0104] Combination Figure 5 In some embodiments, a GRU unit is divided into a first FC layer, a second FC layer, and a first GRU layer; and a first FC layer, a second FC layer, and a second GRU layer.

[0105] The first GRU layer includes a gate unit and a third storage unit. The second GRU layer includes an output unit, a second storage unit and a fourth storage unit.

[0106] The first FC layer and the second FC layer may include two FCs.

[0107] In one example, the function of GRU is implemented by repeatedly calling the first FC layer, the second FC layer, and the first GRU layer; and the first FC layer, the second FC layer, and the second GRU layer.

[0108] In order to realize repeated calling of any one of the first FC layer, the second FC layer, the LSTM layer, the first GRU layer and the second GRU layer without increasing the chip area, an embodiment of the present application provides an NPU.

[0109] Figure 6 It is a simplified structural diagram of an NPU provided according to an embodiment of the present application.

[0110] A first matrix calculation module 601 , a second matrix calculation module 602 and a floating point module 603 ; the first matrix calculation module 601 and the second matrix calculation module 602 are electrically connected to the floating point module 603 , respectively.

[0111] It should be noted that the first matrix calculation module 601, the second matrix calculation module 602 and the floating point module 603 are respectively used as parts of the NPU. As a chip, the NPU exists in the form of hardware, and the first matrix calculation module 601, the second matrix calculation module 602 and the floating point module 603 also exist and run in the form of hardware.

[0112] In addition, the first matrix calculation module 601, the second matrix calculation module 602 and the floating point module 603 are also electrically connected to the external memory respectively.

[0113] Figure 7 It is a schematic diagram of the structure of the first NPU provided according to an embodiment of the present application.

[0114] Combination Figure 7 The first type of NPU is exemplified.

[0115] The NPU includes a first storage unit, a second storage unit, a third storage unit, and a fourth storage unit.

[0116] The floating-point module includes a first format conversion unit, a clock multiplexing unit, a first floating-point multiplication unit, a second floating-point multiplication unit, a first floating-point addition unit, a nonlinear unit, and a third floating-point multiplication unit.

[0117] The first format conversion unit is electrically connected to the first storage unit.

[0118] The second storage unit is electrically connected to the first floating-point multiplication unit, the second floating-point multiplication unit and the clock multiplexing unit respectively.

[0119] The third storage unit is electrically connected to the fourth storage unit and the first floating-point multiplication unit respectively.

[0120] The fourth storage unit is also electrically connected to the nonlinear unit and the first floating-point addition unit respectively.

[0121] The nonlinear unit, the third floating-point multiplication unit and the second format conversion unit are electrically connected in sequence.

[0122] Figure 8 This is a schematic diagram of a process of performing neural network acceleration using a first NPU according to an embodiment of the present application. The process includes the following steps:

[0123] In step 801, the first matrix calculation module is used to obtain the i-th initial data, process the i-th initial data based on the first FC layer, and obtain the i-th intermediate data; the i-th initial data includes the i-th weight data and the i-th image data; i is an integer greater than or equal to 2.

[0124] Wherein, each image data includes at least one state data, and each weight data includes at least one weight.

[0125] In some embodiments, the NPU performs matrix calculation on the i-th image data and the i-th weight data based on the first FC layer to obtain the i-th intermediate data.

[0126] For example, image data is represented by X(t), and weight data is represented by W(X).

[0127] In one example, i is less than or equal to a preset number. The preset number is the number of times the NPU calls the first FC layer. The weight data corresponding to the preset number of image data is the same. That is, in the same application scenario, for the same batch of image data, the weight data remains unchanged. In one example, after the first FC layer processes the preset number of initial data, step 602 is executed. After the first FC layer processes the preset number of initial data, the result obtained is stored by the external memory.

[0128] For example, the first matrix calculation module inputs one image data into the first FC layer each time, and each image data includes a data. Since the first FC layer includes 4 FCs, each intermediate data includes 4a data. That is, the first matrix calculation module continuously inputs a preset number of image data into the first FC layer. That is to say, the first matrix calculation module needs to circulate a preset number of times.

[0129] a may represent the number of hidden layers. a is an integer greater than or equal to 1 and less than or equal to 512.

[0130] The size of a is limited by 512 to prevent the amount of data to be cached from exceeding the actual size of the cache.

[0131] In step 802, the second matrix calculation module is used to obtain the i-1th target hidden result, and processes the i-1th target hidden result based on the second FC layer to obtain the i-1th intermediate hidden data; the i-1th target hidden result includes the i-1th target hidden data and the i-1th target hidden weight data.

[0132] In some embodiments, the second matrix calculation module performs matrix calculation on the i-1th target hidden data and the i-1th target hidden weight data based on the second FC layer to obtain the i-th intermediate hidden data.

[0133] For example, the second matrix calculation module inputs a target hidden data and a target hidden weight data into the second FC layer each time, and each target hidden data includes a data. Since the second FC layer includes 4 FCs, each intermediate hidden data includes 4a data. That is, the second matrix calculation module continuously inputs a preset number of target hidden data into the second FC layer. That is to say, the second matrix calculation module needs to circulate a preset number of times.

[0134] For example: The target hidden data is represented by H(t). The target hidden weight data is represented by W(H).

[0135] In an embodiment of the present application, the NPU multiplexes the first matrix calculation module and the second matrix calculation module multiple times to implement the functions of 8 FCs in the LSTM.

[0136] In step 803, the floating point module is used to process the i-th intermediate hidden data and the i-th intermediate data based on a first floating point algorithm to obtain an i-th hidden floating point result.

[0137] In step 804, the floating-point module is further used to process the i-th hidden floating-point result based on the nonlinear function and the second floating-point algorithm to obtain the i-th nonlinear result.

[0138] In step 805, the floating-point module is further used to process the i-th nonlinear result based on the third floating-point algorithm to obtain the i-th target hidden result.

[0139] In an embodiment of the present application, the NPU calls the first matrix calculation module, the second matrix calculation module and the floating point module to implement the functions of the above-mentioned LSTM layer.

[0140] The first floating-point algorithm includes floating-point multiplication, floating-point division, floating-point addition, and floating-point division.

[0141] The second floating-point arithmetic includes floating-point multiplication, floating-point division, floating-point addition, and floating-point division, etc.

[0142] In some embodiments, the NPU splices the i-th intermediate hidden data and the i-th intermediate data to obtain the i-th spliced ​​data; the floating point module is further used to process the i-th spliced ​​data based on the first floating point algorithm to obtain the i-th hidden floating point data. The i-th spliced ​​data is stored in the external memory.

[0143] For example, each concatenated data includes 4a data. Each hidden floating point data includes 4a data.

[0144] In some embodiments, for floating-point multiplication, a data is used as an example for explanation. The sign bit of the data is the XOR of the sign bits of the corresponding two operands. The exponent of the data is the sum of the exponents of the corresponding two operands. The mantissa of the data is the product of the mantissas of the corresponding two operands. If the mantissa of the product is not normalized, it needs to be normalized.

[0145] In some embodiments, for floating point addition, it can be understood that the exponents of the two floating point numbers are adjusted to be the same. This usually means moving the smaller number to the right (i.e., reducing its exponent) to match the exponent of the larger number. The mantissas of the two floating point numbers are added. If the added mantissa is not normalized (i.e., the highest bit is not 1), it needs to be shifted left and the exponent adjusted accordingly. Due to the limited number of digits in the mantissa, the result of the addition operation may need to be rounded.

[0146] It should be noted that executing step 801 first and then executing step 802 is only one implementation method.

[0147] In some embodiments, step 801 and step 802 may also be performed simultaneously.

[0148] Fig. 9 It is a flowchart of step 803 provided according to an embodiment of the present application.

[0149] Combination Fig. 9 Step 803 is exemplarily described.

[0150] In some embodiments, the above step 803 may include the following steps 8031 ​​to 8034:

[0151] In step 8031, the first format conversion unit is used to process the i-th intermediate hidden data and the i-th intermediate data based on subtraction to obtain the i-th hidden subtraction result.

[0152] In one example, the i-th intermediate hidden data and the i-th intermediate data are data in integer format. The i-th hidden subtraction result is also data in integer format.

[0153] In one example, the first format conversion unit is further configured to obtain the i-th spliced ​​data from an external memory, process the i-th spliced ​​data based on subtraction, and obtain the i-th hidden subtraction result.

[0154] For example: Each hidden subtraction result includes 4a data.

[0155] For example: data of type int8. int8 represents an 8-bit integer.

[0156] In step 8032, the first format conversion unit is further used to perform data format conversion on the i-th hidden division result to obtain the i-th hidden floating-point data.

[0157] In one example, the first format conversion unit converts the i-th hidden division result into data in floating point format, that is, the i-th hidden floating point data is data in floating point format.

[0158] For example: each hidden floating point data includes 4a data.

[0159] In step 8033, the first format conversion unit is used to process the i-th hidden floating-point data based on floating-point multiplication to obtain the i-th hidden multiplication result.

[0160] It should be noted that, in the process of executing step 8033, it takes a certain amount of time for the first format conversion unit to process the i-th hidden floating-point data based on floating-point multiplication. Therefore, the partially hidden multiplication result that has been obtained is stored in the first storage unit until the i-th hidden floating-point data is completely processed.

[0161] For example: Each hidden multiplication result includes 4a data.

[0162] In step 8034, the first format conversion unit is further used to process the i-th hidden multiplication result based on floating-point addition to obtain the i-th hidden floating-point result.

[0163] In one example, the first format conversion unit calls the i-th hidden multiplication result from the cache.

[0164] Fig.10 It is a flowchart of step 804 provided according to an embodiment of the present application.

[0165] Combination Fig.10 Step 804 is exemplarily described.

[0166] In some embodiments, the above step 804 further includes the following steps 8041 and 8042:

[0167] In step 8041, the clock multiplexing unit is used to process the i-th hidden floating-point result based on the nonlinear function to obtain the i-th first intermediate result.

[0168] In step 8042, the clock multiplexing unit is further used to process the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result.

[0169] In some embodiments, the above step 8041 can be implemented in the following ways:

[0170] A first implementation method: The clock multiplexing unit processes the i-th hidden floating-point result based on the forget gate to obtain the i-th hidden forgotten result. A second implementation method: The clock multiplexing unit processes the i-th hidden floating-point result based on the input gate to obtain the i-th hidden input result. A third implementation method: The clock multiplexing unit processes the i-th hidden floating-point result based on the output gate to obtain the i-th hidden output result. A fourth implementation method: The clock multiplexing unit processes the i-th hidden floating-point result based on the second nonlinear function to obtain the i-th hidden processing result.

[0171] For example, the first nonlinear function is Sigmoid, and the second nonlinear function is Tanh.

[0172] It should be noted that, for the above-mentioned first implementation method, the second implementation method, the third implementation method and the fourth implementation method, the clock multiplexing unit can only implement the above-mentioned step 8041 based on one of the implementation methods in a certain clock cycle based on the clock multiplexing principle.

[0173] In an example, the clock multiplexing unit divides the i-th hidden floating-point result into four parts, namely, a first part, a second part, a third part and a fourth part.

[0174] In the kth clock cycle, the first part is processed based on the forget gate to obtain the ith hidden forgetting result. In the k+1th clock cycle, the second part is processed based on the input gate to obtain the ith hidden input result. In the k+2th clock cycle, the third part is processed based on Tanh to obtain the ith hidden output result. In the k+3th clock cycle, the fourth part is processed based on the output gate to obtain the ith hidden processing result. k is an integer greater than or equal to 0.

[0175] Among them, the i-th hidden forgotten result, the i-th hidden input result, the i-th hidden output result and the i-th hidden processing result are cached in the second storage unit, and the i-th hidden forgotten result, the i-th hidden input result, the i-th hidden output result and the i-th hidden processing result are output from the second storage unit for subsequent processing.

[0176] Furthermore, when the clock multiplexing unit caches the i-th hidden forgotten result, the i-th hidden input result, and the i-th hidden output result, the clock multiplexing unit caches the i-th hidden forgotten result to the first cache area, caches the i-th hidden input result to the second cache area, caches the i-th hidden output result to the third cache area, and stores the i-th hidden processing result to the fourth cache area. That is, the i-th hidden forgotten result, the i-th hidden input result, the i-th hidden output result, and the i-th hidden processing result are stored in different cache areas, respectively, for subsequent calls.

[0177] It can be understood that the clock multiplexing unit outputs the above different implementations, thereby reducing the interaction between the NPU and the external memory and reducing the delay.

[0178] In one example, the first part, the second part, the third part, and the fourth part include the same amount of data.

[0179] For example, the i-th hidden floating-point result is divided into 4 parts, each of which includes a data. In the k-th clock cycle, the a data in the first part are processed based on the forget gate to obtain a hidden forgotten data in the i-th hidden forgotten result. In the k+1-th clock cycle, the a data in the second part are processed based on the input gate to obtain a hidden input data in the i-th hidden input result. In the k+2-th clock cycle, the a data in the third part are processed based on Tanh to obtain a hidden output result data in the i-th hidden output result. In the k+3-th clock cycle, the a data in the fourth part are processed based on the output gate to obtain a data in the i-th hidden processing result.

[0180] In one example, Tanh is implemented as follows:

[0181] Calculate the exponent of exp: preprocess the a data in the third part to obtain the preprocessed result. Process the preprocessed result based on floating-point division to obtain the first floating-point division result. Round the floating-point division result to obtain the rounded result; process the rounded result based on floating-point addition to obtain the first floating-point addition result. Process the first floating-point addition result based on floating-point multiplication to obtain the first floating-point multiplication result, that is, obtain the exponent B of exp.

[0182] For example, preprocessing means multiplying each data by 2.

[0183] In one example, the a data are not preprocessed, but are directly preprocessed based on the floating-point division process to obtain a first floating-point division result.

[0184] Calculate exp: .in, It is calculated by cordic (Coordinate Rotation Digital Computer).

[0185] Tanh can be calculated in two ways:

[0186] The first method: calculate 1+exp; then calculate 2 / (1+exp); then add (-1).

[0187] The second method: through exp= , and Tanh is calculated.

[0188] In one example, a Tanh value is obtained for each clock cycle.

[0189] For example, Tanh is implemented by the following formula:

[0190] .

[0191] The above formula can be used to calculate , and finally get .

[0192] However, if x is near 0, due to The calculation error caused by Therefore, when x is small, that is, when the absolute value of x is less than ln2, another method is used for calculation.

[0193] Specifically, in In the calculation process, we obtain and The value of .

[0194] Then, through , and get the calculation result.

[0195] The cordic algorithm can be used to calculate and Therefore, the result of exp can be calculated through the cordic algorithm and floating-point operations, and then the results of Tanh and Sigmoid can be calculated.

[0196] For example: Sigmoid is implemented by the following formula:

[0197] Sigmoid= .

[0198] That is to say, the value of Sigmoid can be obtained by calculating exp.

[0199] exp can be calculated by the following formula:

[0200] .

[0201] Where A is the modulus of X divided by ln2 and B is the remainder. Obtained by shift calculation. This is achieved through cordic approximation.

[0202] .

[0203] Fig.11 It is a flowchart of step 8042 provided according to an embodiment of the present application.

[0204] Combination Fig.11 Step 8042 is exemplarily described.

[0205] In some embodiments, the above step 8042 includes the following steps 80421 to 80423:

[0206] In step 80421, the floating-point module is further used to process the i-th first intermediate result based on floating-point multiplication to obtain the i-th multiplication result.

[0207] Specifically, the first floating-point multiplication unit is used to process the i-th hidden forgotten result and the i-1-th first addition result based on the floating-point multiplication to obtain the i-th first multiplication result; and the second floating-point multiplication unit is used to process the i-th hidden input result and the i-th hidden processing result based on the floating-point multiplication to obtain the i-th second multiplication result.

[0208] It should be noted that the floating-point module is also used to simultaneously calculate the i-th first multiplication result and the i-th second multiplication result, and the i-th second multiplication result is independent of the i-th first multiplication result.

[0209] If the i-th second multiplication result is calculated after the i-th first multiplication result is calculated, an additional cache is required to store the i-th first multiplication result before calculating the i-th second multiplication result. Therefore, calculating the i-th first multiplication result and the i-th second multiplication result at the same time can save the number of storage units, thereby saving hardware area.

[0210] In step 80422, the floating-point module is further used to process the i-th multiplication result based on floating-point addition to obtain the i-th first addition result.

[0211] In some embodiments, the floating point module is further used to cache the i-th first addition result.

[0212] In some embodiments, the first floating-point multiplication unit and the second floating-point multiplication unit perform calculations simultaneously. If one floating-point multiplication unit is set to perform the above two floating-point multiplication calculations, it is necessary to set up another storage unit to store the calculation results of the floating-point multiplication unit. After the second floating-point multiplication calculation starts, it is necessary to read the calculation result of the first floating-point multiplication from the storage unit, thereby increasing the hardware area. Therefore, two floating-point multiplication units are set and the calculations are performed simultaneously.

[0213] In some embodiments, the above step 80422 can be implemented as follows:

[0214] The first floating-point addition unit is used to process the i-th first multiplication result and the i-1-th second multiplication result based on the floating-point addition to obtain the i-th first addition result.

[0215] The i-th first addition result is stored in the fourth storage unit, and the fourth storage unit simultaneously writes the i-th first addition result into the third storage unit and the nonlinear unit.

[0216] In step 80423, the nonlinear unit processes the i-th first addition result based on the floating-point multiplication to obtain the i-th first nonlinear result.

[0217] In some embodiments, the NPU reads the i-th first addition result from the cache and stores it in another cache to facilitate subsequent calls to the i-th first addition result. At the same time, the NPU also processes the i-th first addition result based on floating-point multiplication.

[0218] In some embodiments, before the above step 80423, the nonlinear unit is used to:

[0219] The i-th first addition result is processed based on the second nonlinear function to obtain an i-th intermediate nonlinear result.

[0220] In some embodiments, the above step 80423 can be implemented as follows:

[0221] The third floating-point multiplication unit is used to process the i-th intermediate nonlinear result and the i-th hidden output result based on the floating-point multiplication to obtain the i-th first nonlinear result.

[0222] The i-th first nonlinear result is stored in the external memory.

[0223] Fig.12 It is a flowchart of step 805 provided according to an embodiment of the present application.

[0224] Combination Fig.12 Step 805 is exemplarily described.

[0225] In some embodiments, the above step 805 includes the following steps 8051 to 8053: including:

[0226] In step 8051, the second format conversion unit is used to process the i-th nonlinear result based on floating-point division to obtain the i-th hidden division result.

[0227] In step 8052, the second format conversion unit is further used to perform data format conversion on the i-th hidden division result to obtain the i-th hidden integer result.

[0228] In step 8053, the second format conversion unit is further used to process the i hidden integer results based on integer addition to obtain the i-th target hidden result.

[0229] In the embodiment of the present application, the above steps 801 to 805 can realize the function of LSTM without increasing the chip area by multiplexing the first matrix calculation module, the second matrix calculation module and the floating-point module through the NPU. And the acceleration of matrix operations through the first matrix calculation module and the second matrix calculation module and the acceleration of nonlinear functions based on the floating-point module, the data processing involved can all be completed by the NPU without the participation of the CPU, which greatly improves the speed of neural network processing data. In addition, based on the floating-point module to accelerate nonlinear functions, the sensitivity of the data to precision will not lead to large differences, thereby improving the accuracy of the calculation results.

[0230] Fig.13 It is a schematic diagram of the structure of a second NPU provided according to an embodiment of the present application.

[0231] Combination Fig.13 , the second type of NPU is exemplified.

[0232] The NPU includes a first storage unit, a third storage unit, and a fourth storage unit.

[0233] The floating-point module further includes a first format conversion unit, a clock multiplexing unit, a third floating-point multiplication unit and a second format conversion unit.

[0234] The clock multiplexing unit is electrically connected to the third storage unit.

[0235] The third floating-point multiplication unit is electrically connected to the third storage unit, the fourth storage unit and the second format conversion unit respectively.

[0236] Fig.14 This is a schematic diagram of a second NPU performing neural network acceleration according to an embodiment of the present application. It includes the following steps:

[0237] In step 1401, the first matrix calculation module is used to obtain the i-th initial data, process the i-th initial data based on the first FC layer, and obtain the i-th intermediate data; the i-th initial data includes the i-th weight data and the i-th image data; i is an integer greater than or equal to 2.

[0238] For example, the NPU inputs one image data and one weight hidden weight data into the first FC layer each time. Each image data includes a data. Since the first FC layer includes 2 FCs, each intermediate data includes 2a data.

[0239] For example, the first matrix calculation module continuously inputs a preset number of initial data into the first FC layer, that is, the first matrix calculation module needs to cyclically run a preset number of times.

[0240] In step 1402, the second matrix calculation module is used to obtain the i-1th target hidden result, and processes the i-1th target hidden result based on the second FC layer to obtain the i-1th intermediate hidden data; the i-1th target hidden result includes the i-1th target hidden result and the i-1th target hidden weight data.

[0241] For example, the second matrix calculation module inputs a target hidden data and a target hidden weight data into the second FC layer each time, and each target hidden result includes a data. Since the second FC layer includes 2 FCs, each intermediate hidden data includes 2a data. That is, the second matrix calculation module continuously inputs a preset number of target hidden data into the second FC layer. That is to say, the second matrix calculation module needs to circulate a preset number of times.

[0242] In some embodiments, the NPU splices the i-th intermediate hidden data and the i-th intermediate data to obtain the i-th spliced ​​data; the floating point module is further used to process the i-th spliced ​​data based on the first floating point algorithm to obtain the i-th hidden floating point data. The i-th spliced ​​data is stored in the external memory.

[0243] For example, each concatenated data includes 2a data. Each hidden floating point data includes 2a data.

[0244] In step 1403, the first format conversion unit is further used to process the i-th intermediate hidden data and the i-th intermediate data based on subtraction to obtain the i-th hidden subtraction result.

[0245] In one example, the first format conversion unit is further configured to obtain the i-th spliced ​​data from an external memory, process the i-th spliced ​​data based on subtraction, and obtain the i-th hidden subtraction result.

[0246] For example: Each hidden subtraction result includes 2a data.

[0247] In step 1404, the first format conversion unit is further used to perform data format conversion on the i-th hidden division result to obtain the i-th hidden floating-point data.

[0248] For example: each hidden floating point data includes 2a data.

[0249] In step 1405, the first format conversion unit is further configured to process the i-th hidden floating-point data based on floating-point multiplication to obtain the i-th hidden multiplication result.

[0250] For example: Each hidden multiplication result includes 2a data.

[0251] In step 1406, the first format conversion unit is further configured to process the i-th hidden multiplication result based on floating-point addition to obtain the i-th hidden floating-point result.

[0252] It should be noted that steps 1403 to 1406 are similar to the above steps 8031 ​​to 8034 and will not be described in detail.

[0253] In step 1407, the floating point module is used to process the i-th hidden floating point result based on the nonlinear function to obtain the i-th first intermediate result.

[0254] Part of the i-th first intermediate result is cached, and part is output to the external memory. The part output to the external memory is used as the input data of the second FC layer in the next cycle of the second GRU layer. The other part that is cached participates in the operation of the next cycle of the second GRU layer.

[0255] For example, the i-th first intermediate result includes 2a data, and the NPU caches a of the data and outputs the other a data to the external memory.

[0256] In step 1408, the floating-point module is used to process the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result.

[0257] In some embodiments, the above step 1408 can be implemented as follows:

[0258] The clock multiplexing unit processes the i-th first intermediate result based on the floating-point multiplication to obtain the i-th second nonlinear result; the first intermediate result is a hidden forgotten result.

[0259] In one example, in the k+4th clock cycle, the i-th hidden floating point result is processed based on the forget gate to obtain the i-th hidden forgotten result. The third storage unit caches the i-th hidden forgotten result.

[0260] It can be understood that the i-th hidden forgetting result is cached by the third storage unit, thereby reducing the interaction between the NPU and the external memory and reducing the delay.

[0261] For example, the ith hidden floating point result includes 2a data. In the k+4th clock cycle, the 2a data in the ith hidden floating point result are processed based on Sigmoid to obtain the 2a hidden forgotten data in the ith hidden forgotten result.

[0262] In one example, the third floating-point multiplication unit is used to process the i-th hidden forgetting result and the i-1-th third nonlinear result based on the floating-point multiplication to obtain the i-th second nonlinear result.

[0263] In step 1409, the second format conversion unit is used to process the i-th second nonlinear result based on floating-point division to obtain the i-th second hidden division result.

[0264] In step 1410, the second format conversion unit is further configured to perform data format conversion on the i-th second hidden division result to obtain the i-th second hidden integer result.

[0265] In step 1411, the second format conversion unit is further configured to process the i second hidden integer results based on integer addition to obtain the i-th second target hidden result.

[0266] It should be noted that steps 1409 to 1411 are similar to the above steps 8051 to 8053 and will not be described in detail.

[0267] In the embodiment of the present application, the above steps 1401 to 1411, through the NPU calling the first matrix calculation module, the second matrix calculation module and the floating-point module, can implement the function of the first GRU layer without increasing the chip area.

[0268] Fig.15 It is a schematic diagram of the structure of the third NPU provided according to an embodiment of the present application.

[0269] Combination Fig.15 The third type of NPU is exemplified.

[0270] The NPU includes a first storage unit, a second storage unit, a third storage unit, and a fourth storage unit.

[0271] The floating-point module includes a first format conversion unit, a clock multiplexing unit, a first floating-point multiplication unit, a second floating-point multiplication unit, a first floating-point addition unit, and a second floating-point addition unit.

[0272] The first format conversion unit is electrically connected to the first storage unit.

[0273] The second storage unit is electrically connected to the first floating-point multiplication unit, the second floating-point multiplication unit and the clock multiplexing unit respectively.

[0274] The third storage unit is electrically connected to the fourth storage unit and the first floating-point multiplication unit respectively.

[0275] The fourth storage unit is also electrically connected to the nonlinear unit and the first floating-point addition unit respectively.

[0276] For example, the first storage unit, the second storage unit, the third storage unit, and the fourth storage unit are all single-port SRAMs, so as to save memory.

[0277] Fig.16 This is a schematic diagram of a third NPU performing neural network acceleration according to an embodiment of the present application. It includes the following steps:

[0278] In step 1601, the first matrix calculation module is used to obtain the i-th initial data, process the i-th initial data based on the first FC layer, and obtain the i-th intermediate data; the i-th initial data includes the i-th weight data and the i-th image data; i is an integer greater than or equal to 2.

[0279] For example, the first matrix calculation module inputs one image data and one weight hidden weight data into the first FC layer each time, each image data includes a data, and each intermediate data includes 2a data. For example, the first matrix calculation module continuously inputs a preset number of initial data into the first FC layer. That is to say, the first matrix calculation module needs to circulate a preset number of times.

[0280] In step 1602, the second matrix calculation module is used to obtain the i-1th target hidden result, and processes the i-1th target hidden result based on the second FC layer to obtain the i-1th intermediate hidden data; the i-1th target hidden result includes the i-1th target hidden result and the i-1th target hidden weight data.

[0281] For example, the second matrix calculation module inputs a target hidden data and a target hidden weight data into the second FC layer each time, each target hidden result includes a data, and each intermediate hidden data includes 2a data. That is, the NPU continuously inputs a preset number of target hidden data into the second FC layer. That is, the second FC layer needs to be cyclically run a preset number of times.

[0282] In some embodiments, the NPU splices the ith intermediate hidden data and the ith intermediate data to obtain the ith spliced ​​data; the floating point module processes the ith spliced ​​data based on the first floating point algorithm to obtain the ith hidden floating point data. The ith spliced ​​data is stored in the external memory.

[0283] For example, each concatenated data includes 2a data. Each hidden floating point data includes 2a data.

[0284] In step 1603, the first format conversion unit is used to process the i-th intermediate hidden data and the i-th intermediate data based on subtraction to obtain the i-th hidden subtraction result.

[0285] In one example, the first format conversion unit obtains the i-th splicing data from the external memory, processes the i-th splicing data based on subtraction, and obtains the i-th hidden subtraction result.

[0286] For example: Each hidden subtraction result includes 2a data.

[0287] In step 1604, the first format conversion unit is further used to perform data format conversion on the i-th hidden division result to obtain the i-th hidden floating-point data.

[0288] For example: each hidden floating point data includes 2a data.

[0289] In step 1605, the first format conversion unit is further used to process the i-th hidden floating-point data based on floating-point multiplication to obtain the i-th hidden multiplication result.

[0290] For example: Each hidden multiplication result includes 2a data.

[0291] In step 1606, the first format conversion unit is further used to process the i-th hidden multiplication result based on floating-point addition to obtain the i-th hidden floating-point result.

[0292] It should be noted that steps 1603 to 1606 are similar to the above steps 8031 ​​to 8034 and will not be described in detail.

[0293] In the above step 1607, the floating point module is used to process the i-th hidden floating point result based on the nonlinear function to obtain the i-th first intermediate result.

[0294] In step 1608, the floating-point module is used to process the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result.

[0295] Fig.17 It is a flowchart of step 1608 provided according to an embodiment of the present application.

[0296] Combination Fig.17 Step 1608 is exemplarily described.

[0297] In some embodiments, the above step 1608 may include the following steps 16081 to 16083:

[0298] In step 16081, the second floating-point addition unit is used to process the (i-1)th second nonlinear result based on the floating-point addition to obtain the (i)th second addition result.

[0299] In one example, the second floating-point addition unit is used to process the i-1th second nonlinear result and a preset value based on the floating-point addition to obtain the ith second addition result.

[0300] For example, the preset value is 1. That is, the i-1th second nonlinear result is subtracted from "1" to obtain the i-th second addition result.

[0301] In step 16082, the first floating-point multiplication unit is used to process the i-1th second non-linear result and the i-1th third non-linear result based on the floating-point multiplication to obtain the i-th third multiplication result; and the second floating-point multiplication unit is used to process the i-th second addition result and the i-th first intermediate result based on the floating-point multiplication to obtain the i-th fourth multiplication result.

[0302] In step 16083, the first floating-point addition unit is used to process the i-th fourth multiplication result and the i-th second addition result based on the floating-point addition to obtain the i-th third nonlinear result.

[0303] After the NPU outputs the i-1th third nonlinear result, it caches the i-th third nonlinear result. A portion of the i-1th third nonlinear result participates in the operation of the first GRU layer; another portion of the i-1th third nonlinear result participates in the operation of the second GRU layer.

[0304] Optionally, the i-th third nonlinear result is stored in the second storage unit.

[0305] The i-1th third nonlinear result is stored in the fourth storage unit. After the fourth storage unit outputs the -1th third nonlinear result, the second storage unit writes the i-th third nonlinear result into the fourth storage unit. The i-1th second nonlinear result is stored in the third storage unit.

[0306] For example, the i-1th third nonlinear result includes 2a data. The a data in the i-1th third nonlinear result participate in the operation of the first GRU layer; the other a data in the i-1th third nonlinear result participate in the operation of the second GRU layer.

[0307] In some embodiments, in the above step 16082, the i-th third multiplication result can be obtained by the following implementation method:

[0308] The i-th fourth multiplication result is obtained by processing the i-th second addition result and the i-th hidden processing result based on the floating-point multiplication. The first intermediate result is the hidden processing result.

[0309] In one example, in the k+5th clock cycle, the ith hidden floating point result is processed based on Tanh to obtain the ith hidden processing result. The NPU caches the ith hidden processing result.

[0310] The i-th hidden processing result is cached through the NPU, thereby reducing the interaction between the NPU and the external memory and reducing latency.

[0311] For example, the ith hidden floating point result includes a data. In the k+4th clock cycle, the a data in the ith hidden floating point result are processed based on Tanh to obtain a hidden processed data in the ith hidden processed result.

[0312] In step 1609, the second format conversion unit is used to process the i-th third nonlinear result based on floating-point division to obtain the i-th third hidden division result.

[0313] In step 1610, the second format conversion unit is further used to perform data format conversion on the i-th third hidden division result to obtain the i-th third hidden integer result.

[0314] In step 1611, the second format conversion unit is further configured to process the i third hidden integer results based on integer addition to obtain the i-th third target hidden result.

[0315] It should be noted that steps 1609 to 1611 are similar to the above steps 8051 to 8053 and will not be described in detail.

[0316] In the embodiment of the present application, the above steps 1601 to 1611, through the NPU calling the first matrix calculation module, the second matrix calculation module and the floating-point module, can implement the function of the second GRU layer without increasing the chip area.

[0317] In combination with the above description, the embodiments of the present application can realize the functions of the FC layer, the second GRU layer and the first GRU layer without increasing the chip area through multiple reuse of the first matrix calculation module, the second matrix calculation module and the floating-point module by the NPU, that is, the function of the GUR can be realized. And the acceleration of matrix operations through the first matrix calculation module and the second matrix calculation module and the acceleration of nonlinear functions based on the floating-point module, the data processing involved can all be completed by the NPU without the participation of the CPU, which greatly improves the speed of the neural network processing data. In addition, based on the floating-point module to accelerate nonlinear functions, the sensitivity of the data to precision will not lead to large differences, thereby improving the accuracy of the calculation results.

[0318] Fig.18 It is a schematic diagram of a neural network acceleration process provided according to an embodiment of the present application, including:

[0319] Obtain the i-th initial data, process the i-th initial data based on the first FC layer, and obtain the i-th intermediate data; the i-th initial data includes the i-th weight data and the i-th image data; i is an integer greater than or equal to 2;

[0320] Obtain the i-1th target hidden result, process the i-1th target hidden result based on the second FC layer, and obtain the i-th intermediate hidden data; the i-1th target hidden result includes the i-1th target hidden data and the i-1th target hidden weight data;

[0321] Based on the first floating-point algorithm, the i-th intermediate hidden data and the i-th intermediate data are processed to obtain the i-th hidden floating-point result; based on the nonlinear function and the second floating-point algorithm, the i-th hidden floating-point result is processed to obtain the i-th nonlinear result; based on the third floating-point algorithm, the i-th nonlinear result is processed to obtain the i-th target hidden result.

[0322] In some embodiments, it also includes:

[0323] Processing the i-th intermediate hidden data and the i-th intermediate data based on the subtraction to obtain the i-th hidden subtraction result;

[0324] Perform data format conversion on the i-th hidden division result to obtain the i-th hidden floating-point data;

[0325] Processing the i-th hidden floating-point result based on floating-point multiplication to obtain the i-th hidden multiplication result;

[0326] The i-th hidden multiplication result is processed based on floating-point addition to obtain the i-th hidden floating-point result.

[0327] The i-th hidden subtraction result, the i-th hidden floating-point data, the i-th hidden multiplication result and the i-th hidden floating-point result are stored in the first storage unit.

[0328] In some embodiments, it also includes:

[0329] Processing the i-th hidden floating-point result based on the nonlinear function to obtain an i-th first intermediate result;

[0330] The i-th first intermediate result is processed based on the second floating-point algorithm to obtain the i-th nonlinear result.

[0331] In some embodiments, it also includes:

[0332] Processing the i-th hidden floating point result based on the first nonlinear function to obtain the i-th hidden forgotten result; or

[0333] Processing the i-th hidden floating-point result based on the first nonlinear function to obtain the i-th hidden input result; or

[0334] Processing the i-th hidden floating-point result based on the first nonlinear function to obtain the i-th hidden output result; or

[0335] The i-th hidden floating-point result is processed based on the second nonlinear function to obtain the i-th hidden processing result.

[0336] The i-th hidden forgetting result, the i-th hidden input result, the i-th hidden output result and the i-th hidden processing result are stored in the second storage unit.

[0337] In some embodiments, further comprising:

[0338] Processing the i-th first intermediate result based on floating-point multiplication to obtain an i-th multiplication result;

[0339] Processing the i-th multiplication result based on floating-point addition to obtain the i-th first addition result;

[0340] The i-th first addition result is processed based on the floating-point multiplication to obtain an i-th first nonlinear result.

[0341] In some embodiments, it also includes:

[0342] The i-th hidden and forgotten result and the i-1-th first addition result are processed based on the floating-point multiplication to obtain the i-th first multiplication result.

[0343] In some embodiments, it also includes:

[0344] Based on the floating-point multiplication processing, the i-th hidden input result and the i-th hidden processing result are processed to obtain the i-th second multiplication result.

[0345] In some embodiments, it also includes:

[0346] The i-th first multiplication result and the i-1-th second multiplication result are processed based on the floating-point addition to obtain the i-th first addition result.

[0347] In some embodiments, it also includes:

[0348] The i-th first addition result is processed based on the second nonlinear function to obtain an i-th intermediate nonlinear result.

[0349] In some embodiments, it also includes:

[0350] The i-th intermediate nonlinear result and the i-th hidden output result are processed based on the floating-point multiplication to obtain the i-th first nonlinear result.

[0351] The i-th first nonlinear result is stored in the external memory.

[0352] In some embodiments, it also includes:

[0353] The i-th first intermediate result is processed based on the floating-point multiplication to obtain the i-th second nonlinear result; the first intermediate result is the hidden forgotten result.

[0354] In some embodiments, it also includes:

[0355] The i-th hidden forgetting result and the i-1-th second nonlinear result are processed based on the floating-point multiplication to obtain the i-th second nonlinear result.

[0356] Part of the i-th hidden and forgotten result is stored in the third storage unit, and another part of the i-th hidden and forgotten result is calculated with the i-1-th second nonlinear result.

[0357] In some embodiments, it also includes:

[0358] Processing the i-th second nonlinear result based on the floating-point addition to obtain an i-th second addition result;

[0359] Processing the (i-1)th second nonlinear result and the (i-1)th third nonlinear result based on the floating-point multiplication to obtain an (i)th third multiplication result;

[0360] Processing the i-th second addition result and the i-th first intermediate result based on the floating-point multiplication to obtain an i-th fourth multiplication result;

[0361] The i-th third nonlinear result is obtained by processing the i-th third multiplication result and the i-th second addition result based on the floating-point addition.

[0362] In some embodiments, it also includes:

[0363] Based on the floating-point multiplication processing, the i-th second addition result and the i-th hidden processing result are processed to obtain the i-th fourth multiplication result; the first intermediate result is the hidden processing result.

[0364] In some embodiments, it also includes:

[0365] Processing the i-th nonlinear result based on floating-point division to obtain the i-th hidden division result;

[0366] Perform data format conversion on the i-th hidden division result to obtain the i-th hidden integer result;

[0367] Based on integer addition, i hidden integer results are processed to obtain the i-th target hidden result.

[0368] In the embodiment of the present application, the functions of LSTM or GRU can be realized without increasing the chip area by multiplexing the first matrix calculation module, the second matrix calculation module and the floating-point module by NPU. And the acceleration of matrix operations by the first matrix calculation module and the second matrix calculation module and the acceleration of nonlinear functions based on the floating-point module, the data processing involved can all be completed by NPU without the participation of CPU, which greatly improves the speed of neural network processing data. In addition, based on the acceleration of nonlinear functions based on floating-point modules, the sensitivity of data to precision will not lead to large differences, thereby improving the accuracy of the calculation results.

[0369] An embodiment of the present application further provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the functions of the above NPU when executing the computer program.

[0370] Taking computer equipment as the terminal as an example, Fig.19 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application, see Fig.19 The terminal 1900 may be a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a notebook computer, or a desktop computer. The terminal 1900 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0371] Typically, the terminal 1900 includes: a processor 1901 and a memory 1902 .

[0372] The processor 1901 may include one or more processing cores, such as a 4-core processor, a 5-core processor, etc. The processor 1901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0373] The memory 1902 may include one or more computer-readable storage media, which may be non-transitory. The memory 1902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1902 is used to store at least one program code, which is used to be executed by the processor 1901 to implement the functions of the NPU of the present application.

[0374] In some embodiments, the terminal 1900 may further optionally include: a peripheral device interface 1903 and at least one peripheral device. The processor 1901, the memory 1902 and the peripheral device interface 1903 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1903 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 1904, a camera assembly 1905, an audio circuit 1906 and a power supply 1907.

[0375] The peripheral device interface 1903 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1901 and the memory 1902. In some embodiments, the processor 1901, the memory 1902, and the peripheral device interface 1903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1901, the memory 1902, and the peripheral device interface 1903 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0376] The display screen 1904 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1904 is a touch display screen, the display screen 1904 also has the ability to collect touch signals on the surface or above the surface of the display screen 1904. The touch signal can be input to the processor 1901 as a control signal for processing. At this time, the display screen 1904 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1904 can be one, set on the front panel of the terminal 1900; in other embodiments, the display screen 1904 can be at least two, respectively set on different surfaces of the terminal 1900 or in a folding design; in other embodiments, the display screen 1904 can be a flexible display screen, set on the curved surface or folding surface of the terminal 1900. Even, the display screen 1904 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1904 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0377] The camera assembly 1905 is used to capture images or videos. In some embodiments, the camera assembly 1905 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1905 may also include a flash. The flash may be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0378] The audio circuit 1906 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1901 for processing. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1900. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1901 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1906 may also include a headphone jack.

[0379] The power supply 1907 is used to power various components in the terminal 1900. The power supply 1907 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1907 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0380] Those skilled in the art will understand that Fig.19 The structure shown in the figure does not constitute a limitation on the terminal 1900, and the terminal 1900 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0381] Take the computer device as a server as an example. Fig. 20 This is a structural diagram of a server provided in an embodiment of the present application. The server 2000 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 2001 and one or more memories 2002, wherein the one or more memories 2002 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 2001 to implement the above-mentioned NPU function. Of course, the server 2000 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 2000 may also include other components for implementing device functions, which will not be described in detail here.

[0382] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to perform the functions of the NPU as described above. Optionally, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0383] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0384] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An NPU, characterized in that: include: A first matrix calculation module, a second matrix calculation module and a floating point module; The first matrix calculation module and the second matrix calculation module are electrically connected to the floating point module respectively; A first matrix calculation module, used for obtaining the i-th initial data, processing the i-th initial data based on the first FC layer, and obtaining the i-th intermediate data; The i-th initial data includes the i-th weight data and the i-th image data; i is an integer greater than or equal to 2; The second matrix calculation module is used to obtain the i-1th target hidden result, and process the i-1th target hidden result based on the second FC layer to obtain the i-th intermediate hidden data; The i-1th target hidden result includes the i-1th target hidden data and the i-1th target hidden weight data; A floating point module, used for processing the i-th intermediate hidden data and the i-th intermediate data based on a first floating point algorithm to obtain an i-th hidden floating point result; The floating-point module is further used to process the i-th hidden floating-point result based on a nonlinear function and a second floating-point algorithm to obtain an i-th nonlinear result; The floating-point module is further used to process the i-th nonlinear result based on a third floating-point algorithm to obtain the i-th target hidden result.

2. The NPU according to claim 1, characterized in that The step of processing the i-th intermediate hidden data and the i-th intermediate data based on the first floating-point algorithm to obtain the i-th hidden floating-point result includes: Processing the i-th intermediate hidden data and the i-th intermediate data based on the subtraction to obtain the i-th hidden subtraction result; Perform data format conversion on the i-th hidden division result to obtain the i-th hidden floating-point data; Processing the i-th hidden floating-point result based on floating-point multiplication to obtain an i-th hidden multiplication result; The i-th hidden multiplication result is processed based on floating-point addition to obtain the i-th hidden floating-point result.

3. The NPU according to claim 1, characterized in that The step of processing the i-th hidden floating-point result based on the nonlinear function and the second floating-point algorithm to obtain the i-th nonlinear result includes: Processing the i-th hidden floating-point result based on the nonlinear function to obtain an i-th first intermediate result; The i-th first intermediate result is processed based on the second floating-point algorithm to obtain the i-th nonlinear result.

4. The NPU according to claim 3, characterized in that The step of processing the i-th hidden floating-point result based on the nonlinear function to obtain the i-th first intermediate result includes: Processing the i-th hidden floating point result based on the first nonlinear function to obtain the i-th hidden forgotten result; or Processing the i-th hidden floating-point result based on the first nonlinear function to obtain the i-th hidden input result; or Processing the i-th hidden floating-point result based on the first nonlinear function to obtain the i-th hidden output result; or The i-th hidden floating-point result is processed based on the second nonlinear function to obtain the i-th hidden processing result.

5. The NPU according to claim 4, characterized in that: The step of processing the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result includes: Processing the i-th first intermediate result based on floating-point multiplication to obtain an i-th multiplication result; Processing the i-th multiplication result based on floating-point addition to obtain an i-th first addition result; The i-th first addition result is processed based on the floating-point multiplication to obtain an i-th first nonlinear result.

6. The NPU according to claim 5, characterized in that: The step of processing the i-th first intermediate result based on floating-point multiplication to obtain the i-th multiplication result includes: Processing the i-th hidden-forgetting result and the i-1-th first addition result based on the floating-point multiplication to obtain an i-th first multiplication result; and The i-th hidden input result and the i-th hidden processing result are processed based on the floating-point multiplication to obtain an i-th second multiplication result.

7. The NPU according to claim 6, characterized in that: The step of processing the i-th multiplication result based on floating-point addition to obtain the i-th first addition result includes: The i-th first multiplication result and the i-1-th second multiplication result are processed based on the floating-point addition to obtain the i-th first addition result.

8. The NPU according to claim 7, characterized in that: Before the i-th first addition result is processed based on the floating-point multiplication to obtain the i-th first nonlinear result, the method further includes: The i-th first addition result is processed based on a second nonlinear function to obtain an i-th intermediate nonlinear result.

9. The NPU according to claim 8, characterized in that: The step of processing the i-th first addition result based on the floating-point multiplication to obtain the i-th first nonlinear result includes: The i-th intermediate nonlinear result and the i-th hidden output result are processed based on the floating-point multiplication to obtain the i-th first nonlinear result.

10. The NPU according to claim 9, characterized in that: The step of processing the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result includes: The i-th first intermediate result is processed based on the floating-point multiplication to obtain the i-th second nonlinear result; the first intermediate result is the hidden forgotten result.

11. The NPU according to claim 10, characterized in that: The step of processing the i-th first intermediate result based on the floating-point multiplication to obtain the i-th second nonlinear result includes: The i-th hidden and forgotten result and the i-1-th third nonlinear result are processed based on the floating-point multiplication to obtain the i-th second nonlinear result.

12. The NPU according to claim 11, characterized in that: The step of processing the i-th first intermediate result based on the second floating-point algorithm to obtain the i-th nonlinear result includes: Processing the (i-1)th second nonlinear result based on the floating-point addition to obtain an (i)th second addition result; Processing the i-1th second nonlinear result and the i-1th third nonlinear result based on the floating-point multiplication to obtain an i-th third multiplication result; and processing the i-th second addition result and the i-th first intermediate result based on the floating-point multiplication to obtain an i-th fourth multiplication result; The i-th fourth multiplication result and the i-th second addition result are processed based on the floating-point addition to obtain the i-th third nonlinear result.

13. The NPU according to claim 12, characterized in that: The processing, based on the floating-point multiplication, of the i-th second addition result and the i-th first intermediate result to obtain the i-th fourth multiplication result includes: Based on the floating-point multiplication processing, the i-th second addition result and the i-th hidden processing result are processed to obtain the i-th fourth multiplication result; the first intermediate result is the hidden processing result.

14. The NPU according to claim 13, characterized in that: The step of processing the i-th nonlinear result based on the third floating-point algorithm to obtain the i-th target hidden result includes: Processing the i-th nonlinear result based on floating-point division to obtain an i-th hidden division result; Performing data format conversion on the i-th hidden division result to obtain the i-th hidden integer result; The i-th target hidden result is obtained by processing the i-th hidden integer results based on integer addition.

15. A method for accelerating a neural network, characterized in that: include: Obtaining the i-th initial data, and processing the i-th initial data based on the first FC layer to obtain the i-th intermediate data; The i-th initial data includes the i-th weight data and the i-th image data; i is an integer greater than or equal to 2; Obtain the i-1th target hidden result, and process the i-1th target hidden result based on the second FC layer to obtain the i-th intermediate hidden data; The i-1th target hidden result includes the i-1th target hidden data and the i-1th target hidden weight data target hidden result; Processing the i-th intermediate hidden data and the i-th intermediate data based on the first floating-point algorithm to obtain an i-th hidden floating-point result; Processing the i-th hidden floating-point result based on the nonlinear function and the second floating-point algorithm to obtain an i-th nonlinear result; The i-th nonlinear result is processed based on the third floating-point algorithm to obtain the i-th target hidden result.

16. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the method for accelerating a neural network as described in claim 15.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the method for accelerating a neural network as described in claim 15.

Images