Matchable multifunctional vector calculation device suitable for neural network

Abstract

The invention provides a configurable multifunctional vector calculation device suitable for a neural network, and the device comprises a storage control module which is configured to obtain to-be-calculated data from the neural network, and store the to-be-calculated data in a corresponding register; the operation code storage module is configured to obtain and store a plurality of operation codes from the neural network; the floating point calculation module is configured to perform target calculation on the to-be-calculated data according to the operation code; the target calculation comprises basic calculation, SRT calculation and rapid calculation; and the state machine module is configured to select the corresponding operation code from the operation code storage module and the corresponding to-be-calculated data from the storage control module according to the calculation instruction, and send the operation code and the to-be-calculated data to the floating point calculation module for target calculation of the corresponding operation code. According to the computing device, the problems that an existing floating point computing technology is poor in flexibility and low in universality are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of activation function calculation, and in particular to a configurable multifunctional vector calculation device suitable for neural networks. Background Art

[0002] With the development of neural networks, the existing floating-point computing modules need to have more powerful functions, and the demand for transcendental function calculations is becoming more urgent, and is no longer limited to basic function operations. At present, the main calculation methods for transcendental functions are Taylor polynomial expansion and CORDIC (Coordinate Rotation Digital Computer) algorithm. When the number of expansion items of the former is low, it is difficult to achieve the corresponding accuracy, and a large number of items will lead to a sharp increase in hardware resource consumption. The latter requires multiple iterations to achieve a higher accuracy, and the calculation cycle is long.

[0003] Neural networks introduce nonlinearity through activation functions, which enables the network to learn and simulate complex function mappings. However, at the same time, the increasingly complex and diverse activation functions of neural networks are difficult to deploy on edge devices with limited resources and storage through floating-point computing modules with only specific functions.

[0004] In addition, the lack of pipelines in computing units results in long computing cycles and greatly reduces computing efficiency. For neural networks, the loss of precision caused by low-precision calculations may cause the entire model to fail, while high-precision calculations result in lengthy computing cycles and usually require a lot of computing time and power consumption. The single input data format and function limit the scope of application of computing units, lacking flexibility and versatility. Summary of the invention

[0005] The present application provides a configurable multifunctional vector computing device suitable for neural networks to solve the problems of poor flexibility and low versatility of existing floating-point computing technologies.

[0006] The computing device comprises:

[0007] A storage control module, wherein the storage control module is configured to obtain data to be calculated from the neural network and store the data to be calculated in a corresponding register, wherein the data to be calculated includes intermediate result data, residual data and other parameter data;

[0008] An operation code storage module, wherein the operation code storage module is configured to obtain and store a plurality of operation codes from the neural network, wherein the operation codes are used to call one or more of the data to be calculated to perform target calculation;

[0009] A floating-point calculation module, wherein the floating-point calculation module is configured to perform target calculation on the data to be calculated according to the operation code, and the floating-point calculation module generates a return result after completing any one of the target calculations; the target calculation includes basic calculation, SRT calculation and fast calculation, and the basic calculation, the SRT calculation and the fast calculation all have multiple calculation types;

[0010] A state machine module, wherein the state machine module is configured to select the corresponding operation code from the operation code storage module and the corresponding data to be calculated from the storage control module according to the calculation instruction, and send the operation code and the data to be calculated to the floating-point calculation module to perform the target calculation corresponding to the operation code to obtain the target calculation result; the return result is used to indicate to the state machine module that the previous calculation in the target calculation has been completed and to perform the next calculation in the target calculation; the target calculation result is used to characterize the nonlinear characteristics in the neural network.

[0011] Preferably, the storage control module includes:

[0012] A first format conversion unit, wherein the first format conversion unit is configured to convert the format of the intermediate result data from an initial format to an FP32 format, wherein the initial format of the intermediate result data includes an FP32 format, an INT32 format, an FP16 format, and a BF16 format;

[0013] A second format conversion unit, wherein the second format conversion unit is configured to convert a format of the residual data from an initial format to an FP32 format, wherein the initial format of the residual data includes an FP16 format, a BF16 format, an INT4 format, an INT8 format, and an INT16 format;

[0014] A first register, wherein the first register is configured to store the intermediate result data after the format conversion by the first format conversion unit is completed;

[0015] A second register, wherein the second register is configured to store the residual data after the format conversion by the second format conversion unit is completed.

[0016] a third register, the third register being configured to store the other parameter data;

[0017] A data calling unit, wherein the data calling unit is configured to obtain data to be calculated from the neural network, store corresponding data in the data to be calculated in the first register, the second register and the third register, and store the target calculation result in the corresponding memory of the neural network.

[0018] Preferably, the storage control module further includes:

[0019] A third format conversion unit, wherein the third format conversion unit is configured to convert the target calculation result, the intermediate result data after the format conversion by the first format conversion unit, and the target calculation result from FP32 format to a target output format; the target output format includes INT32 format, FP16 format, BF16 format, INT4 format, INT8 format and INT16 format.

[0020] Preferably, the floating-point calculation module comprises:

[0021] a basic calculation unit, the basic calculation unit being configured to perform a basic calculation corresponding to the target calculation according to the operation code;

[0022] An SRT calculation unit, the SRT calculation unit being configured to perform an SRT calculation corresponding to the target calculation according to the operation code;

[0023] A fast calculation unit is configured to perform a fast calculation corresponding to the target calculation according to the operation code.

[0024] Preferably, the basic computing unit includes:

[0025] A comparison component, wherein the comparison calculation component is configured to compare multiple data in the data to be calculated according to the operation code and output a comparison result;

[0026] a multiplication-addition fusion calculation component, wherein the multiplication-addition fusion calculation component is configured to perform multiplication-addition fusion calculation on a plurality of data in the data to be calculated according to the operation code;

[0027] An addition calculation component, wherein the addition calculation component is configured to perform addition calculation on one or more data in the data to be calculated according to the operation code;

[0028] a multiplication calculation component, the multiplication calculation component being configured to perform a multiplication calculation on one or more data in the data to be calculated according to the operation code;

[0029] A bit operation component is configured to perform decimal point alignment on one type of data to be calculated according to the operation code, and output an alignment result.

[0030] Preferably, the SRT calculation unit includes:

[0031] A division preprocessing component, wherein the division preprocessing component is configured to perform division preprocessing on one of the data to be calculated according to the operation code, and the division preprocessing is used to obtain an initial partial remainder result and a division intermediate result of the corresponding SRT16 division calculation;

[0032] A root preprocessing component, the root preprocessing component is configured to perform root preprocessing on one of the data to be calculated according to the operation code, the root preprocessing being used to obtain an initial partial remainder result and a root intermediate result of the corresponding SRT16 root calculation;

[0033] An SRT algorithm component, wherein the SRT algorithm component is configured to perform the SRT16 division calculation and / or the SRT16 root extraction calculation according to the initial partial remainder result, the division intermediate result and / or the root extraction intermediate result to obtain an SRT16 division result and / or an SRT16 root extraction result;

[0034] A post-division processing component, wherein the post-division processing component is configured to perform normalization processing on the SRT16 division result;

[0035] A post-rooting processing component is configured to perform normalization processing on the SRT16 rooting result.

[0036] Preferably, the fast calculation unit comprises:

[0037] A transcendental function preprocessing component, wherein the transcendental function preprocessing component is configured to perform transcendental function preprocessing on one of the data to be calculated according to the operation code, wherein the transcendental function preprocessing is used to obtain a table lookup index, a polynomial calculation input, and an intermediate calculation result of a corresponding transcendental function;

[0038] A fast activation function preprocessing component, wherein the fast activation function preprocessing component is configured to perform fast activation function preprocessing on one of the data to be calculated according to the operation code, wherein the fast activation function preprocessing is used to obtain a table lookup index, a polynomial calculation input, and an intermediate calculation result of a corresponding activation function;

[0039] A fast calculation component, wherein the fast calculation component is configured to perform approximate calculations on the table lookup index corresponding to the transcendental function and / or the activation function, the polynomial calculation input, and the intermediate calculation result using polynomial fitting to obtain an activation function result and / or a transcendental function result;

[0040] a transcendental function post-processing component, wherein the transcendental function post-processing component is configured to perform normalization processing on the transcendental function result;

[0041] A fast activation function post-processing component is configured to perform normalization processing on the activation function result.

[0042] Preferably, the storage control module further includes:

[0043] a fourth register, the fourth register being configured to temporarily store immediate data obtained by decoding the opcode;

[0044] A fifth register, wherein the fifth register includes a plurality of sub-registers, and the fifth register is configured to store the target calculation result and the returned result.

[0045] Preferably, the computing device further comprises:

[0046] A selector module is configured to select output and input contents of the storage control module, the operation code storage module, the floating point calculation module and the state machine module according to the operation code.

[0047] Preferably, the selector module comprises:

[0048] A first selector, the first selector being configured to select the intermediate result data via the first format conversion unit;

[0049] a second selector configured to select the residual data passed through the second format conversion unit;

[0050] a third selector, the third selector being configured to select the immediate data in the fourth register, select the return result in the fifth register, select the residual data in the second register according to the operation code, and select the other parameter data in the third register according to the operation code;

[0051] a fourth selector, the fourth selector being configured to select the target calculation result to be output in the fifth register and the intermediate result data in the first register according to the operation code;

[0052] a fifth selector, the fifth selector being configured to select the intermediate result data in the first register according to the operation code, select the target calculation result according to the operation code, and send the target calculation result to the third format conversion unit;

[0053] a sixth selector, the sixth selector being configured to select the target calculation result via the third format conversion unit according to the operation code, and send the target calculation result to the data calling unit;

[0054] a seventh selector, the seventh selector being configured to select the target calculation result output by the floating-point calculation module;

[0055] an eighth selector configured to select a sub-register in the fifth register storing the target calculation result;

[0056] a ninth selector, wherein the ninth selector is configured to select the return result to be output from a corresponding sub-register of the fifth register according to the operation code;

[0057] A tenth selector is configured to select the target calculation result to be output from the corresponding sub-register of the fifth register according to the operation code.

[0058] It can be seen from the above content that the present application provides a configurable multifunctional vector computing device suitable for a neural network, the computing device comprising a storage control module, the storage control module being configured to obtain data to be calculated from the neural network and store the data to be calculated in corresponding registers, the data to be calculated comprising intermediate result data, residual data and other parameter data; an operation code storage module, the operation code storage module being configured to obtain and store a plurality of operation codes from the neural network, the operation code being used to call one of the data to be calculated for target calculation; a floating-point calculation module, the floating-point calculation module being configured to perform target calculation on the data to be calculated according to the operation code, the floating-point calculation module performing any calculation in the completion of the target calculation Then generate a return result; the target calculation includes basic calculation, SRT calculation and fast calculation, and the basic calculation, the SRT calculation and the fast calculation all have multiple calculation types; the state machine module, the state machine module is configured to select the corresponding operation code from the operation code storage module and the corresponding data to be calculated from the storage control module according to the calculation instruction, and send the operation code and the data to be calculated to the floating-point calculation module to perform the target calculation corresponding to the operation code to obtain the target calculation result; the return result is used to indicate to the state machine module that the previous calculation in the target calculation has been completed and to perform the next calculation in the target calculation; the target calculation result is used to characterize the nonlinear characteristics in the neural network. The present application solves the problems of poor flexibility and low versatility of existing floating-point calculation technology through the above-mentioned calculation device. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0060] Figure 1 A schematic diagram of a multifunctional vector computing device applicable to a neural network in the present application;

[0061] Figure 2 This is a schematic diagram of a floating-point computing module in a multifunctional vector computing device applicable to a neural network in the present application;

[0062] Figure 3 A schematic diagram of register data flow of a multifunctional vector computing device applicable to a neural network in the present application;

[0063] Figure 4 A schematic diagram of data flow of a floating-point computing module of a multifunctional vector computing device applicable to a neural network in the present application;

[0064] Figure 5 This application discloses a state data diagram of a state machine module in a multifunctional vector computing device suitable for a neural network. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Figure 1 This is a schematic diagram of a multifunctional vector computing device suitable for a neural network according to the present application.

[0067] See also Figure 1 It can be seen that this embodiment provides a configurable multifunctional vector computing device suitable for a neural network, and the computing device includes:

[0068] The storage control module 100 is configured to obtain the data to be calculated from the neural network and store the data to be calculated in the corresponding register, wherein the data to be calculated includes intermediate result data, residual data and other parameter data. Specifically, in the present embodiment, the storage control module 100 is used to obtain the data to be calculated from the neural network, wherein the data to be calculated includes intermediate result data (acc), residual data (resadd) and other parameter data (para).

[0069] Among them, the intermediate result data (acc) is obtained from the intermediate result data random access memory (acc ram) in the neural network, the residual data (resadd) is obtained from the residual data random access memory (resadd ram) in the neural network, and the other parameter data (para) is obtained from the other parameter data random access memory (para ram) in the neural network.

[0070] The intermediate result data (acc) may be the calculation output of the convolution or matrix multiplication unit in the neural network, or the calculation result of the computing device, and the size is 2048×8192.

[0071] The residual data (resadd) is the data required to calculate the residual block of the neural network, and its size is 2048×4096.

[0072] The other parameter data (para) include bias, quantization factor (scale), etc., and the size is 2048×64.

[0073] Figure 3 This is a schematic diagram of register data flow of a multifunctional vector computing device suitable for a neural network according to the present application.

[0074] See also Figure 3 It can be seen that, further, in some embodiments, the storage control module 100 includes:

[0075] A first format conversion unit 110, wherein the first format conversion unit 110 is configured to convert the format of the intermediate result data from an initial format to an FP32 format, wherein the initial format of the intermediate result data includes an FP32 format, an INT32 format, an FP16 format, and a BF16 format;

[0076] A second format conversion unit 120, the second format conversion unit 120 is configured to convert the format of the residual data from an initial format to an FP32 format, the initial format of the residual data including an FP16 format, a BF16 format, an INT4 format, an INT8 format, and an INT16 format;

[0077] A first register 130, wherein the first register 130 is configured to store the intermediate result data after the format conversion by the first format conversion unit 110 is completed;

[0078] The second register 140 is configured to store the residual data after the format conversion by the second format conversion unit 120 is completed.

[0079] A third register 150, the third register 150 is configured to store the other parameter data;

[0080] The data calling unit 160 is configured to obtain the data to be calculated from the neural network, store the corresponding data in the data to be calculated in the first register 130, the second register 140 and the third register 150, and store the target calculation result in the corresponding memory of the neural network.

[0081] Specifically, in this embodiment, the first register 130 , the second register 140 and the third register 150 are respectively used to temporarily store corresponding data, and the data stored in each register is called through the data calling unit 160 .

[0082] The first format conversion unit 110 and the second format conversion unit 120 are used to convert input data into a specified FP32 format.

[0083] It should be noted that, according to different requirements, the data to be calculated may also be directly calculated and processed in the original format without going through the format conversion of the first format conversion unit 110 and the second format conversion unit 120 .

[0084] Furthermore, in some embodiments, the storage control module 100 further includes:

[0085] The third format conversion unit 170 is configured to convert the target calculation result, the intermediate result data after the format conversion by the first format conversion unit 110, and the target calculation result from FP32 format to a target output format; the target output format includes INT32 format, FP16 format, BF16 format, INT4 format, INT8 format and INT16 format.

[0086] Specifically, in this embodiment, the third format conversion unit 170 plays the same effect as the first format conversion unit 110 and the second format conversion unit 120, both of which convert the format of the data to be calculated, and the data to be calculated and the data obtained after calculation can also be processed in the FP32 format without passing through the third format conversion unit 170.

[0087] The computing device further comprises:

[0088] An operation code storage module 200 is configured to obtain and store a plurality of operation codes from a neural network, wherein the operation code is used to call one of the data to be calculated to perform a target calculation. Specifically, in this embodiment, the operation code storage module 200 is used to store the operation code, and when there is a calculation requirement, the corresponding operation code is retrieved from the operation code storage module 200.

[0089] The computing device further comprises:

[0090] A floating-point calculation module 300, wherein the floating-point calculation module 300 is configured to perform target calculation on the data to be calculated according to the operation code, and the floating-point calculation module 300 generates a return result after completing any one of the target calculations; the target calculation includes basic calculation, SRT calculation and fast calculation, and the basic calculation, the SRT calculation and the fast calculation all have multiple types of calculations. Specifically, in this embodiment, the floating-point calculation module 300 is used to perform target calculation on the data to be calculated, wherein the core of this embodiment is to perform different calculations on the data to be calculated, so the target calculation includes several types of calculations, specifically including the basic calculation, the SRT calculation and the fast calculation, wherein the basic calculation, the SRT calculation and the fast calculation each have several independent calculations, and the return result is generated after one of the independent calculations of the basic calculation, the SRT calculation and / or the fast calculation is completed.

[0091] The computing device further comprises:

[0092] The state machine module 400 is configured to select the corresponding operation code from the operation code storage module 200 and the corresponding data to be calculated from the storage control module 100 according to the calculation instruction, and send the operation code and the data to be calculated to the floating-point calculation module 300 to perform the target calculation corresponding to the operation code to obtain the target calculation result; the return result is used to indicate that the previous calculation in the target calculation of the state machine module 400 has been completed and the next calculation in the target calculation is performed; the target calculation result is used to characterize the nonlinear characteristics in the neural network. Specifically, in this embodiment, the state machine module 400 can be regarded as a coordinated controller for the entire set of computing devices, and the corresponding computing instructions are issued through the state machine module 400. The means for issuing the computing instructions is to complete them by calling the operation code, and the progress of the calculation is monitored by the return result fed back after each calculation is completed.

[0093] Figure 2 This is a schematic diagram of a floating-point computing module in a configurable multifunctional vector computing device suitable for a neural network according to the present application.

[0094] See also Figure 2 It can be seen that, further, in some embodiments, the floating point calculation module 300 includes:

[0095] A basic calculation unit 310, wherein the basic calculation unit 310 is configured to perform a basic calculation corresponding to the target calculation according to the operation code;

[0096] An SRT calculation unit 320, the SRT calculation unit 320 is configured to perform an SRT calculation corresponding to the target calculation according to the operation code;

[0097] The fast calculation unit 330 is configured to perform a fast calculation corresponding to the target calculation according to the operation code.

[0098] Specifically, in this embodiment, the basic calculation, the SRT calculation and the fast calculation in the target calculation are respectively completed by the basic calculation unit 310, the SRT calculation unit 320 and the fast calculation unit 330 which are independent of each other.

[0099] Figure 4 This is a schematic diagram of the data flow of a floating-point computing module of a multifunctional vector computing device suitable for a neural network in the present application.

[0100] See also Figure 4 It can be seen that the basic calculation unit 310, the SRT calculation unit 320 and the fast calculation unit 330 each have a number of different calculation components, wherein the basic calculation unit 310 includes:

[0101] A comparison component 311, wherein the comparison calculation component 311 is configured to compare multiple data in the data to be calculated according to the operation code and output a comparison result;

[0102] A multiplication-addition fusion calculation component 312, wherein the multiplication-addition fusion calculation component 312 is configured to perform multiplication-addition fusion calculation on multiple types of data in the data to be calculated according to the operation code;

[0103] An addition calculation component 313, wherein the addition calculation component 313 is configured to perform addition calculation on one or more data in the data to be calculated according to the operation code;

[0104] A multiplication calculation component 314, wherein the multiplication calculation component 314 is configured to perform a multiplication calculation on one or more data in the data to be calculated according to the operation code;

[0105] The bit operation component 315 is configured to perform decimal point alignment on one of the data to be calculated according to the operation code, and output an alignment result.

[0106] Specifically, in this embodiment, the basic computing unit 310 includes the comparison component 311, the multiplication-addition fusion computing component 312, the addition computing component 313, the multiplication computing component 314 and the bit operation component 315. The addition computing component 313 and the multiplication computing component 314 are both implemented by a first-level pipeline and complete the calculation in two time cycles; the multiplication-addition fusion computing component 312 is implemented by a second-level pipeline and completes the calculation in three time cycles; the comparison component 311 has two functions, comp_larger and comp_smaller. When operand 1 is greater than operand 2, comp_larger outputs operand 3, otherwise it outputs operand 4. When operand 1 is less than operand 2, comp_smaller outputs the result of operand 3, otherwise it outputs operand 4. Wherein, operand 1, operand 2, operand 3 and operand 4 refer to Figure 4 .

[0107] The SRT calculation unit 320 includes:

[0108] A division preprocessing component 321, wherein the division preprocessing component 321 is configured to perform division preprocessing on one of the data to be calculated according to the operation code, and the division preprocessing is used to obtain an initial partial remainder result and a division intermediate result of the corresponding SRT16 division calculation;

[0109] A root preprocessing component 322, the root preprocessing component 322 is configured to perform root preprocessing on one of the data to be calculated according to the operation code, the root preprocessing is used to obtain an initial partial remainder result and a root intermediate result of the corresponding SRT16 root calculation;

[0110] An SRT algorithm component 323, wherein the SRT algorithm component 323 is configured to perform the SRT16 division calculation and / or the SRT16 root extraction calculation according to the initial partial remainder result, the division intermediate result and / or the root extraction intermediate result to obtain an SRT16 division result and / or an SRT16 root extraction result;

[0111] A post-division processing component 324, wherein the post-division processing component 324 is configured to perform normalization processing on the SRT16 division result;

[0112] The post-rooting processing component 325 is configured to perform normalization processing on the SRT16 rooting result.

[0113] Specifically, in this embodiment, the SRT calculation unit 320 uses the SRT16 algorithm to calculate the division and square root functions. In the SRT16 algorithm, the iterative equation for taking the root is:

[0114] w[j+1]=16w[j]-2S[j]s j+1 -s j+1 2 16 -(j+1) ;

[0115] Where w[j+1] and w[j] are the partial remainder results of the j+1th and jth calculations, respectively. When j=0, w[0] is the initial partial remainder result of the SRT16 root calculation. S[j] is the SRT16 partial root calculation result, s j+1 It is the next digit of the SRT16 root result. j+1 It is determined by looking up the remainder 16w[j] and 2S[j] after the shift.

[0116] The iterative equation for division is:

[0117] w[j+1]=16w[j]-dq j+1 ;

[0118] w[j+1] and w[j] are the partial remainder results of the j+1th and jth calculations, respectively. When j=0, w[0] is the initial partial remainder result of the SRT16 division calculation, d is the divisor, and q j+1 It is the next SRT16 division result. Due to the similarity between division and square root, the lookup intervals of division and square root are combined and the same lookup table is used.

[0119] The fast calculation unit 330 includes:

[0120] A transcendental function preprocessing component 331, wherein the transcendental function preprocessing component 331 is configured to perform transcendental function preprocessing on one of the data to be calculated according to the operation code, wherein the transcendental function preprocessing is used to obtain a table lookup index, a polynomial calculation input, and an intermediate calculation result of a corresponding transcendental function;

[0121] A fast activation function preprocessing component 332, wherein the fast activation function preprocessing component 332 is configured to perform fast activation function preprocessing on one of the data to be calculated according to the operation code, wherein the fast activation function preprocessing is used to obtain a table lookup index, a polynomial calculation input, and an intermediate calculation result of a corresponding activation function;

[0122] A fast calculation component 333, wherein the fast calculation component 333 is configured to perform approximate calculations on the table lookup index corresponding to the transcendental function and / or the activation function, the polynomial calculation input, and the intermediate calculation result using polynomial fitting to obtain an activation function result and / or a transcendental function result;

[0123] A transcendental function post-processing component 334, wherein the transcendental function post-processing component 334 is configured to perform normalization processing on the transcendental function result;

[0124] The fast activation function post-processing component 335 is configured to perform normalization processing on the activation function result.

[0125] Specifically, in this embodiment, the fast calculation unit 330 uses a polynomial fitting method to approximate the function, and expands different functions including sine function, cosine function, reciprocal, reciprocal of square root, logarithm with base 2, exponential with base 2, tanh, sigmoid, swish, mish, gelu into quadratic polynomials, and stores the polynomial coefficients of different sections of the transcendental function obtained by using the maximum and minimum approximation method in a lookup table. The input data is pre-processed to obtain the corresponding intermediate calculation results and the lookup table index, and the polynomial calculation input is obtained through the lookup table. The processed intermediate calculation results and polynomial calculation input are sent to the fast calculation component 333, and finally the final output is obtained after the corresponding processing. The fast calculation unit 330 uses fixed-point calculation internally, which greatly reduces the hardware complexity compared to floating-point calculation. The calculation does not need to go through multiple iterations, and adopts a three-stage pipeline design to further improve the efficiency of transcendental function calculation.

[0126] Furthermore, in some embodiments, the storage control module 100 further includes:

[0127] a fourth register 180, the fourth register 180 being configured to temporarily store immediate data obtained by decoding the opcode;

[0128] The fifth register 190 includes a plurality of sub-registers, and the fifth register 190 is configured to store the target calculation result and the returned result.

[0129] Specifically, the fourth register 180 and the fifth register 190 have the same effect as the above-mentioned registers, and both temporarily store corresponding data.

[0130] See also Figure 1 It is also known that, further, in some embodiments, the computing device further includes:

[0131] The selector module 500 is configured to select output and input contents of the storage control module 100 , the operation code storage module 200 , the floating point calculation module 300 and the state machine module 400 according to the operation code.

[0132] Specifically, in this embodiment, since this embodiment aims to achieve different processing for different data, which results in an increase in the types of data, this embodiment sets the selector module 500 to select different data and perform input, calculation and output according to the selected data.

[0133] See also Figure 3 and Figure 4 It can be seen that, further, in some embodiments, the selector module 500 includes:

[0134] A first selector 511, wherein the first selector 511 is configured to select the intermediate result data via the first format conversion unit 110;

[0135] A second selector 512, the second selector 512 is configured to select the residual data via the second format conversion unit 120;

[0136] A third selector 513, wherein the third selector 513 is configured to select the immediate data in the fourth register 180, select the return result in the fifth register 190, select the residual data in the second register 140 according to the operation code, and select the other parameter data in the third register 150 according to the operation code;

[0137] a fourth selector 514, wherein the fourth selector 514 is configured to select the target calculation result to be output in the fifth register 190 and the intermediate result data in the first register 130 according to the operation code;

[0138] a fifth selector 515, wherein the fifth selector 515 is configured to select the intermediate result data in the first register 130 according to the operation code, select the target calculation result according to the operation code, and send the target calculation result to the third format conversion unit 170;

[0139] a sixth selector 516, the sixth selector 516 being configured to select the target calculation result via the third format conversion unit 170 according to the operation code, and send the target calculation result to the data calling unit 160;

[0140] a seventh selector 521, wherein the seventh selector 521 is configured to select the target calculation result output by the floating-point calculation module 300;

[0141] an eighth selector 522, the eighth selector 522 being configured to select a sub-register in the fifth register 190 storing the target calculation result;

[0142] a ninth selector 523, wherein the ninth selector 523 is configured to select the return result to be output from the corresponding sub-register of the fifth register 190 according to the operation code;

[0143] The tenth selector 524 is configured to select the target calculation result to be output from the corresponding sub-register of the fifth register 190 according to the operation code.

[0144] Specifically, in this embodiment, various selectors are used to implement input, calculation, and output of various types of data in the computing device, and the function of storing different data in different registers is added to facilitate subsequent calculation calls.

[0145] Figure 5 This application discloses a state data diagram of a state machine module in a multifunctional vector computing device suitable for a neural network.

[0146] See also Figure 5 It can be seen that the state machine module 400 processes data in the following embodiments:

[0147] IDLE: Idle state. When idle_insn_read_done is pulled high, it means that the instruction (insn) reading has been completed and the state jumps to DATAPREPARE.

[0148] DATAPREPARE: Data preparation stage, through the configuration information of insn, the read enable signal and read address are sent to each sram. This stage completes the reading of acc_ram, resadd_ram, para_ram, opcode_ram and other data.

[0149] The data_prepare_done signal is pulled high to indicate that the data has been read and the state is switched to COMPUTE.

[0150] COMPUTE: The compute_done signal is pulled high, indicating that the calculation of an operation is completed, and jumps to OPCODEPREPARE to configure the next opcode. The fpu_done signal is pulled high, indicating that all operation steps have been completed for the current data, and jumps to the WRITE state to write back the calculated data. The change_para signal is pulled high, indicating that another set of para data is needed for the next calculation, and jumps to the CHANGE PARA state. The vcu_execute_done signal is pulled high, indicating that all data have been processed, and jumping to the DONE state at this time indicates that the calculation is completed.

[0151] OPCODE PREPARE: The next cycle unconditionally jumps to the COMPUTE state, which is used to load the next opcode.

[0152] CHANGE PARA: The next cycle unconditionally jumps to the DATAPREPARE state to load the data after switching the para_ram type.

[0153] WRITE: Write the calculation results back to the on-chip storage. The wready signal is pulled high, indicating that the result data has been written back and jumps to the DATAPREPARE state to prepare for the next batch of uncalculated data.

[0154] DONE: Indicates that the calculation is completed. After the last data is written back to the on-chip storage space, the wready signal is pulled high and returns to the IDLE state.

[0155] This embodiment has the following advantages:

[0156] This embodiment designs instructions and opcodes to achieve fine-grained functional splitting, thereby controlling the combination of various basic operators, with flexible configuration capabilities, and can be adjusted according to the needs of different application scenarios, providing more computing possibilities. Applicable practical scenarios include activation function calculation, convolution layer bias addition, residual calculation, quantization and dequantization, softmax operation in transformer, Butterworth filtering, etc.

[0157] The vector computing device supports basic floating-point computing operators including multiplication, addition, and multiplication-addition fusion operations; uses a polynomial fitting method to calculate transcendental functions, including sine function, cosine function, exponential with base 2, logarithm with base 2, reciprocal of square root, and reciprocal, which greatly reduces the calculation time and improves the utilization rate of hardware resources; supports high-precision division and square root calculation with an accuracy of 0ulp; supports tanh, sigmoid, swish, mish, and gelu activation function calculations in both high-precision and fast calculation precision modes, supports relu and selu function calculations, and can configure leakyrelu (α) and softplus (β, threshold) functions with different parameters, wherein the high-precision activation function is controlled by a state machine to complete the calculation, and the fast calculation activation function is implemented by a polynomial fitting method.

[0158] The computing device supports multiple data format conversion. The internal calculation of the device uses the fp32 data format to obtain high-precision results. The input / output data type conversion module can complete the conversion between int4, int8, int16, int32, fp16, bf16 and fp32 formats.

Claims

1. A multifunctional vector computing device suitable for neural networks, characterized in that: The computing device comprises: A storage control module (100), the storage control module (100) being configured to obtain data to be calculated from the neural network and store the data to be calculated in a corresponding register, the data to be calculated comprising intermediate result data, residual data and other parameter data; An operation code storage module (200), the operation code storage module (200) being configured to obtain and store a plurality of operation codes from the neural network, the operation codes being used to call one or more data in the data to be calculated to perform target calculation; A floating-point calculation module (300), the floating-point calculation module (300) being configured to perform target calculation on the data to be calculated according to the operation code, the floating-point calculation module (300) generating a return result after completing any one of the target calculations; the target calculations include basic calculations, SRT calculations and fast calculations, and the basic calculations, the SRT calculations and the fast calculations all have multiple calculation types; A state machine module (400) is configured to select the corresponding operation code from the operation code storage module (200) and the corresponding data to be calculated from the storage control module (100) according to a calculation instruction, and send the operation code and the data to be calculated to the floating point calculation module (300) to perform a target calculation corresponding to the operation code to obtain a target calculation result; the return result is used to indicate to the state machine module (400) that the previous calculation in the target calculation has been completed and to perform the next calculation in the target calculation; the target calculation result is used to characterize the nonlinear characteristics in the neural network.

2. A configurable multifunctional vector computing device suitable for a neural network according to claim 1, characterized in that: The storage control module (100) comprises: A first format conversion unit (110), the first format conversion unit (110) being configured to convert the format of the intermediate result data from an initial format to an FP32 format, wherein the initial format of the intermediate result data includes an FP32 format, an INT32 format, an FP16 format, and a BF16 format; A second format conversion unit (120), the second format conversion unit (120) being configured to convert the format of the residual data from an initial format to an FP32 format, the initial format of the residual data comprising an FP16 format, a BF16 format, an INT4 format, an INT8 format and an INT16 format; A first register (130), the first register (130) being configured to store the intermediate result data after the format conversion by the first format conversion unit (110); A second register (140), the second register (140) being configured to store the residual data after the format conversion by the second format conversion unit (120); a third register (150), the third register (150) being configured to store the other parameter data; A data calling unit (160), wherein the data calling unit (160) is configured to obtain data to be calculated from the neural network, store corresponding data in the data to be calculated in the first register (130), the second register (140) and the third register (150), and store the target calculation result in the corresponding memory of the neural network.

3. A configurable multifunctional vector computing device suitable for a neural network according to claim 2, characterized in that: The storage control module (100) further comprises: A third format conversion unit (170), the third format conversion unit (170) is configured to convert the target calculation result, the intermediate result data that has completed the format conversion of the first format conversion unit (110), and the target calculation result from the FP32 format to a target output format; the target output format includes an INT32 format, a FP16 format, a BF16 format, an INT4 format, an INT8 format, and an INT16 format.

4. A configurable multifunctional vector computing device suitable for a neural network according to claim 3, characterized in that: The floating point calculation module (300) comprises: A basic calculation unit (310), the basic calculation unit (310) being configured to perform a basic calculation corresponding to the target calculation according to the operation code; An SRT calculation unit (320), the SRT calculation unit (320) being configured to perform an SRT calculation corresponding to the target calculation according to the operation code; A fast calculation unit (330), wherein the fast calculation unit (330) is configured to perform a fast calculation corresponding to the target calculation according to the operation code.

5. A configurable multifunctional vector computing device suitable for a neural network according to claim 4, characterized in that: The basic computing unit (310) comprises: A comparison component (311), wherein the comparison calculation component (311) is configured to compare multiple types of data in the data to be calculated according to the operation code and output a comparison result; A multiplication-addition fusion calculation component (312), wherein the multiplication-addition fusion calculation component (312) is configured to perform multiplication-addition fusion calculation on multiple types of data in the data to be calculated according to the operation code; An addition calculation component (313), the addition calculation component (313) being configured to perform addition calculation on one or more data in the data to be calculated according to the operation code; A multiplication calculation component (314), wherein the multiplication calculation component (314) is configured to perform a multiplication calculation on one or more data in the data to be calculated according to the operation code; A bit operation component (315) is configured to perform decimal point alignment on one of the data to be calculated according to the operation code, and output an alignment result.

6. A configurable multifunctional vector computing device suitable for a neural network according to claim 5, characterized in that: The SRT calculation unit (320) comprises: A division preprocessing component (321), the division preprocessing component (321) being configured to perform division preprocessing on one of the data to be calculated according to the operation code, the division preprocessing being used to obtain an initial partial remainder result and a division intermediate result of a corresponding SRT16 division calculation; A root opening preprocessing component (322), the root opening preprocessing component (322) being configured to perform root opening preprocessing on one of the data to be calculated according to the operation code, the root opening preprocessing being used to obtain an initial partial remainder result and a root opening intermediate result of the corresponding SRT16 root opening calculation; An SRT algorithm component (323), wherein the SRT algorithm component (323) is configured to perform the SRT16 division calculation and / or the SRT16 root extraction calculation according to the initial partial remainder result, the division intermediate result and / or the root extraction intermediate result, to obtain an SRT16 division result and / or an SRT16 root extraction result; A post-division processing component (324), wherein the post-division processing component (324) is configured to perform normalization processing on the SRT16 division result; A post-rooting processing component (325), wherein the post-rooting processing component (325) is configured to perform normalization processing on the SRT16 rooting result.

7. A configurable multifunctional vector computing device suitable for a neural network according to claim 6, characterized in that: The fast calculation unit (330) comprises: A transcendental function preprocessing component (331), wherein the transcendental function preprocessing component (331) is configured to perform transcendental function preprocessing on one of the data to be calculated according to the operation code, wherein the transcendental function preprocessing is used to obtain a table lookup index, a polynomial calculation input, and an intermediate calculation result of a corresponding transcendental function; A fast activation function preprocessing component (332), wherein the fast activation function preprocessing component (332) is configured to perform fast activation function preprocessing on one of the data to be calculated according to the operation code, wherein the fast activation function preprocessing is used to obtain a table lookup index, a polynomial calculation input, and an intermediate calculation result of a corresponding activation function; A fast calculation component (333), wherein the fast calculation component (333) is configured to use polynomial fitting to perform approximate calculations on the table lookup index corresponding to the transcendental function and / or the activation function, the polynomial calculation input, and the intermediate calculation result to obtain the activation function result and / or the transcendental function result; A transcendental function post-processing component (334), wherein the transcendental function post-processing component (334) is configured to perform normalization processing on the transcendental function result; A fast activation function post-processing component (335), wherein the fast activation function post-processing component (335) is configured to perform normalization processing on the activation function result.

8. A configurable multifunctional vector computing device suitable for a neural network according to claim 7, characterized in that: The storage control module (100) further comprises: a fourth register (180), the fourth register (180) being configured to temporarily store immediate data obtained by decoding the operation code; A fifth register (190), wherein the fifth register (190) includes a plurality of sub-registers, and the fifth register (190) is configured to store the target calculation result and the returned result.

9. A configurable multifunctional vector computing device suitable for a neural network according to claim 8, characterized in that: The computing device further comprises: A selector module (500) is configured to select output and input contents of the storage control module (100), the operation code storage module (200), the floating point calculation module (300) and the state machine module (400) according to the operation code.

10. A configurable multifunctional vector computing device suitable for a neural network according to claim 9, characterized in that: The selector module (500) comprises: A first selector (511), the first selector (511) being configured to select the intermediate result data via the first format conversion unit (110); a second selector (512), the second selector (512) being configured to select the residual data passed through the second format conversion unit (120); a third selector (513), the third selector (513) being configured to select the immediate data in the fourth register (180), select the return result in the fifth register (190), select the residual data in the second register (140) according to the operation code, and select the other parameter data in the third register (150) according to the operation code; a fourth selector (514), the fourth selector (514) being configured to select the target calculation result to be output in the fifth register (190) and the intermediate result data in the first register (130) according to the operation code; a fifth selector (515), the fifth selector (515) being configured to select the intermediate result data in the first register (130) according to the operation code, select the target calculation result according to the operation code, and send the target calculation result to the third format conversion unit (170); a sixth selector (516), the sixth selector (516) being configured to select the target calculation result via the third format conversion unit (170) according to the operation code, and send the target calculation result to the data calling unit (160); a seventh selector (521), the seventh selector (521) being configured to select the target calculation result output by the floating-point calculation module (300); an eighth selector (522), the eighth selector (522) being configured to select a sub-register in the fifth register (190) storing the target calculation result; a ninth selector (523), the ninth selector (523) being configured to select the return result to be output from the corresponding sub-register of the fifth register (190) according to the operation code; A tenth selector (524), the tenth selector (524) is configured to select the target calculation result to be output from the corresponding sub-register of the fifth register (190) according to the operation code.

Images