Hardware acceleration circuit, data processing acceleration method, chip and accelerator

Abstract

The application relates to a hardware acceleration circuit, a data processing acceleration method, a chip and an accelerator. The circuit comprises a storage module configured to store a first lookup table and a second lookup table; a lookup table circuit configured to output, based on the first lookup table, a plurality of exponential function values corresponding to a plurality of data elements in response to respective index values of the plurality of data elements; and output, based on the second lookup table, a reciprocal corresponding to an addition operation result in response to an index value of the addition operation result; an adder configured to output the addition operation result to the lookup table circuit, the addition operation result being a result of an addition operation on the plurality of exponential function values; and a multiplier configured to output a multiplication operation result of the exponential function value of an i-th data element in the plurality of data elements and the reciprocal of the addition operation result, so as to obtain a flexible maximum value of the i-th data element. The scheme provided by the application can improve the data processing speed in the Softmax function calculation process and obtain the Softmax function value more quickly.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a hardware acceleration circuit, a data processing acceleration method, a chip, and an accelerator. Background Technology

[0002] Nonlinear functions introduce nonlinear characteristics into artificial neural networks, playing a crucial role in their ability to learn and understand complex scenarios. Nonlinear functions include, but are not limited to, the softmax function and the sigmoid function.

[0003] The Softmax function, widely used in deep learning, is a prime example. In related technologies, its value can be calculated using general-purpose computing units such as CPUs or GPUs. However, when neural network processing is executed by hardware circuits such as Deep Learning Accelerators (DLAs) or Neural Network Processing Units (NPUs), if the Softmax function layer is located in the middle layer of the neural network, it leads to job migration overhead between the DLA / NPU and the CPU / GPU. This makes the scheme of using the CPU / GPU to determine the nonlinear function value inefficient, resulting in increased system bandwidth and higher power consumption. Summary of the Invention

[0004] To address or partially address the problems existing in related technologies, this application provides a hardware acceleration circuit, a data processing acceleration method, and an accelerator, which can improve the data processing speed during the Softmax function calculation process and accelerate the acquisition of the Softmax function value.

[0005] This application provides a hardware acceleration circuit, including:

[0006] The storage module is used to store the first lookup table and the second lookup table;

[0007] A lookup table circuit is configured to, in response to the index values ​​of multiple data elements in a dataset, output multiple exponential function values ​​corresponding to the multiple data elements based on a first lookup table; and, in response to the index value of the addition result, output the reciprocal corresponding to the addition result based on a second lookup table.

[0008] An adder is used to output the addition result to the lookup table circuit, wherein the addition result is obtained by adding the multiple exponential function values;

[0009] A multiplier is used to output the result of multiplying the exponential function value of the i-th data element with its reciprocal among the plurality of data elements, so as to obtain the flexible maximum value of the i-th data element.

[0010] In one embodiment, the exponential function value is data with a bit width of N1 bits, the addition result is data with a bit width of N2 bits, and the index value of the addition result is data with a bit width of N3 bits, wherein N1 and N3 are less than N2;

[0011] The hardware acceleration circuit further includes: a first conversion circuit, used to convert the addition result into the index value of the addition result based on the index value conversion parameter.

[0012] In one embodiment, the storage module includes a static storage module;

[0013] The index value conversion parameters are stored in the static storage module;

[0014] The index value conversion parameter is determined based on the Gaussian distribution data obtained by statistically analyzing the results of multiple addition operations on multiple sample datasets.

[0015] In one embodiment, the hardware acceleration circuit further includes:

[0016] An index value conversion parameter acquisition circuit is used to determine and output the index value conversion parameter based on the result of the addition operation;

[0017] The first conversion circuit is used to convert the addition result into the corresponding index value based on the index value conversion parameter;

[0018] The lookup table circuit outputs the reciprocal corresponding to the addition result based on the second lookup table. Specifically, it outputs the reciprocal corresponding to the addition result based on a selected second lookup table, wherein the second lookup table is the second lookup table corresponding to the index value conversion parameter among a plurality of candidate second lookup tables.

[0019] In one embodiment, the storage module includes a static storage module, in which the plurality of alternative second lookup tables are stored; or,

[0020] The storage module includes a dynamic storage module, and the selected second lookup table is stored in the dynamic storage module.

[0021] In one embodiment, the storage module includes a first storage area and a second storage area, wherein the first lookup table is stored in the first storage area and the second lookup table is stored in the second storage area;

[0022] The lookup table circuit includes:

[0023] The first basic lookup table circuit unit includes a first logic circuit, a first input terminal group, a first control terminal group, and a first output terminal group. The first input terminal group is connected to the first storage area. The first logic circuit is configured to: output the corresponding exponential function value from the first output terminal group in response to the index value of the i-th data element in the data set input from the first control terminal group.

[0024] The second basic lookup table circuit unit includes a second logic circuit, a second input terminal group, a second control terminal group, and a second output terminal group. The second input terminal group is connected to the second storage area. The second logic circuit is used to output the corresponding reciprocal from the second output terminal group in response to the index value of the addition operation result input from the second control terminal group.

[0025] Wherein: the first basic lookup table circuit unit is N0 input and N1 output, the second basic lookup table circuit unit is N3 input and N4 output, and the value range of N0, N1, N3, and N4 is [8, 32].

[0026] In one embodiment, the storage module includes a first storage area for time-sharing storage of the first lookup table and the second lookup table; or, the storage module includes a first storage area and a second storage area, wherein the first lookup table is stored in the first storage area and the second lookup table is stored in the second storage area.

[0027] The lookup table circuit includes:

[0028] A first basic lookup table circuit unit includes a first logic circuit, a first input group, a first control group, and a first output group. The first input group is connected to the storage module. The first logic circuit is configured to: respond to the index value of the i-th data element input from the first input group in a first time period, output the exponential function value corresponding to the i-th data element from the first output group based on the first lookup table; and, in a second time period after the first time period, respond to the index value of the addition result input from the first input group, output the reciprocal corresponding to the addition result from the first output group based on the second lookup table.

[0029] in:

[0030] The first basic lookup table circuit unit is in a state of N0 input and N1 output; or,

[0031] The first basic lookup table circuit unit further includes a state control terminal group, used to input a first state control signal in a first time period and a second state control signal in a second time period, so as to configure the basic lookup table circuit unit to: input N0 and output N1 in the first time period and input N3 and output N4 in the second time period; wherein at least one pair of N0 and N3, and N1 and N4 are not equal.

[0032] In one embodiment, the hardware acceleration circuit further includes:

[0033] The second conversion circuit is used to convert the multiplication result from a data with a width of N4 bits to a flexible maximum value with a width of N5 bits based on the index value conversion parameters, wherein N4 is greater than N5.

[0034] In one embodiment, the hardware acceleration circuit further includes:

[0035] A subtractor is used to output the result of subtraction between multiple initial data in the initial dataset and the maximum value among the multiple initial data, so as to obtain the data set containing the multiple data elements;

[0036] The third conversion unit is used to convert the plurality of data elements into a plurality of index values ​​corresponding to the first lookup table.

[0037] In one embodiment, the exponential function value, the addition result, the multiplication result, and the reciprocal of the addition result are fixed-point integers.

[0038] This application also provides an artificial intelligence chip, which includes the hardware acceleration circuit described above.

[0039] This application also provides a data processing acceleration method, the method comprising:

[0040] Based on the first lookup table, obtain multiple exponential function values ​​corresponding to multiple data elements in the dataset;

[0041] Obtain the result of the addition operation of the multiple exponential function values;

[0042] The reciprocal of the addition result is obtained based on the second lookup table;

[0043] The result of multiplying the exponential function value of the i-th data element among the plurality of data elements with the reciprocal of the addition operation result is obtained to obtain the flexible maximum value of the i-th data element.

[0044] In one embodiment, obtaining the reciprocal corresponding to the addition result based on the second lookup table includes:

[0045] Based on the index value conversion parameter, the addition result is converted from data with a width of N2 bits to an index value with a width of N3 bits, where N3 is less than N2;

[0046] Based on the second lookup table and the index value of the addition result, the reciprocal corresponding to the addition result is obtained.

[0047] In one embodiment, the method further includes:

[0048] The compiler writes the index value conversion parameters into the static storage module.

[0049] The index value conversion parameter is determined based on the Gaussian distribution data obtained by statistically analyzing the results of multiple addition operations on multiple sample datasets.

[0050] In one embodiment, the method further includes:

[0051] The index value conversion parameters are determined based on the result of the addition operation;

[0052] Obtaining the reciprocal corresponding to the addition result based on the second lookup table and the index value of the addition result includes:

[0053] From a plurality of alternative second lookup tables, a selected second lookup table corresponding to the index value conversion parameter is determined;

[0054] Based on the selected second lookup table and the index value of the addition result, the reciprocal corresponding to the addition result is obtained.

[0055] In one embodiment, the method further includes:

[0056] The compiler writes the plurality of alternative second lookup tables into the static storage module; and / or,

[0057] The selected second lookup table is loaded into the dynamic storage module.

[0058] In one embodiment, before obtaining the multiple exponential function values ​​corresponding to multiple data elements in the data set based on the first lookup table, the method further includes: subtracting the maximum value among multiple initial data in the initial dataset to obtain the data set including the multiple data elements; and / or,

[0059] After obtaining the result of multiplying the exponential function value of the i-th data element among the plurality of data elements with the reciprocal, the method further includes: based on the index value conversion parameter, converting the multiplication result from data with a bit width of N4 bits to data with a bit width of N5 bits, wherein N4 is greater than N5.

[0060] In one embodiment, the exponential function value, the addition result, the multiplication result, and the reciprocal of the addition result are fixed-point integers;

[0061] The bit width range of the exponential function value and the reciprocal is [8, 32].

[0062] In one embodiment, the method is used to implement a flexible maximum function layer of a neural network, the neural network being used to classify data to be processed; wherein,

[0063] The data to be processed includes at least one of voice data, text data, and image data.

[0064] This application further provides an artificial intelligence accelerator, comprising:

[0065] Processor; and

[0066] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0067] The technical solution provided in this application may include the following beneficial effects:

[0068] The technical solution of this application embodiment obtains the exponential function value of each data element and the reciprocal of the addition operation result of the exponential function value of each data element by looking up a table. This avoids complex exponential and reciprocal operations, improves the data processing speed in the Softmax function calculation process, and obtains the Softmax function value faster.

[0069] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0070] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0071] Figure 1 This is a schematic diagram of the structure of a neural network shown in one embodiment of this application;

[0072] Figure 2 This is a schematic diagram of the structure of a neural network for classification shown in one embodiment of this application;

[0073] Figure 3 This is a structural block diagram of a hardware acceleration circuit according to an embodiment of this application;

[0074] Figure 4This is a schematic diagram of the basic lookup table circuit unit according to an embodiment of this application;

[0075] Figure 5 This is a structural block diagram of a hardware acceleration circuit according to another embodiment of this application;

[0076] Figure 6 This is a structural block diagram of a hardware acceleration circuit according to another embodiment of this application;

[0077] Figure 7 This is a structural block diagram of a hardware acceleration circuit according to another embodiment of this application;

[0078] Figure 8 This is a structural block diagram of a hardware acceleration circuit according to another embodiment of this application;

[0079] Figure 9 This is a schematic flowchart of a data processing acceleration method according to an embodiment of this application;

[0080] Figure 10 This is a schematic flowchart of a data processing acceleration method according to another embodiment of this application;

[0081] Figure 11 This is a schematic flowchart of a data processing acceleration method according to another embodiment of this application;

[0082] Figure 12 This is a structural block diagram of an artificial intelligence accelerator according to an embodiment of this application. Detailed Implementation

[0083] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0084] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0085] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0086] The computation of nonlinear functions may involve operations on exponential functions and / or their reciprocals. For example, the computation of the Softmax function may involve operations on the exponent (exp) and the reciprocal of the sum of exponents (1 / sum_of_exp). Dedicated hardware pipelines for the Softmax function are not feasible for large-scale computational needs. For instance, increasing computational power would lead to expensive hardware costs.

[0087] To address the aforementioned issues, this application provides a data processing acceleration solution that obtains the exponential function value of each data element and the reciprocal of the sum of the exponential function values ​​of each data element by looking up a table. This avoids complex exponential and reciprocal operations and improves the processing speed of the Softmax function.

[0088] Figure 1 This is a schematic diagram of the structure of a neural network shown in one embodiment of this application.

[0089] See Figure 1 The diagram illustrates the topology of a neural network 100, including an input layer, hidden layers, and an output layer. This neural network 100 is capable of performing calculations or operations based on data elements I1 and I2 received from the input layer, and generating output data O1 and O2 based on the results of the calculations.

[0090] For example, neural network 100 can be a deep neural network (DNN) that includes one or more hidden layers. Figure 1 The neural network 100 in the diagram includes an input layer L1, two hidden layers L2 and L3, and an output layer L4. DNNs include, but are not limited to, convolutional neural networks (CNNs) and recurrent neural networks (RNNs).

[0091] It should be noted that, Figure 1The four layers shown are for illustrative purposes only and should not be construed as limiting the scope of this application. For example, a neural network may include more or fewer hidden layers.

[0092] Nodes in different layers of the neural network 100 can be connected to each other for data transmission. For example, a node can receive data from other nodes to perform calculations on the received data and output the calculation results to nodes in other layers.

[0093] Each node can determine its output data based on the output data and weights received from nodes in previous layers. For example, Figure 1 middle This represents the weight between the first node of the first layer and the first node of the second layer. This represents the output data of the first node in the first layer. Let represent the bias value of the first node in the second layer. Then, the output data of the first node in the second layer can be represented as: The calculation method for the output data of other nodes is similar, and will not be described in detail here.

[0094] In some embodiments, the neural network is configured with activation function layers, such as softmax function layers, which can convert the result values ​​for each class into probability values.

[0095] In some embodiments, the neural network is configured with a loss function layer after the flexible maximum function layer, which is capable of calculating the loss as an objective function for training or learning.

[0096] Understandably, a neural network can respond to the data to be processed, and after processing the data, obtain a recognition result; the data to be processed may include at least one of voice data, text data, and image data.

[0097] A typical type of neural network is a classification neural network. Classification neural networks determine the category to which a data element belongs by calculating the probability of that element corresponding to each class.

[0098] Figure 2 This is a schematic diagram of the structure of a neural network for classification, as shown in one embodiment of this application.

[0099] See Figure 2 The neural network 200 used for classification in this embodiment may include a hidden layer 210, a fully connected layer (FC layer) 220, a flexible maximum function layer 230, and a loss function layer 240.

[0100] like Figure 2As shown, the neural network 200 responds to the data to be classified by sequentially calculating the hidden layer 210 and the fully connected (FC) layer 220. The FC layer 220 outputs a calculation result s, which corresponds to the classification probability of the data element. The FC layer 220 may include multiple nodes corresponding to multiple classes, each node outputting a result value corresponding to the probability that the data element is classified into the corresponding class. For example, see also... Figure 1 FC layer 220 corresponds to Figure 1 The output layer L4 has two nodes, corresponding to two categories (Class 1 and Class 2). The output value of one node can be a result value representing the probability that a data element is classified into Class 1, and the output value of the other node can be a result value representing the probability that a data element is classified into Class 2. The FC layer 220 outputs the calculation result s to the flexible maximal function layer 230, which converts the calculation result s into a probability value y and can also normalize the probability value y.

[0101] The flexible maximum function layer 230 outputs the probability value y to the loss function layer 240, which can calculate the cross-entropy loss L of the result s based on the probability value y.

[0102] During the backpropagation learning process, the flexible maximum function layer 230 calculates the gradient of the cross-entropy loss L. Then, the FC layer 220 performs gradient learning based on the cross-entropy loss L. For example, the weights of the FC layer 220 can be updated according to the gradient descent algorithm. Further, subsequent learning processes can be performed in the hidden layer 210.

[0103] The neural network 200 can be implemented in software, in hardware circuits, or a combination of both. For example, in the case of hardware circuit implementation, the hidden layer 210, the fully connected (FC) layer 220, the flexible maximum function layer 230, and the loss function layer 240 are all implemented in hardware circuits, which can be integrated into a single AI chip or distributed across multiple chips. This configuration avoids the data migration between other layers of the neural network and processors such as CPUs / GPUs when the flexible maximum function layer 230 is implemented in a CPU / GPU, thereby improving the efficiency of neural network data processing, reducing data processing latency and power consumption, and avoiding increased bandwidth usage.

[0104] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0105] Figure 3This is a structural block diagram of a hardware acceleration circuit according to an embodiment of this application. In this application, the hardware acceleration circuit can be used, for example, but not limited to, to implement the flexible maximum function layer 230 in the neural network 200 described above. The hardware acceleration circuit can be, for example, but not limited to, a circuit component in a CPLD (Complex Programming logic device) chip, an FPGA (Field Programmable Gate Array) chip, a dedicated chip, etc.

[0106] To facilitate understanding of this application, the Softmax function, which maximizes flexibility, is explained below. Assume we have an array X, then the i-th element x... i The formula for calculating the Softmax function value is shown in equation (1).

[0107]

[0108] In equation (1), σ(x) i Represents the i-th element x i The value of the Softmax function, where e is the natural constant, x i Let x represent the i-th element of array X. max This represents the maximum element in array X. This represents the result of adding the exponential function values ​​of at least some elements in array X.

[0109] See Figure 3 A hardware acceleration circuit 300 includes a storage module 10, a lookup table circuit 11, an adder 12, and a multiplier 13.

[0110] Storage module 10 is used to store the first lookup table and the second lookup table. Storage module 10 may be, for example, RAM (Random-Access Memory), ROM (Read-Only Memory), FLASH, etc.

[0111] The lookup table (LUT) circuit 11 is used to output multiple exponential function values ​​corresponding to the multiple data elements based on a first lookup table in response to the index values ​​of the multiple data elements in the data set, and to output the reciprocal corresponding to the addition result based on a second lookup table in response to the index value of the addition result.

[0112] Adder 12 is used to output the addition result to lookup table circuit 11. The addition result is obtained by adding multiple exponential function values.

[0113] Multiplier 13 is used to output the result of multiplying the exponential function value of the i-th data element among multiple data elements with the reciprocal of the addition result, so as to obtain the flexible maximum value of the i-th data element.

[0114] In some embodiments, the lookup table circuit 11 includes at least one basic lookup table circuit unit 20.

[0115] See Figure 4 As shown, in a specific implementation, the basic lookup table circuit unit 20 includes a logic circuit 21, an input terminal group 22, a control terminal group 23, and an output terminal group 24. The input terminal group 22 is connected to the storage module 10, inputting the lookup table data into the logic circuit 21. The logic circuit 21 selects the value corresponding to the index value (also called the address) input from the control terminal group 23 and outputs it from the output terminal group 24. The logic circuit 21 can be, for example, a logic gate circuit or a logic switch circuit. It is understood that in this application, a terminal group refers to a set of connection terminals, including one or more connection terminals. If the control terminal group 23 has A control terminals and the output terminal group 24 has B output terminals, the basic lookup table circuit unit 20 is referred to as A-input B-output.

[0116] It is understandable that the result of adding exponential function values ​​can be obtained by directly adding the exponential function values, or by performing a specific transformation on the exponential function values ​​before adding them. In the case of a specific transformation, the subsequent data processing results may be subjected to an inverse transformation or no inverse transformation may be performed, depending on the type of transformation. Similarly, the processing of other data should be broadly understood to include both of the above situations, and should not be limited to processing only the data itself. Other embodiments are similar and will not be described further below.

[0117] In this embodiment, a hardware lookup table circuit is used to obtain the exponential function value of each data element and the reciprocal of the addition result of the exponential function value. This avoids complex exponential and reciprocal operations, improves the data processing speed in the Softmax function calculation process, and allows for faster acquisition of the Softmax function value. Furthermore, it avoids the excessive hardware circuit area and high cost associated with implementing exponential and reciprocal operations.

[0118] Figure 5 This is a structural block diagram of a hardware acceleration circuit according to another embodiment of this application.

[0119] See Figure 5 A hardware acceleration circuit 500 includes a storage module 10, a lookup table circuit 11, an adder 12, a multiplier 13, an index value conversion parameter acquisition circuit 14, a first conversion circuit 15, and a second conversion circuit 16.

[0120] Storage module 10 is used to store the first lookup table and the second lookup table.

[0121] The lookup table circuit 11 is used to output multiple exponential function values ​​corresponding to the multiple data elements based on the first lookup table, in response to the respective index values ​​of multiple data elements in the data set. The index value of the data element is data with a bit width of N0 bits.

[0122] In one embodiment, the index values ​​of multiple data elements are sequentially input into a lookup table circuit 11, and the lookup table circuit 11 sequentially outputs the exponential function values ​​corresponding to each data element in a first lookup table. Each exponential function value in the first lookup table is data with a bit width of N1 bits.

[0123] Adder 12 is used to output the addition result to lookup table circuit 11. The addition result is obtained by adding multiple exponential function values.

[0124] In one embodiment, adder 12 accumulates the exponential function values ​​of each data element and outputs the addition result with a bit width of N2 bits.

[0125] The index value conversion parameter acquisition circuit 14 and the first conversion circuit 15 are used to obtain the index value corresponding to the addition result output by the adder 12.

[0126] The index value conversion parameter acquisition circuit 14 is used to determine and output the index value conversion parameter based on the addition operation result.

[0127] The first conversion circuit 15 is used to convert the result of the addition operation into the corresponding index value based on the index value conversion parameter. The index value output by the first conversion circuit 22 is data with a bit width of N3 bits.

[0128] In response to the index value of the addition result, the lookup table circuit 11 outputs the reciprocal corresponding to the index value of the addition result, based on a selected second lookup table. The selected second lookup table is a subset of candidate second lookup tables corresponding to the conversion parameter of the index value. Each reciprocal stored in the second lookup table is N4-bit data; that is, the reciprocal of the addition result output by the lookup table circuit 11 is N4-bit data.

[0129] Multiplier 13 is used to output the result of multiplying the exponential function value of the i-th data element among multiple data elements with the reciprocal of the addition result. The result of this multiplication operation output by multiplier 13 is a data with a width of N5 bits.

[0130] The second conversion circuit 16 is used to convert the multiplication result output by the multiplier 13 into data with a bit width of N6 bits based on the index value conversion parameter, thereby outputting the flexible maximum value of the i-th data element.

[0131] In one specific implementation, the index value of the data element is an 8-bit fixed-point integer. Each exponential function value in the first lookup table is an 8-bit fixed-point integer, and the result of adding multiple exponential function values ​​is a 32-bit fixed-point integer. The index value and reciprocal of the addition result are both 8-bit fixed-point integers. That is, the second lookup table has 8 inputs and 8 outputs. The result of multiplication is a 16-bit fixed-point integer, and the result after conversion is an 8-bit fixed-point integer. That is, N0, N1, N3, N4, and N6 are 8, N2 is 32, and N5 is 16.

[0132] Understandably, in some implementations, N0 to N6 can be other values; for example, the range of N0, N1, N3, and N4 can be [8, 32], and in some specific instances, the range can be [8, 12]. For example, N0, N1, N3, N4, and N6 can also be unequal; for example, N0 and N3 can be 9, 10, 11, or 12, while N1 and N4 are 8. Because the dynamic range of the Softmax function is very wide, related technologies often use software modules to implement this function. This application provides a hardware circuit solution that is essentially based on 8 bits and can effectively balance important indicators such as circuit cost, power consumption, bandwidth, performance, and data accuracy.

[0133] In this embodiment, during the process of obtaining the reciprocal of the addition result of the exponential function values ​​of each data element through a table lookup, an index value conversion parameter is determined based on the addition result. Based on this parameter, the addition result is converted into a corresponding index value, and a second lookup table is selected from multiple candidate tables. Then, based on the index value of the addition result and the selected second lookup table, the reciprocal corresponding to that index value is output. Since the index value conversion parameter is determined in real-time based on the addition result of each table lookup, the reliability of the obtained lookup result is guaranteed.

[0134] Furthermore, by setting the calculation process of the softmax function to integer data processing and configuring the input / output data bit width of the two lookup tables within a small range, the storage resources occupied by the first and second lookup tables and the area of ​​the lookup table circuit can be reduced, as can the bandwidth occupied. On the other hand, the lookup speed and fixed-point operation speed can be improved within the allowable accuracy range, thereby further accelerating the circuit response speed and reducing power consumption.

[0135] In one embodiment, the index value conversion parameter includes an index value truncation parameter. The first conversion circuit 15 extracts the index value of the addition operation result from the corresponding position in the addition operation result based on the index value truncation parameter.

[0136] In one embodiment, the index value conversion parameter acquisition circuit 14 includes a leading zero count (LZC) circuit. The leading zero count circuit outputs the number of leading zeros in the addition result to the first conversion circuit 22; the number of leading zeros is the number of zeros that appear from the highest bit of the binary data up to the first 1.

[0137] In another embodiment, the index value conversion parameter acquisition circuit 14 includes a leading 1 detection circuit, which is used to output the position data of the leading 1 in the addition operation result to the first conversion circuit 22. The leading 1 is the first 1 scanned from the highest bit of the binary data. The number of leading 0s or the position data of the leading 1 can be used as the index value truncation parameter.

[0138] In one embodiment, the first conversion circuit 15 may include a first shifter. In a specific implementation, the first shifter uses the number of leading 0s as the shift number, shifts the addition result left by this shift number, and outputs shifted data with a width of N3 bits. That is, it extracts N3 consecutive bits from the addition result, starting from the leading 1 and moving towards the lower bits, as the index value of the addition result. It is understood that the specific configuration of the first conversion circuit can be determined according to the specific data structure of the index value.

[0139] The second conversion circuit 16 is used to convert the result of the multiplication operation from data with a width of N5 bits to data with a width of N6 bits. In a specific implementation, the second conversion circuit 16 includes a second shifter. Understandably, depending on actual needs, the second conversion circuit 16 can perform saturation, rounding, and other processing to ensure that the data conversion result of the second conversion circuit 16 corresponds to the data conversion result of the first conversion circuit 22. Rounding includes, for example, rounding to the nearest integer, rounding up, rounding down, and rounding towards zero.

[0140] In one embodiment, the storage module 10 includes a static storage module. In one specific implementation, the static storage module is a ROM, and multiple alternative second lookup tables are written into the static storage module by the compiler; in another specific implementation, the static storage module is an SRAM, and multiple alternative second lookup tables are loaded into the SRAM after power-on; after the index value conversion parameter acquisition circuit 14 outputs the index value conversion parameter, the lookup table circuit 11 outputs the reciprocal corresponding to the index value of the addition operation result based on the selected second lookup table.

[0141] Multiple alternative second lookup tables correspond to different index value conversion parameters. Taking the number of leading zeros as the index value conversion parameter as an example, for a 32-bit addition result, the minimum possible value of the number of leading zeros is 0 (i.e., the highest bit of the addition result is 1), and the maximum possible value is 31 (i.e., the lowest bit of the addition result is 1, and all other bits before it are 0). Therefore, the number of leading zeros can be any integer value within the range [0, 31], for a total of 32 possibilities. Different index value conversion parameters represent different value ranges of the addition result, and thus the value range of the reciprocal of the addition result is also different. Therefore, corresponding to the 32 possible index value conversion parameters, the number of alternative second lookup tables is also 32. The corresponding second lookup table can be selected according to the specific value of the number of leading zeros.

[0142] In another embodiment, the storage module 10 includes a dynamic storage module, such as DRAM, for storing selected second lookup tables corresponding to the selective writing of index value conversion parameters. Multiple alternative second lookup tables can be stored in other memory, for example, they can be written into ROM by the compiler. After the index value conversion parameter acquisition circuit 14 outputs the index value conversion parameters, the selected second lookup table is loaded into the dynamic storage module connected to the lookup table circuit 11. The lookup table circuit 11 outputs the reciprocal corresponding to the addition result based on the selected second lookup table stored in the dynamic storage module.

[0143] Figure 6 This is a structural block diagram of a hardware acceleration circuit according to another embodiment of this application.

[0144] See Figure 6 A hardware acceleration circuit 600 includes a storage module 10, a lookup table circuit 11, an adder 12, a multiplier 13, a first conversion circuit 15, a subtractor 17, and a third conversion unit 18.

[0145] Subtractor 17 is used to output the subtraction results of multiple initial data in the initial dataset with the maximum value among the multiple initial data, so as to obtain a data set containing multiple data elements.

[0146] By performing the above subtraction operation, the value range of the data element can be reduced, thereby facilitating the implementation of the present application scheme with less bit width data and corresponding hardware circuitry. On the other hand, since each data element in the data set takes the value of a negative number or 0, the exponential function value of the data element with base e can be normalized to the range of (0, 1].

[0147] The third conversion unit 18 is used to convert each data element in the data set into the index value of the first lookup table.

[0148] To better understand the lookup process in this embodiment, Table 1 below shows a specific example of the first lookup table, which has N0 inputs and N1 outputs, where both N0 and N1 are 8 bits. The data elements of the first lookup table are index values ​​with a bit width of N0 bits, and the output data are exponential function values ​​with a bit width of N1 bits. For ease of understanding, the data in Table 1 is represented in decimal format. It can be understood that the first lookup table in the storage module 10 only stores the true value of the exponential function. The lookup table circuit is used to implement the mapping relationship between the index value and the true value of the exponential function. The data elements and the normalized exponential function value are included in the table together for better understanding of this application.

[0149] Table 1

[0150]

[0151] As shown in Table 1, the data elements output by subtractor 17 are negative numbers or 0, and the value range of the data elements is defined as [-10, 0]. For table lookup, the value range [-10, 0] is discretized into 256 (i.e., 2^356) as shown in the column "Data Elements". N0 There are 10 data points, and the corresponding exponential function value for each point is shown in the column "Normalized Exponential Function Value". Each data element point corresponds to an integer value in the range [0, 255] shown in the column "Index Value". Each normalized exponential function value corresponds to an integer value in the range [0, 255] shown in the column "Exponential Function Value". The data in the column "Exponential Function Value" is stored as true values ​​in the storage module 10. The table can be looked up by the index value.

[0152] The storage module 10 may include a static storage module. The first lookup table and the second lookup table are stored in different storage units of the static storage module. The static storage module also stores index value conversion parameters. The first lookup table, the second lookup table, and the index value conversion parameters may be written into the static storage module by the compiler, for example.

[0153] In one embodiment, the index value conversion parameters can be determined offline and then written into the static storage module by the compiler. The first conversion circuit 15 can directly obtain the index value of the addition operation result based on the index value conversion parameters already written into the static storage module. The index value conversion parameters can be determined by statistically analyzing the Gaussian distribution data of multiple addition operation results from multiple sample data sets. In a specific implementation, multiple sample data sets can be obtained; for each sample data set, multiple exponential function values ​​corresponding to its multiple sample data elements are obtained through the lookup table circuit 11, and the addition operation result of the multiple exponential function values ​​is obtained through the adder 12. The addition operation result is data with a bit width of N2 bits; then, the Gaussian distribution of the multiple addition operation results from the multiple sample data sets is statistically analyzed, and the N3 bits with the most frequent numerical distribution among the multiple addition operation results are determined based on the Gaussian distribution data. The position data corresponding to this N3 bits (e.g., the start and / or end bits of the N3 bits) is used as the index value truncation parameter. The first conversion circuit 15 can extract N3 consecutive bits of data from the addition result output by the adder according to the index value truncation parameter pre-written into the static storage module (for example, if the 32-bit addition result is 00000000_00000000_00000001_11000001, and the index value truncation parameter is [23, 30], then the extracted data is 8 bits from the 23rd to the 30th bit in the direction from the high bit to the low bit: 11100000). The extracted data is used as the index value of the addition result.

[0154] The lookup table circuit 11 is used to output multiple exponential function values ​​corresponding to the multiple data elements based on the first lookup table, in response to the index values ​​of the multiple data elements in the data set.

[0155] Adder 12 is used to output the addition result to lookup table circuit 11. The addition result is obtained by adding multiple exponential function values.

[0156] The first conversion circuit 15 is used to convert the result of the addition operation into an index value with a width of N3 bits, i.e., the index value of the second lookup table, based on the index value conversion parameters stored in the storage module 10.

[0157] The lookup table circuit 11 is also used to output the reciprocal of the addition result based on the second lookup table in response to the N3-bit index value of the addition result.

[0158] The second lookup table corresponds to the index value conversion parameter. After determining the index value conversion parameter through the offline method described above, the corresponding second lookup table can be determined. The second lookup table can then be written into the storage module 10 by the compiler or loaded into the storage module 10 after power-on.

[0159] Multiplier 13 is used to output the result of multiplying the exponential function value of the i-th data element with the reciprocal of the addition operation result to obtain the flexible maximum value of the i-th data element.

[0160] In this embodiment, index value conversion parameters are predetermined offline. Based on these parameters, the addition result is converted into a corresponding index value. Then, based on the index value of the addition result and a second lookup table, the reciprocal corresponding to that index value is output. Since the index value conversion parameters are predetermined, the process of determining the index value conversion parameters and selecting the second lookup table from multiple candidate tables is avoided. Furthermore, storing all candidate second lookup tables is unnecessary. Therefore, the amount of data processing is reduced, the circuit response speed is improved, and the required hardware resources and power consumption are lowered.

[0161] Figure 7 This is a structural block diagram of a hardware acceleration circuit according to another embodiment of this application.

[0162] See Figure 7 A hardware acceleration circuit 700 includes a storage module 10, a lookup table circuit 11, an adder 12, a multiplier 13, and a first conversion circuit 15.

[0163] Storage module 10 includes a first storage area 10A and a second storage area 10B. A first lookup table is stored in the first storage area 10A and a second lookup table is stored in the second storage area 10B.

[0164] The lookup table circuit 11 includes a first basic lookup table circuit unit 117 and a second basic lookup table circuit unit 118.

[0165] The first basic lookup table circuit unit 117 includes a first input terminal group 1171, a first control terminal group 1172, a first output terminal group 1173, and a first logic circuit 1174. The first input terminal group 1171 is connected to the first storage area 10A. The first logic circuit 1174 is used to output the corresponding exponential function value stored in the first storage area 10A from the first output terminal group 1173 in response to the index value of the data element input from the first control terminal group 1172.

[0166] The second basic lookup table circuit unit 118 includes a second input terminal group 1181, a second control terminal group 1182, a second output terminal group 1183, and a second logic circuit 1184. The second input terminal group 1181 is connected to the second storage area 10B. The second logic circuit 1184 is used to: respond to the index value of the addition operation result input from the second control terminal group 1182, output the corresponding reciprocal stored in the second storage area 10B from the second output terminal group 1183. The first basic lookup table circuit unit is N0 input and N1 output, and the second basic lookup table circuit unit is N3 input and N4 output. The value range of N0 to N4 is [8, 32], and in some specific instances, the value range can be [8, 12].

[0167] In one embodiment, the first control terminal group 1172 sequentially inputs the index values ​​of multiple data elements in the data set into the first logic circuit 1174; the first logic circuit 1174 responds to the index values ​​and outputs the corresponding exponential function value from the first output terminal group 1173.

[0168] Adder 12 performs addition operations on multiple exponential function values ​​corresponding to multiple data elements output by the first output terminal group 1173 to obtain the addition result of multiple exponential function values.

[0169] The first conversion circuit 15 is used to convert the result of the addition operation into an index value with a bit width corresponding to the result.

[0170] The second control terminal group 1182 inputs the index value of the addition result into the second basic lookup table circuit unit 118; the second logic circuit 1184 responds to the index value of the addition result input from the second control terminal group 1182 and outputs the corresponding reciprocal from the second output terminal group 1183.

[0171] Multiplier 13 performs a multiplication operation on the exponential function value corresponding to the i-th data element output by the first output group 1173 and the reciprocal corresponding to the addition operation result output by the second output group 1183 to obtain the multiplication operation result, which is the flexible maximum value corresponding to the i-th data element.

[0172] Figure 8 This is a structural block diagram of a hardware acceleration circuit according to another embodiment of this application.

[0173] See Figure 8 A hardware acceleration circuit 800 includes a storage module 10, a lookup table circuit 11, an adder 12, a multiplier 13, and a first conversion circuit 15.

[0174] This embodiment and Figure 5 The hardware acceleration circuit shown is similar to 400, but the difference is:

[0175] The lookup table circuit 11 includes a first basic lookup table circuit unit 117.

[0176] The first basic lookup table circuit unit 117 includes a first input terminal group 1171, a first control terminal group 1172, a first output terminal group 1173, and a first logic circuit 1174. The first input terminal group 1171 is connected to the storage module 10. The first logic circuit is used to: respond to the index value of the i-th data element input from the first input terminal group 1171 in a first time period, output the exponential function value corresponding to the i-th data element from the first output terminal group 1173 based on the first lookup table; and in a second time period after the first time period, respond to the index value of the addition operation result input from the first input terminal group 1171, output the reciprocal corresponding to the addition operation result from the first output terminal group 1173 based on the second lookup table.

[0177] In one specific implementation, the storage module 10 includes a first storage area, in which a first lookup table and a second lookup table are stored in a time-sharing manner. Since only one storage area needs to be configured for time-sharing storage of either the first or second lookup table, the storage space occupied by the lookup tables is effectively reduced, thus reducing hardware costs.

[0178] In another specific implementation, the storage module 10 includes a first storage area and a second storage area, with a first lookup table stored in the first storage area and a second lookup table stored in the second storage area.

[0179] In one specific implementation, N3 is equal to N0, and N4 is equal to N1, meaning both the first and second lookup tables are N0 inputs and N1 outputs. Correspondingly, the first basic lookup table circuit unit 117 is fixed in an N0 input and N1 output state. In another specific implementation, the first lookup table is N0 input and N1 output, and the second lookup table is N3 input and N4 output, where N3 is not equal to N0, and / or N4 is not equal to N1. That is, at least one pair of N0 and N3, and N1 and N4, are not equal. The first basic lookup table circuit unit 117 also includes a state control terminal group 1175, used to input a first state control signal in a first time period and a second state control signal in a second time period, to configure the first basic lookup table circuit unit 117 as follows: in the N0 input and N1 output state during the first time period, and in the N2 input and N3 output state during the second time period.

[0180] Understandably, this embodiment also includes a first selector 30 and a second selector 32. The first selector 30 is used to output the exponential function value corresponding to the data element output by the first output group 1173 to the adder 12, and to output the reciprocal corresponding to the addition result output by the first output group 1173 to the second selector 32. The second selector 32 is used to selectively input the index value of the data element or the index of the addition result output by the first conversion circuit 15 into the first logic circuit 1174.

[0181] In this embodiment, by reusing the basic lookup table circuit unit, only one basic lookup table circuit unit needs to be configured, which can effectively reduce the area and cost of the lookup table circuit.

[0182] This application also provides embodiments of data processing acceleration methods.

[0183] Figure 9 This is a schematic flowchart illustrating a data processing acceleration method according to an embodiment of this application.

[0184] See Figure 9 A data processing acceleration method includes:

[0185] In step S910, multiple exponential function values ​​corresponding to multiple data elements in the data set are obtained based on the first lookup table.

[0186] In step S920, the result of the addition operation of multiple exponential function values ​​is obtained.

[0187] In step S930, the reciprocal corresponding to the addition result is obtained based on the second lookup table.

[0188] In step S940, the result of multiplying the exponential function value of the i-th data element among multiple data elements with the reciprocal of the addition result is obtained to obtain the flexible maximum value of the i-th data element.

[0189] Figure 10 This is a schematic flowchart of a data processing acceleration method according to another embodiment of this application.

[0190] See Figure 10 The data processing acceleration method in this embodiment includes:

[0191] In step S1010, multiple initial data are subtracted from the maximum value to obtain a data set.

[0192] A subtractor can be used to obtain the maximum value of multiple initial data in the initial dataset, and then the multiple initial data can be subtracted from the maximum value to obtain a dataset containing multiple data elements.

[0193] Through the above subtraction operation, the value of each data element in the data set is either negative or 0, thereby normalizing the exponential function value of the data element to the base e to the range of (0, 1].

[0194] In step S1020, each data element in the data set is converted into an index value of the first lookup table.

[0195] The third transformation unit can be used to convert each data element in the dataset into an index value of the first lookup table.

[0196] Through conversion, data elements can be transformed from negative numbers or 0 into index values ​​of the first lookup table. The index value is a fixed-point integer with a bit width of N0 bits.

[0197] In step S1030, based on the first lookup table and index value, multiple exponential function values ​​corresponding to multiple data elements are obtained.

[0198] The lookup table module can be used to obtain multiple exponential function values ​​corresponding to multiple data elements in the dataset, based on the first lookup table and the index value.

[0199] Understandably, the lookup process for multiple data elements can be a parallel process, meaning the lookup module is a multi-input multi-output module, where the index values ​​of multiple data elements are input into the lookup module in parallel, and the lookup module outputs the corresponding multiple exponential function values ​​in parallel. Alternatively, the lookup process for multiple data elements can also be a serial process, where the index values ​​of multiple data elements are input into the lookup module sequentially, and the lookup module outputs the exponential function value of each data element sequentially.

[0200] By looking up a table, the exponential function value corresponding to the data element can be obtained. The exponential function value can be a fixed-point integer with a bit width of N1 bits.

[0201] In step S1040, the result of the addition operation of multiple exponential function values ​​is obtained.

[0202] An adder can be used to obtain the result of adding multiple exponential function values. The result of the addition operation is a fixed-point integer with a bit width of N2 bits.

[0203] In step S1050, the addition result is converted into an index value based on the index value conversion parameter.

[0204] The addition result can be converted from N2-bit data to an N3-bit index value, which is the index value of the second lookup table, by the first conversion circuit based on the preset index value conversion parameters, where N3 is less than N2.

[0205] In this embodiment, the index value conversion parameters can be determined offline.

[0206] In a specific implementation, multiple sample data sets can be obtained. For each sample data set, multiple exponential function values ​​corresponding to its multiple sample data elements are obtained through a lookup table circuit. The addition result of the multiple exponential function values ​​is obtained through an adder. The addition result is data with a bit width of N2 bits. Then, the Gaussian distribution of the multiple addition results of the multiple sample data sets is statistically analyzed. Based on the Gaussian distribution data, the N3 bits with the most values ​​in the multiple addition results are determined. The position data corresponding to the N3 bits (e.g., the start and / or end bits of the N3 bits) is used as the index truncation parameter. The first conversion circuit 15 can extract N3 consecutive bits of data from the addition result output by the adder according to the index value truncation parameter pre-written into the static storage module (for example, if the 32-bit addition result is 00000000_00000000_00000001_11000001, and the index value truncation parameter is [23, 30], then the extracted data is 8 bits from the 23rd to the 30th bit in the direction from the high bit to the low bit: 11100000). The extracted data is used as the index value of the addition result.

[0207] The second lookup table corresponds to the index value conversion parameter. After determining the index value conversion parameter through the offline method described above, the corresponding second lookup table can be determined.

[0208] The second lookup table can be written to ROM by the compiler, or it can be loaded into RAM after the circuit is powered on.

[0209] In step S1060, the reciprocal corresponding to the addition result is obtained based on the second lookup table and the index value of the addition result.

[0210] The reciprocal of the addition result can be obtained using the lookup table module, based on the index value of the second lookup table and the addition result.

[0211] By looking up a table, the reciprocal of the result of adding multiple exponential function values ​​can be obtained. The reciprocal can be a fixed-point integer with a bit width of N4 bits.

[0212] In step S1070, the result of the multiplication of the exponential function value of the i-th data element among multiple data elements and the reciprocal of the addition result is obtained.

[0213] A multiplier can be used to obtain the result of multiplying the exponential function value of the i-th data element among multiple data elements with the reciprocal of the addition result. The multiplication result can be a fixed-point integer with a bit width of N5 bits.

[0214] In some embodiments, after obtaining the multiplication result, the second conversion circuit converts the multiplication result from data with a bit width of N5 bits to data with a bit width of N6 bits based on the index value conversion parameter, where N5 is greater than N6.

[0215] Figure 11 This is a schematic flowchart of a data processing acceleration method according to another embodiment of this application.

[0216] See Figure 11 The data processing acceleration method in this embodiment includes:

[0217] In step S1110, multiple initial data are subtracted from the maximum value to obtain a data set.

[0218] A subtractor can be used to obtain the maximum value of multiple initial data in the initial dataset, and then the multiple initial data can be subtracted from the maximum value to obtain a dataset containing multiple data elements.

[0219] In step S1120, each data element in the data set is converted into an index value of the first lookup table.

[0220] The third transformation unit can be used to convert each data element in the dataset into an index value of the first lookup table.

[0221] In step S1130, based on the first lookup table and index value, multiple exponential function values ​​corresponding to multiple data elements are obtained.

[0222] The lookup table module can be used to obtain multiple exponential function values ​​corresponding to multiple data elements in the dataset, based on the first lookup table and the index value.

[0223] In this embodiment, the base-e exponential function value of the data element is obtained by looking up a table. It can be understood that the table lookup process for the data element can be a parallel process, i.e., the lookup table module is a multi-input multi-output module, where the index values ​​of multiple data elements are input into the lookup table module in parallel, and the lookup table module outputs the corresponding multiple exponential function values ​​in parallel. Alternatively, the table lookup process for multiple data elements can also be a serial process, i.e., the index values ​​of multiple data elements are input into the lookup table module sequentially, and the lookup table module outputs the exponential function value of each data element sequentially.

[0224] In step S1140, the addition result of multiple exponential function values ​​is obtained, and the index value conversion parameter is determined based on the addition result of multiple exponential function values.

[0225] The addition result of multiple exponential function values ​​can be obtained through the adder, and the index value conversion parameter can be determined based on the addition result of multiple exponential function values.

[0226] In step S1150, a selected second lookup table corresponding to the index value conversion parameter is determined from a plurality of alternative second lookup tables.

[0227] The second lookup table can be determined from multiple alternative second lookup tables by the module, which determines the selected second lookup table corresponding to the index value conversion parameter.

[0228] In one specific implementation, the compiler writes multiple alternative second lookup tables into the static storage module. After determining the selected second lookup table, the selected second lookup table can be loaded into the dynamic storage module.

[0229] In step S1160, the addition result is converted into the corresponding index value based on the index value conversion parameter.

[0230] The result of an addition operation can be converted into the corresponding index value using the first conversion circuit based on the index value conversion parameter.

[0231] In step S1170, based on the selected second lookup table and the index value of the addition result, the reciprocal corresponding to the addition result is obtained.

[0232] The reciprocal of the addition result can be obtained by using the lookup table module, based on the selected second lookup table and the index value of the addition result.

[0233] In step S1180, the result of the multiplication of the exponential function value of the i-th data element among multiple data elements and the reciprocal of the addition result is obtained.

[0234] A multiplier can be used to obtain the result of multiplying the exponential function value of the i-th data element among multiple data elements with the reciprocal of the addition result.

[0235] The relevant features of the data processing acceleration method in this application embodiment can be found in the relevant content of the foregoing hardware acceleration circuit embodiment, and will not be repeated here.

[0236] The data processing acceleration method according to the embodiments of this application can be applied to artificial intelligence accelerators. Figure 12 This is a schematic diagram of the structure of an artificial intelligence accelerator according to an embodiment of this application. See also... Figure 12 The artificial intelligence accelerator 1200 includes a memory 1210 and a processor 1220.

[0237] The artificial intelligence accelerator 1220 can be a general-purpose processor, such as a CPU (Central Processing Unit), or an artificial intelligence processor (IPU) for performing artificial intelligence operations. Artificial intelligence operations can include machine learning operations, neuromorphic operations, etc. Machine learning operations include neural network operations, k-means operations, support vector machine operations, etc. The artificial intelligence processor can, for example, include one or a combination of GPU (Graphics Processing Unit), DLA (Deep Learning Accelerator), NPU (Neural-Network Processing Unit), DSP (Digital Signal Processing Unit), Field-Programmable Gate Array (FPGA), and Application Specific Integrated Circuit (ASIC). This application does not limit the specific type of processor.

[0238] Memory 1210 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by processor 1220 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 1210 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1210 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0239] The memory 1210 stores executable code, which, when processed by the processor 1220, can cause the processor 1220 to execute part or all of the methods described above.

[0240] In one possible implementation, the artificial intelligence accelerator may include multiple processors, each of which can independently run various assigned tasks. This application does not limit the processors or the tasks they run.

[0241] It is understood that, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or in software program modules.

[0242] When the integrated unit / module is implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the artificial intelligence processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage module can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0243] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0244] In one possible implementation, an artificial intelligence chip is also disclosed, which includes the aforementioned hardware acceleration circuitry.

[0245] In one possible implementation, a board is also disclosed, which includes a storage device, an interface device, a controller, and the aforementioned artificial intelligence chip; wherein the artificial intelligence chip is connected to the storage device, the controller, and the interface device respectively; the storage device is used to store data; the interface device is used to realize data transmission between the artificial intelligence chip and external devices; and the controller is used to monitor the status of the artificial intelligence chip.

[0246] In one possible implementation, an electronic device is disclosed that includes the aforementioned artificial intelligence chip. The electronic device includes data processing devices, robots, computers, printers, scanners, tablets, smart terminals, mobile phones, dashcams, navigators, sensors, cameras, servers, cloud servers, cameras, camcorders, projectors, watches, headphones, mobile storage, wearable devices, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound scanners, and / or electrocardiographs.

[0247] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0248] Alternatively, this application may be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the steps of the methods described above according to this application.

[0249] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A hardware acceleration circuit, characterized in that, include: A storage module is used to store a first lookup table and a second lookup table, wherein the storage module includes a first storage area, which is used to store the first lookup table and the second lookup table in a time-sharing manner. A lookup table circuit is configured to, in response to the index values ​​of multiple data elements in a dataset, output multiple exponential function values ​​corresponding to the multiple data elements based on a first lookup table; and, in response to the index value of the addition result, output the reciprocal corresponding to the addition result based on a second lookup table, wherein the lookup table circuit is configured to call the first lookup table and the second lookup table at different time periods; An adder is used to output the addition result to the lookup table circuit, wherein the addition result is obtained by adding the multiple exponential function values; A multiplier is used to output the result of multiplying the exponential function value of the i-th data element with its reciprocal among the plurality of data elements, so as to obtain the flexible maximum value of the i-th data element.

2. The hardware acceleration circuit as described in claim 1, characterized in that, The exponential function value is data with a bit width of N1 bits, the addition operation result is data with a bit width of N2 bits, and the index value of the addition operation result is data with a bit width of N3 bits, wherein N1 and N3 are less than N2; The hardware acceleration circuit further includes: a first conversion circuit, used to convert the addition result into the index value of the addition result based on the index value conversion parameter.

3. The hardware acceleration circuit as described in claim 2, characterized in that: The storage module includes a static storage module; The index value conversion parameters are stored in the static storage module; The index value conversion parameter is determined based on the Gaussian distribution data obtained by statistically analyzing the results of multiple addition operations on multiple sample datasets.

4. The hardware acceleration circuit as described in claim 2, characterized in that, Also includes: An index value conversion parameter acquisition circuit is used to determine and output the index value conversion parameter based on the result of the addition operation; The first conversion circuit is used to convert the addition result into the corresponding index value based on the index value conversion parameter; The lookup table circuit outputs the reciprocal corresponding to the addition result based on the second lookup table. Specifically, it outputs the reciprocal corresponding to the addition result based on a selected second lookup table, wherein the second lookup table is the second lookup table corresponding to the index value conversion parameter among a plurality of candidate second lookup tables.

5. The hardware acceleration circuit as described in claim 4, characterized in that: The storage module includes a static storage module, in which the plurality of alternative second lookup tables are stored; or... The storage module includes a dynamic storage module, and the selected second lookup table is stored in the dynamic storage module.

6. The hardware acceleration circuit as described in claim 1, characterized in that, The storage module includes a first storage area and a second storage area, wherein the first lookup table is stored in the first storage area and the second lookup table is stored in the second storage area; The lookup table circuit includes: The first basic lookup table circuit unit includes a first logic circuit, a first input terminal group, a first control terminal group, and a first output terminal group. The first input terminal group is connected to the first storage area. The first logic circuit is configured to: output the corresponding exponential function value from the first output terminal group in response to the index value of the i-th data element in the data set input from the first control terminal group. The second basic lookup table circuit unit includes a second logic circuit, a second input terminal group, a second control terminal group, and a second output terminal group. The second input terminal group is connected to the second storage area. The second logic circuit is used to output the corresponding reciprocal from the second output terminal group in response to the index value of the addition operation result input from the second control terminal group. Wherein: the first basic lookup table circuit unit is N0 input and N1 output, the second basic lookup table circuit unit is N3 input and N4 output, the value range of N0, N1, N3, and N4 is [8, 32], N0 is the bit width of the index value of the data element, N1 is the bit width of the exponential function value, N3 is the bit width of the index value of the addition operation result, and N4 is the bit width of the reciprocal.

7. The hardware acceleration circuit as described in claim 1, characterized in that, The storage module includes a first storage area and a second storage area, wherein the first lookup table is stored in the first storage area and the second lookup table is stored in the second storage area; The lookup table circuit includes: A first basic lookup table circuit unit includes a first logic circuit, a first input group, a first control group, and a first output group. The first input group is connected to the storage module. The first logic circuit is configured to: respond to the index value of the i-th data element input from the first input group in a first time period, output the exponential function value corresponding to the i-th data element from the first output group based on the first lookup table; and, in a second time period after the first time period, respond to the index value of the addition result input from the first input group, output the reciprocal corresponding to the addition result from the first output group based on the second lookup table. in: The first basic lookup table circuit unit is in a state of N0 input and N1 output; or, The first basic lookup table circuit unit further includes a state control terminal group, used to input a first state control signal in a first time period and a second state control signal in a second time period, so as to configure the first basic lookup table circuit unit to: input N0 and output N1 in the first time period, and input N3 and output N4 in the second time period; wherein at least one pair of N0 and N3, and N1 and N4 are not equal, N0 is the bit width of the index value of the data element, N1 is the bit width of the exponential function value, N3 is the bit width of the index value of the addition operation result, and N4 is the bit width of the reciprocal.

8. The hardware acceleration circuit as described in claim 2, characterized in that, Also includes: The second conversion circuit is used to convert the multiplication result from a data with a width of N4 bits to a flexible maximum value with a width of N5 bits based on the index value conversion parameters, wherein N4 is greater than N5.

9. The hardware acceleration circuit as described in claim 2, characterized in that, Also includes: A subtractor is used to output the result of subtraction between multiple initial data in the initial dataset and the maximum value among the multiple initial data, so as to obtain the data set containing the multiple data elements; The third conversion unit is used to convert the plurality of data elements into a plurality of index values ​​corresponding to the first lookup table.

10. The hardware acceleration circuit as described in any one of claims 1 to 9, characterized in that: The exponential function value, the result of addition, the result of multiplication, and the reciprocal of addition are fixed-point integers.

11. An artificial intelligence chip, characterized in that, The chip includes the hardware acceleration circuitry as described in any one of claims 1 to 10.

12. A data processing acceleration method, characterized in that, include: Based on the first lookup table, obtain multiple exponential function values ​​corresponding to multiple data elements in the dataset; Obtain the result of the addition operation of the multiple exponential function values; The reciprocal of the addition result is obtained based on the second lookup table, wherein the first lookup table and the second lookup table are stored in the first storage area in a time-sharing manner, and the first lookup table and the second lookup table are called by the lookup table circuit at different time periods; The result of multiplying the exponential function value of the i-th data element among the plurality of data elements with the reciprocal of the addition operation result is obtained to obtain the flexible maximum value of the i-th data element.

13. The method according to claim 12, characterized in that, The step of obtaining the reciprocal corresponding to the addition result based on the second lookup table includes: Based on the index value conversion parameter, the addition result is converted from data with a width of N2 bits to an index value with a width of N3 bits, where N3 is less than N2; Based on the second lookup table and the index value of the addition result, the reciprocal corresponding to the addition result is obtained.

14. The method according to claim 13, characterized in that, Also includes: The compiler writes the index value conversion parameters into the static storage module. The index value conversion parameter is determined based on the Gaussian distribution data obtained by statistically analyzing the results of multiple addition operations on multiple sample datasets.

15. The method according to claim 13, characterized in that, Also includes: The index value conversion parameters are determined based on the result of the addition operation; Obtaining the reciprocal corresponding to the addition result based on the second lookup table and the index value of the addition result includes: From a plurality of alternative second lookup tables, a selected second lookup table corresponding to the index value conversion parameter is determined; Based on the selected second lookup table and the index value of the addition result, the reciprocal corresponding to the addition result is obtained.

16. The method according to claim 15, characterized in that, Also includes: The compiler writes the multiple alternative second lookup tables into the static storage module; And / or, The selected second lookup table is loaded into the dynamic storage module.

17. The method according to claim 12, characterized in that, Before obtaining the multiple exponential function values ​​corresponding to multiple data elements in the data set based on the first lookup table, the method further includes: subtracting the maximum value among multiple initial data in the initial dataset to obtain the data set including the multiple data elements; and / or, After obtaining the result of multiplying the exponential function value of the i-th data element among the plurality of data elements with the reciprocal, the method further includes: based on the index value conversion parameter, converting the multiplication result from data with a bit width of N4 bits to data with a bit width of N5 bits, wherein N4 is greater than N5.

18. The method as described in claim 12, characterized in that: The exponential function value, the result of addition, the result of multiplication, and the reciprocal of the result of addition are fixed-point integers; The bit width range of the exponential function value and the reciprocal is [8, 32].

19. The method according to any one of claims 12 to 18, characterized in that, The method described above is used to implement a flexible maximum function layer in a neural network, which is used to classify the data to be processed; wherein, The data to be processed includes at least one of voice data, text data, and image data.

20. An artificial intelligence accelerator, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 12-19.

Images