Activation function computation circuit, method and system

Abstract

The application provides an activation function calculation circuit, method and system, which can be applied to the technical field of hardware. The activation function calculation circuit comprises a data input end configured to receive input data in floating-point type; a lookup table module configured to store a lookup table, the lookup table comprising function values of an activation function stored in association with index addresses, the function values being obtained by sampling the activation function according to a predetermined sampling step; and a floating-point number to address module configured to: receive input data in floating-point type from the data input end; quantize a value of the input data by taking the predetermined sampling step as a quantization step to obtain a quantized value; and determine a target index address based on the quantized value, wherein the activation function calculation circuit is configured to query a target function value from the lookup table based on the target index address.

Description

Technical Field

[0001] This application relates to the field of hardware technology, and more specifically to an activation function calculation circuit, method, and system. Background Technology

[0002] With the rapid development of artificial intelligence servers, the requirements for computing circuits related to neural networks are also increasing. Furthermore, for the hardware circuits used to implement activation function calculations, reducing the hardware resources consumed and computational latency during computational tasks has become an urgent problem to be solved. Summary of the Invention

[0003] In view of the above problems, this application provides an activation function calculation circuit, method and system.

[0004] According to a first aspect of this application, an activation function calculation circuit is provided, comprising: a data input terminal for receiving floating-point type input data; a lookup table module for storing a lookup table, the lookup table including function values ​​of activation functions stored in association with index addresses, the function values ​​being obtained by sampling the activation functions according to a predetermined sampling step size; and a floating-point to address module for: receiving floating-point type input data from the data input terminal; quantizing the value of the input data with the predetermined sampling step size as the quantization step size to obtain a quantized value; and determining a target index address based on the quantized value, wherein the activation function calculation circuit is used to query the target function value from the lookup table based on the target index address.

[0005] According to a second aspect of this application, an activation function calculation method is provided, comprising: receiving floating-point type input data and quantizing the value of the input data based on a quantization step size to obtain a quantized value; determining a target index address based on the quantized value; and querying a target function value from a lookup table based on the target index address, wherein the lookup table includes function values ​​of activation functions stored in association with the index address, the function values ​​being obtained by sampling the activation function according to a predetermined sampling step size, the sampling step size being used as the quantization step size.

[0006] According to a third aspect of this application, an activation function calculation system is provided, comprising: a host and an external memory; and the aforementioned activation function calculation circuit, wherein the external memory is used to store a lookup table written by the host, the activation function calculation circuit retrieves the lookup table from the external memory, the host sends floating-point type input data to the activation function calculation circuit, and the activation function calculation circuit returns a function value corresponding to the input data to the host.

[0007] According to embodiments of this application, since the function value associated with the index address in the lookup table is obtained by sampling the activation function according to a predetermined sampling step size, the floating-point to address module can, upon receiving floating-point input data, quantize the input data value using the predetermined sampling step size as the quantization step size to obtain a quantized value. Then, based on the quantized value, the target index address is determined to query the corresponding function value. In this way, calculations on the data retrieved from the lookup table can be avoided at least partially, reducing the computational power consumed by hardware resources and the time spent calculating the activation function value, thereby improving the efficiency of calculating the activation function value. Attached Figure Description

[0008] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, which will be explained in conjunction with the drawings.

[0009] Figure 1 A schematic diagram of an activation function calculation circuit according to an embodiment of this application is shown.

[0010] Figure 2A A schematic diagram of the activation functions of GELU and SiLU according to embodiments of this application is shown.

[0011] Figure 2B A schematic diagram of input data of a floating-point type according to an embodiment of this application is shown.

[0012] Figure 3A A schematic diagram of an activation function calculation system according to an embodiment of this application is shown.

[0013] Figure 3B A schematic diagram of a lookup table configuration method according to an embodiment of this application is shown.

[0014] Figure 4 A schematic diagram of an activation function calculation circuit according to another embodiment of this application is shown.

[0015] Figure 5 A schematic diagram of an activation function calculation system according to another embodiment of this application is shown.

[0016] Figure 6 A schematic diagram of the upper and lower boundary detection module according to an embodiment of this application is shown.

[0017] Figure 7 A schematic diagram illustrating the operations performed by the result selection module according to an embodiment of this application is shown.

[0018] Figure 8 A schematic diagram of an activation function calculation method according to an embodiment of this application is shown. Detailed Implementation

[0019] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0020] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0021] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0022] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0023] It should be noted that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. The terms "installed," "connected," and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, or integral connections; they can be mechanical connections or electrical connections; they can be direct connections or indirect connections through an intermediate medium; they can be internal connections between two elements. The terms "parallel," "perpendicular," and "equal" include the described situation and situations similar to the described situation, the range of which is within an acceptable deviation range, wherein the acceptable deviation range is determined by those skilled in the art taking into account the measurement under discussion and the error associated with the measurement of a particular quantity (i.e., the limitations of the measurement system). For example, "parallel" includes absolute parallelism and approximate parallelism, where an acceptable deviation range for approximate parallelism can be, for example, within 5°; "perpendicular" includes absolute perpendicularity and approximate perpendicularity, where an acceptable deviation range for approximate perpendicularity can also be, for example, within 5°. "Equal" includes absolute equality and approximate equality, where an acceptable deviation range for approximate equality can be, for example, a difference between the two equal items being less than or equal to 5% of either one. Those skilled in the art will understand the specific meaning of the above terms in this application based on the specific circumstances.

[0024] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] In some schemes, the activation function value can be determined based on a lookup table. For example, the activation function value can be determined in two types of ways: direct and indirect methods.

[0026] For example, in the direct approach, the input data for the activation function can be used directly as the index address for querying the function value of the activation function. Then, the lookup table can be queried based on the index address to obtain the function value of the activation function corresponding to the input data.

[0027] For example, in an indirect approach, intermediate addresses can be generated based on the input data. Then, intermediate information can be retrieved from a lookup table using these addresses. This intermediate information can be the coefficients of a polynomial, allowing the activation function's value to be calculated using interpolation based on these coefficients and the corresponding polynomial. Specifically, after obtaining the intermediate information, the activation function's value can be obtained through simple calculations (e.g., multiplication and addition for integers or fixed-point numbers).

[0028] As discussed above, the direct method is more efficient at obtaining the activation function value compared to the indirect method. However, not all types of input data can be used as index addresses. For example, the decimal number 7 can be represented as 0111 in binary, which can be directly used as an index address in hardware circuits, allowing the corresponding function value to be retrieved through a lookup table. However, if the decimal "7" is represented in binary using FP16 (16-bit floating-point number), it can be represented as 0100011100000000. Therefore, it is difficult to directly use floating-point type input data with complex representation mechanisms as index addresses.

[0029] Therefore, when the input data is of integer or fixed-point type, either a direct or indirect method can be used to determine the corresponding function value. However, for floating-point data, it is difficult to obtain the activation function value directly. This is because floating-point data is difficult to convert into the address of the corresponding numerical range; that is, there is a contradiction between the representation format of floating-point data and the lookup table address that requires data monotonicity. Even if the address is generated by rounding the floating-point input data, the resulting address granularity is too large to be used in the direct method. Therefore, when the input data is of floating-point type, the indirect method is used. Due to the complexity of the floating-point data format, the input data needs to be rounded / classified to obtain an intermediate address, and then the activation function value is calculated using the method described above. Furthermore, since the intermediate information obtained by using the lookup table still needs to be calculated to obtain the activation function value, the computational power consumed by hardware resources is increased, and the delay in obtaining the activation function value is also increased.

[0030] In view of this, this application provides an activation function calculation circuit that can perform activation function calculation for floating-point numbers based on a lookup table, wherein the lookup table supports reconfiguration. Thus, through the activation function calculation circuit of this application embodiment, a direct lookup table-based activation function calculation can be performed on floating-point input data, thereby directly obtaining the function value of the activation function. This eliminates the step of recalculating based on intermediate information, reducing the computational power consumed by hardware resources and the delay in obtaining the function value. Furthermore, the activation function calculation circuit and system of this application can support dynamic reconfiguration of the lookup table to achieve flexible switching between lookup tables for various activation functions, reducing the occupation of hardware resources.

[0031] Figure 1 A schematic diagram of an activation function calculation circuit according to an embodiment of this application is shown.

[0032] like Figure 1 As shown, the activation function calculation circuit of this embodiment may include a lookup table module, a data input terminal, and a floating-point to address module.

[0033] In this embodiment, the lookup table module may store a lookup table. The lookup table may include the function value of the activation function stored in association with the index address.

[0034] Specifically, activation functions are widely used in various artificial intelligence (AI) neural network models, providing nonlinear capabilities to the neural network. For some activation functions, their graphs generally exhibit piecewise monotonicity.

[0035] Figure 2A The diagram illustrates the activation functions of the Gaussian Error Linear Unit (GELU) and the Sigmoid Gated Linear Unit (SiLU) according to embodiments of this application.

[0036] like Figure 2A As shown, the graph of the activation function exhibits interval characteristics, meaning that when the input data x (i.e., the value of the horizontal axis) is greater than or less than a certain value, its output function value (i.e., the value of the vertical axis) becomes predictable. For example, for GELU, when the input data is small, the output is essentially close to 0, and when the input data is large, the output function value is essentially equal to the input data. Therefore, calculation is only necessary when the input data lies within certain specific intervals. Furthermore, the formulas for the GELU and SiLU activation functions are shown below:

[0037] (1)

[0038] (2)

[0039] Where x represents the input data. GELU(x) represents the function value of the GELU activation function. SiLU(x) represents the function value of the SiLU activation function. tanh represents the hyperbolic tangent function. It is the standard Sigmoid function.

[0040] Based on the above formulas (1) and (2), the calculation formulas for activation functions in nonlinear computations typically involve complex mathematical operations, placing a huge computational burden on the hardware, and this burden increases with the scale of parallel computing. In this regard, the function value determination method for activation functions implemented using lookup tables can achieve a good balance between computational speed and accuracy. However, it should be understood that the above two activation functions are merely examples, and this application is not limited to these two activation functions.

[0041] Furthermore, in the embodiments of this application, the activation function (especially in the specific intervals that need to be calculated as described above) can be sampled according to a predetermined sampling step size to obtain the function value of the activation function. For example, the range of input data and the number of sampling points corresponding to SiLU and GELU are shown in formula (3). The symbols in formula (3) can be referred to the previous text and will not be repeated here.

[0042] (3)

[0043] according to Figure 2A The function graph can be sampled within a range of ±8; this should be understood as merely an example. Furthermore, for input data greater than +8, the corresponding function value can be considered the input data itself; for input data less than -8, the corresponding function value can be considered always 0. Based on this, for the input data range between ±8, a predetermined parameter STEP=256 (i.e., 2...) can be set. 8 If the sampling step size is 1 / STEP, then sampling is performed within ±8, for a total of [(+8)-(-8)] / (1 / STEP)=4096 points. The function value corresponding to each sampling point can be calculated, thus generating corresponding lookup tables for SiLU and GELU respectively. The size of each lookup table for SiLU and GELU contains 4096 final result values, corresponding to 4096 address inputs. The number of sampling points can be varied according to the required precision; 4096 values ​​are used here only as an example.

[0044] It should be understood that the method of this application can also be used for other activation functions, which will not be elaborated here. For example, a general-purpose high-level programming language (Python) tool can be used to generate the function value of the sampling points within the sampling interval according to the standard formula of the activation function, and a lookup table can be generated based on the function value and the corresponding sampling point index ("index address"). For example, when sampling in the interval (0, +8], the function value of the first sampling point (index "1"), such as SiLU (1*1 / 256), can be stored in the lookup table in association with the corresponding sampling point index "1" (actually represented as "0" in storage, because the index address starts from "0"). The function value of the second sampling point (index "2"), such as SiLU (2*1 / 256), can be stored in the lookup table in association with the corresponding sampling point index "2" (actually represented as "1" in storage, as mentioned above), and so on. In this way, the corresponding function value can be obtained directly from the lookup table based on the index address.

[0045] In this embodiment of the application, the data input terminal can receive floating-point type input data.

[0046] Figure 2B A schematic diagram of input data of a floating-point type according to an embodiment of this application is shown.

[0047] like Figure 2B As shown, the decimal number 10 is illustrated as an example, and its corresponding floating-point number is shown. In the FP16 floating-point data type, a floating-point number can have 16 bits. The input data for a floating-point type can include a sign bit and a numeric part. The numeric part can include an exponent part (5 bits) and a mantissa part (10 bits). It should be noted that the mantissa part also implicitly includes a highest-order bit that is "1". Based on this, 10 = 1 × 2^32. 3 ×1.25. 0x4900 corresponds to hexadecimal. Of course, this application is not limited to the FP16 floating-point data type, but can also be applied to other floating-point data types, such as FP32.

[0048] Furthermore, the floating-point to address conversion module can be connected to the data input terminal and the lookup table module to convert floating-point type input data into the corresponding index address, so that the corresponding target function value can be obtained from the lookup table module based on the index address.

[0049] For example, a floating-point to address module can receive floating-point type input data from the data input terminal. Then, using the predetermined sampling step size as the quantization step size, the value of the input data is quantized to obtain a quantized value. Specifically, based on the predetermined sampling step size, the corresponding sampling precision can be calculated, and then based on this sampling precision and the exponent and mantissa parts of the input data, the value of the input data is quantized to obtain a quantized value. Then, based on the quantized value, the target index address can be determined. For example, since sampling points are stored in hardware starting with 0, rather than starting with 1, a predetermined value (e.g., 1) can be subtracted from the quantized value to obtain the target index address. Thus, the activation function calculation circuit can look up the target function value from a lookup table based on the target index address.

[0050] Based on this, since the function values ​​associated with the index addresses in the lookup table are obtained by sampling the activation function according to a predetermined sampling step size, the floating-point to address module can, upon receiving floating-point input data, quantize the input data using the predetermined sampling step size as the quantization step size to obtain a quantized value. Then, based on the quantized value, the target index address is determined to retrieve the corresponding function value. In this way, computation on the data retrieved from the lookup table can be avoided at least partially, reducing the computational power consumed by hardware resources and the time spent calculating the activation function value, thus improving the efficiency of calculating the activation function value.

[0051] Furthermore, since there are many types of activation functions, setting up a separate lookup table for each type in hardware would be a significant waste of hardware resources. However, in this application, the activation functions can be reconfigured, allowing for convenient and efficient configuration of lookup tables for different activation functions in the activation function calculation circuit. This is explained below with reference to the accompanying drawings.

[0052] Figure 3A A schematic diagram of an activation function calculation system according to an embodiment of this application is shown.

[0053] like Figure 3AAs shown, the activation function computation system may include a host, external memory, and activation function computation circuitry. The activation function computation circuitry may include a data input terminal, a floating-point to address module, a lookup table module, a register module, and a Direct Memory Access (DMA) engine. The floating-point to address module, lookup table module, register module, and DMA engine may be integrated into a Field-Programmable Gate Array (FPGA) or an Application-Specific Integrated Circuit (ASIC). For example, the modules of the activation function computation circuitry may be implemented based on a Register-Transfer Level (RTL) hardware description language. It should be noted that the external memory may include at least one of memory integrated into the circuit board of the activation function computation circuitry and memory independent of the circuit board; this application does not limit this.

[0054] In this embodiment, the external memory can store different lookup tables written by the host, allowing the Direct Memory Access Engine (DMI) to write these different lookup tables into the lookup table module. For example, the DMI can retrieve different lookup tables from the external memory and write them into the lookup table module. This enables the configuration of different activation functions. Thus, this application can, based on actual needs, instantly configure the lookup table of the target activation function among multiple activation functions into the activation function calculation circuit, thereby enabling the calculation of multiple activation functions while saving hardware resources. Furthermore, it allows for the reconfiguration of activation functions, enabling the determination of different activation function values ​​through simple configuration while saving hardware resources.

[0055] For example, the host can control the direct memory access engine to read lookup tables for different activation functions from external memory by controlling the values ​​of registers in the register module. For instance, the register module may include a lookup table data ready register, a lookup table data length register, and a lookup table data address register.

[0056] The direct memory access engine in the activation function computation circuit can read the lookup table from external memory in response to a change in the value of the lookup table data ready register from a first value to a second value by the host. This is based on the data length value in the lookup table data length register and the data address value in the lookup table data address register. After reading the lookup table, the lookup table data ready register is changed from the second value back to the first value so that the host can re-change the value of the data ready register the next time the lookup table needs to be reconfigured. The following is combined with... Figure 3B Please provide a detailed explanation.

[0057] Figure 3B A schematic diagram of a lookup table configuration method according to an embodiment of this application is shown.

[0058] like Figure 3B As shown, the lookup table configuration method of this embodiment may include operations S310~S360.

[0059] When operating S310, the host writes the lookup table to external memory.

[0060] When operating the S320, the host configures the lookup table data address register. For example, the host can write the storage address of the lookup table in external memory (e.g., the start address, the end address, or both) into the lookup table data address register.

[0061] When operating the S330, the host configures the lookup table data length register. For example, the host can write the data length of the lookup table into the lookup table data length register.

[0062] When operating S340, the host configures the value of the lookup table data ready register, changing the value of the lookup table data ready register from the first value to the second value.

[0063] When operating the S350, the direct memory access engine responds to the change of the value of the lookup table data ready register from the first value to the second value, and writes the corresponding lookup table data from external memory to the lookup table module based on the address in the lookup table data address register and the data length in the lookup table data length register.

[0064] When operating the S360, the host changes the value of the lookup table data ready register from the second value back to the first value.

[0065] Based on this, the host can write the pre-prepared lookup table to a specific address in external memory via a high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIe) link. Then, the host can write the corresponding data address value to the lookup table data address register, the data length value to the lookup table data length register, and configure the lookup table data ready register to 1 (i.e., the second value, indicating that the lookup table data is stored in external memory). When the direct memory access engine detects that the lookup table data ready register is 1, it can read the corresponding lookup table data from external memory according to the lookup table data address register and the lookup table data length register, and write the lookup table data into the lookup table module. This completes the reconfiguration of the lookup table. Thus, since the direct memory access engine can automatically query the lookup table data and configure the lookup table module when it detects that the data ready register has changed to the second value, it achieves the reconfiguration of the activation function, at least partially reducing the hardware resources occupied by configuring lookup tables for multiple activation functions in the activation function calculation circuit. Furthermore, the host and the register module can also be interconnected via a PCIe link. The Direct Memory Access engine can also interconnect with external storage via a PCIe link.

[0066] After the reconfiguration operation is completed, the lookup table data ready register is written to 0 (i.e., the first value). The host can access the register module and configure the values ​​in the corresponding registers through hardware modules and standard protocols provided by relevant manufacturers (such as PCIe modules and the Advanced Dextensible Interface 4 (AXI4) protocol). It should be understood that the method of configuring registers in this application is not limited; this only describes the function of each register in the register module.

[0067] Based on this, after configuring the lookup table, the floating-point to address module can use the index address obtained by converting the input data of the floating-point type to look up the corresponding function value. For example, the floating-point to address module can look up the function value by converting the index address obtained by the mantissa of the input data (e.g., in...). Figure 2B In the example, subtract the sampling precision and the exponent of the input data (e.g., in 10-bit) from the sample precision. Figure 2B In the example, the shift value is calculated by summing 5b'10010 - the exponent bias value = 18 – 15 = 3), where the sampling precision is expressed in bits and corresponds to the sampling precision of a predetermined sampling step size (e.g., in a sampling step size of 1 / 2). 8(Sampling precision is 8 bits). Thus, based on the shift value, the mantissa portion of the input data (including the implicit highest bit "1") can be shifted to obtain shifted data. Then, based on the shifted data, the quantization value can be determined, and subsequently, the query address can be obtained based on this quantization value. This at least partially reduces the hardware overhead of calculating the activation function's value and improves the efficiency of obtaining the function value.

[0068] Based on this, the host can send floating-point type input data to the activation function calculation circuit, and the activation function calculation circuit returns the function value corresponding to the input data to the host, thus realizing the calculation of the activation function.

[0069] Specifically, the address conversion process of the floating-point to address module will be described below with reference to the accompanying drawings.

[0070] Figure 4 A schematic diagram of an activation function calculation circuit according to another embodiment of this application is shown.

[0071] like Figure 4 As shown, the floating-point to address module of this embodiment may include a first subtractor, a second subtractor, a third subtractor, a comparator, a quantization value selector, an address selector, a first shift unit, and a second shift unit. It should be noted that, to avoid the influence of line overlap on the illustration, therefore... Figure 4 The overlapping lines are shown using different shades of gray, but it should be understood that this is not intended to limit the scope of this application.

[0072] In this embodiment, the floating-point input data can be pre-split to obtain the sign bit, exponent part, and mantissa part. Furthermore, the exponent width, exponent bias, and mantissa width of the exponent part can be calculated. For example, in FP16 format, the exponent width exp_width=5, the mantissa width man_width=10, and the exponent bias exp_bias=15. Moreover, log2_step can be used to represent the base-2 logarithm of the predetermined parameter STEP=256, i.e., the sampling precision, which can be 8.

[0073] Further, for the exponent part, it can be input into the first subtractor. The first subtractor can receive the exponent bias value exp_bias of the input data as the minuend and the exponent part of the input data as the subtrahend. It subtracts the exponent bias value exp_bias from the exponent part to obtain the negative of the exponent value exp_tmp. Here, the exponent value is the actual exponent value corresponding to the exponent part, that is, exponent part – exponent bias value. Then, the first subtractor can input the negative of the exponent value exp_tmp into the second subtractor. The second subtractor can receive the sampling precision log2_step as the minuend and the output of the first subtractor as the subtrahend. It subtracts the sampling precision log2_step from the negative of the exponent value exp_tmp to obtain the sum of the sampling precision log2_step and the exponent value. Then, this sum is input into the third subtractor. The third subtractor can receive the mantissa width man_width as the minuend and the output of the second subtractor as the subtrahend. It subtracts the mantissa width man_width from the sum to obtain the shift value log. 2_ step_tmp.

[0074] For the mantissa (including the implicit highest bit "1"), it can be shifted left or right based on the shift value log2_step_tmp to obtain the shifted data, and then the quantized value. As mentioned above, since the 1 bit "1" at the highest bit of the mantissa is omitted when storing it, the mantissa can be padded with "1"s at the high bits before shifting to facilitate the shift. In the following text, the part padded with "1"s at the high bits will still be referred to as the "mantissa".

[0075] In some embodiments, the mantissa portion can be input into a first shift unit and a second shift unit. The first shift unit can right-shift the mantissa portion based on the shift value log2_step_tmp to obtain a first candidate shifted data. The second shift unit can left-shift the mantissa portion based on the two's complement of the shift value log2_step_tmp to obtain a second candidate shifted data. Then, the quantization selector can select either the first candidate shifted data or the second candidate shifted data as the shifted data based on the most significant bit of the shift value log2_step_tmp. Specifically, the most significant bit of the shift value log2_step_tmp indicates whether the shift value log2_step_tmp is positive or negative. Thus, when the most significant bit of the shift value log2_step_tmp indicates that the shift value log2_step_tmp is positive, the quantization selector selects the first candidate shifted data as the shifted data; when the most significant bit of the shift value log2_step_tmp indicates that the shift value log2_step_tmp is negative, the quantization selector selects the second shifted data as the shifted data. It should be understood that this process is essentially a selection of quantized values. Thus, based on the numerical relationship between the mantissa bit width and the sum of the aforementioned sampling precision log2_step and the exponent, the mantissa portion can be shifted to achieve quantization, allowing it to be numerically quantized into an index address. By left-shifting and right-shifting the mantissa data according to the bit offset, corresponding shifted data is obtained. These left-shifted and right-shifted data can be used as candidate quantized values. Then, based on the highest bit of the shifted value, the shifted data that best matches the input data can be selected as the quantized value, improving the accuracy of the determined quantized value.

[0076] Furthermore, the comparator can receive the output of the first subtractor and the sampling precision log2_step, compare the negative of the exponent value exp_tmp with the sampling precision log2_step, and output a comparison signal. The address selector can output shifted data or 0 as the quantized value based on the comparison signal. For example, if the negative of the exponent value exp_tmp is greater than the sampling precision log2_step, the address selector can choose to output 0. This is because the input data is very small in this case and can be considered as 0 under the quantization step size determined by the sampling precision. If the negative of the exponent value exp_tmp is less than or equal to the sampling precision log2_step, shifted data can be selected as the quantized value. In this way, the quantization of the floating-point input data is completed, thus obtaining the target index address.

[0077] The address generated by the floating-point address translation module in the example above can read 2048 function values. Furthermore, this module can also be used for floating-point formats such as FP32 (32-bit floating-point) or FP64 (64-bit floating-point), requiring only the modification of at least one of the parameters described above, such as exp_width, man_width, or exp_bias.

[0078] After obtaining the target index address, the target index address can be entered into the lookup table module to query the corresponding function value. In some embodiments of this application, function values ​​in other intervals can also be considered, as described below with reference to the accompanying drawings.

[0079] Figure 5 A schematic diagram of an activation function calculation system according to another embodiment of this application is shown.

[0080] like Figure 5 As shown, the activation function calculation system of this embodiment may include a host, external memory, and activation function calculation circuitry. The activation function calculation circuitry may include a data input terminal, a floating-point to address conversion module, a lookup table module, upper and lower boundary detection modules, a result selection module, a register module, and a direct memory access engine.

[0081] In the embodiments of this application, the lookup table module may include two Static Random-Access Memory (SRAM) modules, where the SRAM provides a standard read / write interface. Thus, the host, by controlling the aforementioned register module and direct memory access engine, uses this write interface to complete the reconfiguration operation of the lookup table. The lookup table module can then read the corresponding function value from the lookup table using the address generated by the floating-point to address module. In this example, since SiLU and GELU have 4096 sampling points, these 4096 data points are divided into two parts according to the positive or negative value of the input data. Each part contains 2048 data points, corresponding to the two input intervals [-8, 0) and (0, +8], which can be referred to as the negative value interval and the positive value interval, respectively. When the input is 0, the output is also 0; this value does not require a lookup table and will not be elaborated upon.

[0082] Thus, in practice, the lookup table module is divided into two independent parts, corresponding to the positive and negative intervals of the input range. It can use the index address generated by the floating-point to address module to read both independent parts at the same time and output two candidate function values ​​so that the result selection module can select the appropriate function value for output.

[0083] For example, the lookup table includes a first sub-table storing function values ​​corresponding to input data within a positive range and a second sub-table storing function values ​​corresponding to input data within a negative range. The floating-point to address conversion module can quantize the numerical portion of the input data to obtain a quantized value. The lookup table module can query the first and second candidate function values ​​from the first and second sub-tables respectively, based on the target index address. Then, the upper and lower boundary detection module can generate an overflow signal based on the data range information and the input data. Here, the data range information characterizes the data range of the input data for the activation function in the lookup table, and the overflow signal indicates whether the input data belongs to the data range. Finally, the result selection module, if the overflow signal indicates that the input data belongs to the data range, can select either the first or second candidate function value as the target function value based on the sign bit of the input data.

[0084] Thus, the reconfigurable lookup table module can retrieve the corresponding first and second candidate function values ​​from the lookup table data based on the target index address. Furthermore, the upper and lower boundary detection module can detect whether the floating-point input data belongs to the data range of the target activation function's input data based on data range information, and generate a corresponding detection signal. Therefore, when the detection signal indicates that the floating-point input data belongs to the data range, the result selection module can determine the target function value from the first or second candidate function value based on the sign bit of the input data, improving the accuracy of the determined target function value.

[0085] Furthermore, the upper and lower boundary detection module can determine whether floating-point input data belongs to the aforementioned data range based on the values ​​of the registers in the register module. Specifically, the register module can be used to control the reconfiguration lookup table, or to detect whether the input data overflows the upper and lower boundaries. For example, the register module includes registers for upper boundary values, registers for lower boundary values, registers for upper boundary overflow enable flags, registers for upper boundary overflow values, registers for lower boundary overflow enable flags, and registers for lower boundary overflow values.

[0086] For example, the register module may include the following registers: a lookup table data address register, with a 64-bit width, used to store the address of the lookup table data in external memory, from which the direct memory access engine reads the lookup table data; a lookup table data length register, with a 32-bit width, used to store the total byte length of the lookup table data; a lookup table data ready register, with a 1-bit width, used to store that the lookup table data address register and the lookup table data length register are ready, and the direct memory access engine can be started to move the lookup table data; a lookup table data upper bound register used to store the maximum value of the input data, the width of which varies depending on the type of floating-point data. In this application, the floating-point data type can be FP16, so the width can be 16 bits, and the value is +8 in FP16 format; a lookup table data lower bound register can be used to store the minimum value of the input data, the width of which varies depending on the type of floating-point data. In this application, the floating-point data type is FP16, so the width is 16 bits. The bit, with a value of -8 in FP16 format; The lookup table data upper bound overflow value enable register, which controls what value is used as the output register when the input data value is greater than the value of the lookup table data upper bound register. It can have a 1-bit width. When its value is 0, it means that when an upper bound overflow occurs, the input data is directly used as the output data. When its value is 1, it means that when an upper bound overflow occurs, the value in the lookup table data upper bound overflow value register is used as the output data. In this application, the input data is directly output when the upper bound overflow occurs, so this register is set to 0 in this application; The lookup table data upper bound overflow value register is used to control... When the input data exceeds the maximum value, and the lookup table data upper bound overflow enable register is 1, the value stored therein will be used as the output data. Its bit width varies depending on the floating-point data type; in this application, the floating-point data type is FP16, therefore the bit width is 16 bits. The lookup table data lower bound overflow enable register can be used to control which value is used as the output register when the input data value is less than the value of the lookup table data lower bound register. It has a 1-bit width; when its value is 0, it indicates that when a lower bound overflow occurs, the input data is directly used as the output data; when its value is 1, it indicates that when a lower bound overflow occurs, the lookup table data lower bound register is used. The value in the lower bound overflow register is used as the output data. In this application, 0 is directly output when the lower bound overflow occurs, so this register is set to 1 in this example. The lookup table data lower bound overflow register can be used to control the output data when the input data is less than the minimum value and the lookup table data lower bound overflow enable register is 1. Its bit width varies depending on the type of floating-point data. In this application, the floating-point data type is FP16, so the bit width is 16 bits. In this application, this register is set to 0. Combined with the lookup table data lower bound overflow enable register being set to 1, it means that 0 will be directly output when the lower bound overflow occurs.Furthermore, the upper and lower boundary detection modules can operate based on the values ​​of the aforementioned registers, as described below. Figure 6 Please provide an explanation.

[0087] Figure 6 A schematic diagram of the upper and lower boundary detection module according to an embodiment of this application is shown.

[0088] refer to Figure 5 and Figure 6 The upper and lower boundary detection module can include an upper boundary comparator and a lower boundary comparator.

[0089] The upper bound comparator can read the upper bound value from the lookup table data upper bound register in the register module, compare the upper bound value with the input data, and generate an upper bound overflow signal. The upper bound value represents the upper bound value of the input data of the activation function in the lookup table, and the upper bound overflow signal indicates whether the input data overflows the upper bound value.

[0090] The lower boundary comparator can read the lower boundary value from the lookup table data lower boundary register in the register module, compare the lower boundary value with the input data, and generate a lower boundary overflow signal. The lower boundary value represents the lower boundary value of the input data of the activation function in the lookup table, and the lower boundary overflow signal indicates whether the input data overflows the lower boundary value.

[0091] For example, the upper bound comparator compares the value of the lookup table data upper bound register with the value of the input data. If the value of the input data is greater than the value of the register, the upper bound overflow signal is "1", otherwise it is "0". Here, "1" indicates that the input data has overflowed the upper bound value, and "0" indicates that the input data has not overflowed the upper bound value.

[0092] The lower bound comparator compares the value of the lower bound register in the lookup table with the value of the input data. If the value of the input data is less than the value of the register, the lower bound overflow signal is "1"; otherwise, it is "0". "1" indicates that the input data has overflowed the lower bound value, and "0" indicates that the input data has not overflowed the lower bound value.

[0093] The result selection module can be implemented based on multiple selectors, which will not be elaborated here. Furthermore, when the upper boundary overflow signal indicates that the input data has overflowed the upper boundary value, the result selection module can select either the upper boundary overflow value or the input data as the target function value based on the upper boundary overflow value enable flag (i.e., the value of the lookup table data upper boundary overflow enable register). Alternatively, when the lower boundary overflow signal indicates that the input data has overflowed the lower boundary value, the module can select either the lower boundary overflow value or the input data as the target function value based on the lower boundary overflow value enable flag (i.e., the value of the lookup table data lower boundary overflow enable register). The following combines... Figure 7 Please provide an explanation.

[0094] Figure 7 A schematic diagram illustrating the operations performed by the result selection module according to an embodiment of this application is shown.

[0095] like Figure 7 As shown, the input to the result selection module may include the first and second candidate function values ​​from the reconfigurable lookup table module, the upper and lower boundary overflow signals from the upper and lower boundary detection module, the lookup table data upper or lower boundary overflow enable value and the lookup table data upper or lower boundary overflow value from the register module, and the sign bit from the input data.

[0096] Based on this, the result selection module can determine whether the upper boundary overflow signal indicates that the input data has overflowed the upper boundary. If the upper boundary overflow signal indicates that the input data has overflowed the upper boundary, it determines whether the upper boundary overflow enable value indicates that the upper boundary overflow value is enabled. If it is determined that the upper boundary overflow enable value indicates that the upper boundary overflow value is enabled, the upper boundary overflow value can be used as the target function value. Otherwise, the input data is used as the target function value.

[0097] The result selection module can also determine whether the lower boundary overflow signal indicates that the input data has overflowed the lower boundary. If the lower boundary overflow signal indicates that the input data has overflowed the lower boundary, it determines whether the lower boundary overflow enable value indicates that the lower boundary overflow value is enabled. If the lower boundary overflow enable value indicates that the lower boundary overflow value is enabled, the lower boundary overflow value can be used as the target function value. Otherwise, the input data is used as the target function value.

[0098] If the upper boundary overflow signal does not indicate that the input data overflows the upper boundary, and the lower boundary overflow signal does not indicate that the input data overflows the lower boundary, then it is determined whether the sign bit of the input data is 1. If it is not 1, it means that the input data is positive, and the first candidate function value is used as the target function value. Otherwise, the second candidate function value is used as the target function value.

[0099] Based on this, the upper and lower boundary detection modules detect whether the input data falls within the data range of the activation function's input data based on the upper and lower boundary values. This allows the result selection module to output accurate target function values ​​based on the overflow status of the input data, avoiding the problem of inaccurate target function values ​​determined due to the activation function's function value exhibiting different patterns in different input data intervals.

[0100] The overall architecture of the activation function calculation system of this application has been described above. The following will take the SiLU activation function as an example to describe the overall workflow of the activation function calculation system under FP16.

[0101] In this embodiment, the host can download pre-prepared SiLU lookup table data into a reconfigurable lookup table via a reconfiguration function. Then, the host can set the upper bound overflow enable register of the lookup table data to 0; set the lower bound overflow enable register of the lookup table data to 1; and set the lower bound overflow register of the lookup table data to 0; that is, when the upper bound overflows, the input data itself is directly output, and when the lower bound overflows, 0 is output.

[0102] Assuming the input data is 4.12, since this data is within ±8, no overflow occurs. Based on this, the binary representation of the corresponding FP16 floating-point number can be represented as 0_10001_0000011110, with "_" separating the different parts, from left to right: the sign bit, the exponent bit, and the mantissa bit.

[0103] At this time, the floating-point address conversion module works as follows: the opposite of the exponent value exp_tmp = exp_biasexp_bias-5'b10001=15-16=-2=6'b111110.

[0104] Furthermore, the shift value log2step_tmp = mantissa width - (sampling precision log2_step - the negative of the exponent exp_tmp) = 10 - (8 - (-2)) = 6'b000000. The highest bit of the shift value is 0, and the quantization selector selects the result of the first shift unit for output. Additionally, the highest bit of the mantissa can be padded with 1, resulting in 11'b10000011110, then shifted right by 0 bits, and finally subtracted by 1, resulting in 11'b10000011110 - 1 = 1053.

[0105] Thus, 1053 can be used as the address to read the reconfigurable lookup table of SiLU. The address of the lookup table starts counting from 0, and 1053 represents the 1054th number. According to theoretical calculation, 1 / STEP*1054=4.1172. Therefore, the lookup table actually obtains the result when the input data equals 4.1172. Since the sign bit of the input data is 0, that is, the input data is a positive number, the result of the lookup table in the positive region is used as the objective function value, which is 4.0510. The standard SiLU calculation result corresponding to 4.12 is 4.0547, and the accuracy basically meets the requirements.

[0106] Thus, this application, through a register module and a general direct memory access engine, implements the function of reconfiguring the lookup table for data of different activation functions. Furthermore, this application implements a floating-point to address conversion module to directly convert floating-point numbers to addresses, thereby enabling the approximate calculation of activation functions using a direct lookup method when the input data is a floating-point number.

[0107] Figure 8 A schematic diagram of an activation function calculation method according to an embodiment of this application is shown.

[0108] like Figure 8 As shown, the activation function calculation method of this embodiment may include operations S810~S840.

[0109] When operating the S810, it receives floating-point type input data.

[0110] When operating the S820, the value of the input data is quantized based on the quantization step size to obtain the quantized value.

[0111] When operating the S830, the target index address is determined based on the quantization value.

[0112] When operating S840, the target function value is retrieved from the lookup table based on the target index address.

[0113] The lookup table includes the function values ​​of activation functions stored in association with index addresses. The function values ​​are obtained by sampling the activation functions according to a predetermined sampling step size, which is used as the quantization step size.

[0114] It should be understood that the activation function calculation method of this application may also include other methods described above, such as retrieving a lookup table from external memory via a direct memory access engine and writing the lookup table into a lookup table module. The external memory is used to store different lookup tables written by the host, so that the direct memory access engine can write different lookup tables into the lookup table module. For details, please refer to the preceding description; further elaboration is not required here.

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0116] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0117] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. An activation function calculation circuit, characterized in that, The activation function calculation circuit includes: The data input terminal is used to receive floating-point type input data; A lookup table module for storing lookup tables, the lookup tables including function values ​​of activation functions stored in association with index addresses, the function values ​​being obtained by sampling the activation functions according to a predetermined sampling step size; and The floating-point to address module is used for: Receive the input data from the data input terminal; Using the predetermined sampling step size as the quantization step size, the values ​​of the input data are quantized to obtain quantized values; and Based on the quantization value, the target index address is determined. The activation function calculation circuit is used to query the target function value from the lookup table based on the target index address.

2. The activation function calculation circuit according to claim 1, characterized in that, The floating-point to address module is also used for: The shift value is calculated by subtracting the sum of the sampling precision and the exponent value of the input data from the mantissa bit width of the input data, wherein the sampling precision is the sampling precision corresponding to the predetermined sampling step size in terms of bit width. Based on the shift value, the mantissa portion of the input data is shifted to obtain shifted data; and The quantization value is determined based on the shifted data.

3. The activation function calculation circuit according to claim 2, characterized in that, The floating-point to address conversion module includes: The first subtractor is used to receive the exponent bias value of the input data as the minuend, receive the exponent part of the input data as the subtrahend, and perform subtraction calculation on the exponent bias value and the exponent part to obtain the opposite of the exponent value; The second subtractor is used to receive the sampling precision as the minuend and the output of the first subtractor as the subtrahend, and to perform subtraction calculation on the sampling precision and the opposite of the exponent value to obtain the sum of the sampling precision and the exponent value; The third subtractor is used to receive the mantissa width as the minuend, receive the output of the second subtractor as the subtrahend, and perform subtraction calculation on the mantissa width and the sum to obtain the shift value.

4. The activation function calculation circuit according to claim 3, characterized in that, The floating-point to address conversion module also includes: A comparator is used to receive the output of the first subtractor and the sampling precision, compare the opposite of the exponent value with the sampling precision, and output a comparison signal; An address selector is used to output the shifted data or output 0 as the quantization value based on the comparison signal.

5. The activation function calculation circuit according to any one of claims 2 to 4, characterized in that, The floating-point to address module is also used for: Based on the shift value, the mantissa portion is shifted left or right to obtain the shifted data.

6. The activation function calculation circuit according to claim 5, characterized in that, The floating-point to address conversion module also includes: The first shift unit is used to right-shift the mantissa portion based on the shift value to obtain the first candidate shift data; The second shift unit is used to left-shift the mantissa portion based on the two's complement of the shift value to obtain the second candidate shift data; and A quantization value selector is used to select either the first candidate shift data or the second candidate shift data as the shift data based on the most significant bit of the shift value.

7. The activation function calculation circuit according to claim 6, characterized in that, The quantization value selector is also used for: If the most significant bit of the shift value indicates that the shift value is positive, the first candidate shift data is selected as the shift data. If the highest bit of the shift value indicates that the shift value is negative, the second candidate shift data is selected as the shift data.

8. The activation function calculation circuit according to any one of claims 1 to 4, characterized in that, The activation function calculation circuit further includes: A direct memory access engine is used to retrieve the lookup table from external storage and write the lookup table into the lookup table module; wherein the external storage is used to store different lookup tables written by the host, so that the direct memory access engine can write different lookup tables into the lookup table module.

9. The activation function calculation circuit according to any one of claims 1 to 4, characterized in that, The lookup table includes a first sub-table that stores function values ​​corresponding to input data in the positive range and a second sub-table that stores function values ​​corresponding to input data in the negative range. The floating-point to address module is also used to quantize the numerical part of the input data to obtain the quantized value; The activation function calculation circuit is also used to query the first candidate function value and the second candidate function value from the first sub-table and the second sub-table respectively based on the target index address; The activation function calculation circuit further includes: The upper and lower boundary detection module is used to generate an overflow signal based on the data range information and the input data, wherein the data range information represents the data range of the input data of the activation function in the lookup table, and the overflow signal is used to indicate whether the input data belongs to the data range; The result selection module is used to select either the first candidate function value or the second candidate function value as the target function value based on the sign bit of the input data when the overflow signal indicates that the input data belongs to the data range.

10. The activation function calculation circuit according to claim 9, characterized in that, The upper and lower boundary detection module includes: An upper boundary comparator is used to compare an upper boundary value with the input data and generate an upper boundary overflow signal, wherein the upper boundary value represents the upper boundary value of the input data of the activation function in the lookup table, and the upper boundary overflow signal indicates whether the input data overflows the upper boundary value; A lower boundary comparator is used to compare the lower boundary value with the input data and generate a lower boundary overflow signal, wherein the lower boundary value represents the lower boundary value of the input data of the activation function in the lookup table, and the lower boundary overflow signal indicates whether the input data overflows the lower boundary value; The result selection module is also used for: If the upper boundary overflow signal indicates that the input data overflows the upper boundary value, then based on the upper boundary overflow value enable flag, either the upper boundary overflow value or the input data is selected as the target function value; or If the lower boundary overflow signal indicates that the input data overflows the lower boundary value, the lower boundary overflow value or the input data is selected as the target function value based on the lower boundary overflow value enable flag.

11. The activation function calculation circuit according to claim 10, characterized in that, The activation function calculation circuit further includes a register module, which includes a register for the upper boundary value, a register for the lower boundary value, a register for the upper boundary overflow value enable flag, a register for the upper boundary overflow value, a register for the lower boundary overflow value enable flag, and a register for the lower boundary overflow value.

12. The activation function calculation circuit according to claim 11, characterized in that, The register module also includes a lookup table data ready register, a lookup table data length register, and a lookup table data address register; The activation function calculation circuit is also used for: In response to the host changing the value of the lookup table data ready register from a first value to a second value, the lookup table is read from external memory based on the data length value in the lookup table data length register and the data address value in the lookup table data address register; After reading the lookup table, the value of the lookup table data ready register is changed from the second value to the first value.

13. A method for calculating an activation function, characterized in that, The activation function calculation method includes: Receive floating-point type input data, and quantize the value of the input data based on the quantization step size to obtain the quantized value; Based on the quantization value, the target index address is determined; Based on the target index address, retrieve the target function value from the lookup table. The lookup table includes function values ​​of activation functions stored in association with index addresses. These function values ​​are obtained by sampling the activation functions according to a predetermined sampling step size, which is used as the quantization step size.

14. The activation function calculation method according to claim 13, characterized in that, The activation function calculation method further includes: The lookup table is retrieved from external storage via the direct memory access engine and then written to the lookup table module. The external memory is used to store different lookup tables written by the host, so that the direct memory access engine can write different lookup tables into the lookup table module.

15. An activation function calculation system, characterized in that, include: Main unit and external storage; as well as The activation function calculation circuit as described in any one of claims 1 to 12, The external memory is used to store a lookup table written by the host, and the activation function calculation circuit retrieves the lookup table from the external memory. The host sends floating-point type input data to the activation function calculation circuit, and the activation function calculation circuit returns a function value corresponding to the input data to the host.

Images