Computing circuits and AI accelerators

Abstract

The embodiment of the present application provides a computing circuit and an artificial intelligence accelerator, which relate to the field of integrated circuit technology. The computing circuit receives multiple first blocks of floating-point numbers, calculates the multiple first blocks of floating-point numbers, and obtains multiple second blocks of floating-point numbers, which includes: a first computing unit, a conversion unit, and a second computing unit; the first computing unit receives multiple first blocks of floating-point numbers, calculates the mantissa items of the multiple first blocks of floating-point numbers, and obtains multiple first intermediate floating-point numbers; the conversion unit normalizes the multiple first intermediate floating-point numbers into multiple normalized mantissa items; the second computing unit calculates the multiple normalized mantissa items to obtain multiple second intermediate floating-point numbers, and normalizes the multiple second intermediate floating-point numbers into multiple second blocks of floating-point numbers based on the normalized mantissa items. The computing circuit reduces the logic complexity and power consumption of the circuit, effectively reduces the data storage requirements, and solves the problem of insufficient storage space and bandwidth for artificial intelligence services.

Description

Technical Field

[0001] The present application relates to the field of integrated circuit technology, and in particular to a computing circuit and an artificial intelligence accelerator. Background Art

[0002] An AI (Artificial Intelligence) accelerator is a specially designed hardware accelerator or computer system whose main function is to accelerate artificial intelligence applications, especially artificial neural networks, machine learning, machine vision, and other data-intensive or sensor-driven tasks. Artificial intelligence algorithms involve huge amounts of computation and storage. For example, the normalized exponential function (Softmax function) is one of the common operators in artificial intelligence models. It compresses an arbitrary K-dimensional real vector into another K-dimensional real vector, and makes the value range of each element in the new vector within the interval (0,1), and the sum of all elements is 1. In probability theory, the output of the normalized exponential function can be used to represent a classification probability distribution. The mathematical expression of the normalized exponential function is shown as follows:

[0003]

[0004] However, the currently commonly used normalized exponential function calculation circuit uses floating-point numbers as the circuit. The normalized exponential function calculation process involves a large number of exponentiation and division operations. Traditional circuit designs all use floating-point calculation units, that is, floating-point numbers are used as the input and output of the circuit, which has a large area, high power consumption, and high storage and bandwidth requirements, resulting in challenges such as insufficient computing power, storage space, and bandwidth for artificial intelligence services. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, an embodiment of the present application provides a computing circuit and an artificial intelligence accelerator.

[0006] In a first aspect, an embodiment of the present application provides a calculation circuit, the calculation circuit receiving a plurality of first blocks of floating-point numbers, calculating the plurality of first blocks of floating-point numbers, and obtaining a plurality of second blocks of floating-point numbers; the plurality of first blocks of floating-point numbers share a first exponent term, and the plurality of second blocks of floating-point numbers share a second exponent term;

[0007] The computing circuit comprises: a first computing unit, a conversion unit, and a second computing unit; the first computing unit is coupled to the conversion unit, and the conversion unit is coupled to the second computing unit;

[0008] The first calculation unit receives the plurality of first blocks of floating-point numbers, and calculates the mantissa items of the plurality of first blocks of floating-point numbers to obtain a plurality of first intermediate floating-point numbers;

[0009] The conversion unit receives the plurality of first intermediate floating-point numbers, and normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items;

[0010] The second calculation unit calculates the multiple normalized mantissa items to obtain multiple second intermediate floating-point numbers, and normalizes the multiple second intermediate floating-point numbers into the multiple second block floating-point numbers based on the normalized mantissa items.

[0011] In an optional embodiment, the first calculation unit includes a maximum value extraction subunit, a vector subtraction subunit, and a vector index calculation subunit;

[0012] The maximum value extraction subunit receives the plurality of first-block floating-point numbers, obtains the largest mantissa item among the mantissa items of the plurality of first-block floating-point numbers, and uses the largest mantissa item as the first mantissa item;

[0013] The vector subtraction subunit calculates the difference between the mantissa item of each of the first blocks of floating-point numbers and the first mantissa item respectively, and uses the difference as the second mantissa item;

[0014] The vector exponent calculation subunit constructs a plurality of third-block floating-point numbers based on the first exponent term and the second mantissa term, and performs exponential operation on the third-block floating-point numbers to obtain the plurality of first intermediate floating-point numbers.

[0015] In an optional embodiment, the maximum value extraction subunit includes a comparator array, and the comparator array compares the mantissa items of the plurality of first block floating-point numbers to obtain the largest mantissa item among the mantissa items of the plurality of first block floating-point numbers.

[0016] In an optional embodiment, the vector subtraction subunit comprises a subtractor array, and the subtractor array calculates the difference between the mantissa term of each of the first block floating-point numbers and the first mantissa term.

[0017] In an optional embodiment, the vector exponent calculation subunit includes an exponent calculation module, which concatenates the first exponent term and the second mantissa term, queries the exponential function value corresponding to the concatenation result by a table lookup method, and the exponential function value is the first intermediate floating-point number; or, the exponential calculation module obtains the first intermediate floating-point number by a fitting method.

[0018] In an optional embodiment, the conversion unit includes a shared index extraction subunit and a normalization subunit;

[0019] The shared exponent extraction subunit receives the plurality of first intermediate floating-point numbers, obtains the largest exponent term among the exponent terms of the first intermediate floating-point numbers, and uses the largest exponent term as the third exponent term;

[0020] The normalization subunit normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa terms according to the third exponent term.

[0021] In an optional embodiment, the normalization subunit includes a plurality of basic circuits, wherein the basic circuits include an exponent extractor, a mantissa extractor, a subtractor, and a shift circuit;

[0022] The exponent extractor obtains the exponent term of the first intermediate floating-point number;

[0023] The mantissa extractor obtains the mantissa term of the first intermediate floating-point number;

[0024] The subtractor obtains a difference between the exponential term of the first intermediate floating-point number and the third exponential term;

[0025] The shift circuit shifts and aligns the mantissa term of the first intermediate floating-point number according to the difference between the exponent term of the first intermediate floating-point number and the third exponent term to obtain the normalized mantissa term.

[0026] In an optional embodiment, the second calculation unit includes a summation subunit and a multiplication subunit;

[0027] The summing subunit calculates the sum of the normalized mantissa items, and takes the reciprocal of the sum of the normalized mantissa items as an intermediate value;

[0028] The multiplication subunit calculates the product of the mantissa term of the intermediate value and the normalized mantissa term, normalizes and aligns the third mantissa term with the exponent term of the intermediate value, and obtains a target mantissa term and a target exponent term, wherein the target mantissa term is the mantissa term of the second block of floating-point numbers, and the target exponent term is a second exponent term shared by the second block of floating-point numbers.

[0029] In an optional embodiment, the multiplication subunit includes a multiplier and the normalization subunit;

[0030] The multiplier calculates the product of the mantissa term of the intermediate value and the normalized mantissa term to obtain a third mantissa term;

[0031] The normalization subunit normalizes and aligns the third mantissa term with the exponent term of the intermediate value to obtain a target mantissa term and a target exponent term.

[0032] In a second aspect, an embodiment of the present application provides an artificial intelligence accelerator, which includes the computing circuit described in any embodiment of the present application.

[0033] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0034] The computing circuit provided in the embodiment of the present application receives a plurality of first blocks of floating-point numbers, calculates the plurality of first blocks of floating-point numbers, and outputs a plurality of second blocks of floating-point numbers. The computing circuit includes a first computing unit, a conversion unit, and a second computing unit. The first computing unit receives a plurality of first blocks of floating-point numbers, calculates the mantissa items of the plurality of first blocks of floating-point numbers, and obtains a plurality of first intermediate floating-point numbers; the conversion unit normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items; the second computing unit calculates the plurality of normalized mantissa items, obtains a plurality of second intermediate floating-point numbers, and normalizes the plurality of second intermediate floating-point numbers into a plurality of second blocks of floating-point numbers based on the normalized mantissa items.

[0035] The computing circuit provided in the embodiment of the present application uses block floating-point numbers as the input and output of the circuit. Compared with the traditional circuit using floating-point numbers as input and output, the logic complexity and power consumption are greatly reduced, and the data storage requirements can be effectively reduced, solving the problem of insufficient storage space and insufficient bandwidth for artificial intelligence services such as large models. Moreover, based on the characteristics of block floating-point number shared exponents, the computing circuit supports parallel computing, which can effectively improve the computing processing capability, and is easily compatible with other block floating-point operators in the artificial intelligence model algorithm (such as matrix multiplication, etc.), without the need for numerical conversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0037] FIG. 1( a ) shows a schematic diagram of the structure of a single-precision floating point number;

[0038] Figure 1(b) shows a schematic diagram of the structure of Google BFloat16;

[0039] Figure 1(c) shows a schematic diagram of the structure of a block floating point number;

[0040] Figure 2 A schematic diagram of the structure of a computing circuit according to an embodiment of the present application is shown;

[0041] Figure 3 A schematic diagram showing the structure of a first computing unit in a computing circuit according to an embodiment of the present application is shown;

[0042] Figure 4 A schematic diagram showing the structure of a maximum value extraction subunit in a calculation circuit in an embodiment of the present application is shown;

[0043] Figure 5 A schematic diagram showing the structure of a vector subtraction subunit in a calculation circuit in an embodiment of the present application is shown;

[0044] Figure 6A schematic diagram showing the structure of a vector index calculation subunit in a calculation circuit in an embodiment of the present application is shown;

[0045] Figure 7 A schematic diagram showing the structure of a conversion unit in a computing circuit according to an embodiment of the present application is shown;

[0046] Figure 8 A schematic diagram showing the structure of a shared index extraction subunit in a calculation circuit in an embodiment of the present application is shown;

[0047] Fig. 9 A schematic diagram of the structure of a normalization subunit in a calculation circuit according to an embodiment of the present application is shown;

[0048] Fig.10 A schematic diagram showing the structure of a second computing unit in a computing circuit according to an embodiment of the present application is shown;

[0049] Fig.11 A schematic diagram showing the structure of a summing subunit in a calculation circuit in an embodiment of the present application is shown;

[0050] Fig.12 A schematic diagram of the structure of a multiplication subunit in a calculation circuit in an embodiment of the present application is shown;

[0051] Fig.13 A data flow diagram of a computing circuit according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0053] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0054] For ease of understanding, the technical terms related to this application are explained below.

[0055] A floating-point number is a number that is expressed in a special encoding format. This real number is obtained by multiplying an integer or fixed-point number (i.e., the mantissa) by an integer power of a base number (usually 2 in computers). In computer systems, traditional floating-point number definitions such as float32 (32-bit floating-point number), float64 (64-bit floating-point number), etc. (refer to the binary floating-point arithmetic standard IEEE-754) are mainly composed of three parts: the sign bit, the exponent term, and the mantissa term. Fixed-point numbers are a method of representing numbers used in computers, and their characteristic is that the decimal point position of the numbers involved in the calculation is fixed. Fixed-point numbers include fixed-point integers and fixed-point decimals. Compared with fixed-point numbers, floating-point numbers have higher expression accuracy and dynamic range.

[0056] Block floating point is an innovative floating point number. In recent years, due to its small data storage space and high precision, it has gradually become a hot spot in the acceleration of artificial intelligence model reasoning, especially for bandwidth storage bottleneck businesses such as large models. Figure 1 (a), Figure 1 (b), and Figure 1 (c) respectively show single-precision floating point (float32), Google BFloat16 (Google Brain Floating point, 16-bit brain floating point, the main idea is to provide a 16-bit floating point format, its dynamic range is the same as the standard IEEE-FP32, but the precision is lower), and the structural diagram of block floating point. In Figure 1 (a), Figure 1 (b), and Figure 1 (c), a0, a1, a2, and an represent different floating point numbers, the purple rectangle represents the sign bit (sign), the blue rectangle represents the exponent term (exponent), and the yellow rectangle represents the mantissa term (mantissa). As shown in Figure 1 (c), the block floating point number consists of a group of sign bits and mantissa terms, and all data in the group share an exponent term.

[0057] Figure 2 The schematic diagram of the structure of the calculation circuit of an embodiment of the present application is shown. The calculation circuit receives a plurality of first blocks of floating point numbers, denoted as z i =2 p ·d i , where p, d i All are fixed-point values, p represents the first floating-point number z i The shared index term (i.e. the first index term), d i Represents the mantissa of each first block of floating-point numbers, where i is a positive integer greater than 1, such as 15.

[0058] The calculation circuit calculates the first plurality of floating point numbers to obtain the second plurality of floating point numbers, which are recorded as h i =2 q ·m i , where q, mi All are fixed-point numbers, q represents the second floating-point number z i The shared index term (i.e., the second index term), m i Represents the mantissa of each second block of floating-point numbers.

[0059] The computing circuit of the embodiment of the present application can be applied to an artificial intelligence accelerator. For example, it can be applied in an artificial intelligence accelerator to improve the computational efficiency of probability-based multi-classification methods such as multinomial logistic regression, multinomial linear discriminant analysis, and deep learning algorithms, reduce the power consumption of the accelerator and data storage requirements, and effectively save computing resources and storage resources. As an optional example, in the large model Transformer structure, the normalized exponential function is widely used in the multi-head attention layer, as shown in the following formula, so the computing circuit can be applied to the artificial intelligence accelerator to accelerate the computing efficiency of the multi-head attention layer.

[0060]

[0061] like Figure 2 As shown, the calculation circuit includes a first calculation unit H1, a conversion unit H2, and a second calculation unit H3. The first calculation unit H1 is coupled to the conversion unit H2, and the conversion unit H2 is coupled to the second calculation unit H3. In the calculation circuit provided in the embodiment of the present invention, the first calculation unit H1, the conversion unit H2, and the second calculation unit H3 are all hardware circuit units composed of hardware circuits. As for the composition of each hardware circuit unit such as the first calculation unit H1, the conversion unit H2, and the second calculation unit H3, any hardware circuit unit that can realize its corresponding function is acceptable, and no limitation is made here.

[0062] The first calculation unit H1 receives a plurality of first blocks of floating-point numbers, calculates the mantissa items of the plurality of first blocks of floating-point numbers, and obtains a plurality of first intermediate floating-point numbers.

[0063] The goal of the calculation circuit in the embodiment of the present application is to calculate in, Represents the first floating point number z of the i-th block i The corresponding calculation result, K represents the number of floating-point numbers in the first block. Overflow, in the first calculation unit, it is mathematically equivalent, that is, before performing exponential calculation, the mantissa of each first block of floating-point numbers is processed, and the maximum mantissa is subtracted from it, and the expression is substituted to obtain:

[0064] Therefore, the first computing unit H1 receives multiple first block floating-point numbers, determines the maximum value among the mantissa terms of the multiple first block floating-point numbers, calculates the difference between the mantissa term of each first block floating-point number and the maximum value of the mantissa term, uses the difference as an exponent term adjustment term, performs exponent calculation based on the first exponent term and the exponent term adjustment term shared by the first block floating-point numbers, and obtains multiple first intermediate floating-point numbers.

[0065] In an alternative embodiment, Figure 3 FIG. 4 shows a schematic diagram of the structure of the first computing unit in an embodiment of the present application. Figure 3 As shown, the first calculation unit H1 includes a maximum value extraction subunit Q1, a vector subtraction subunit Q2, and a vector index calculation subunit Q3.

[0066] The maximum value extraction subunit Q1 receives a plurality of first blocks of floating point numbers z i =2 p ·d i , get the mantissa d of each first block of floating point numbers i , compare the mantissa d of each first block of floating point numbers i , get the largest mantissa item among the mantissa items of multiple first block floating point numbers, and take the largest mantissa item as the first mantissa item d max .

[0067] In some optional implementation scenarios, the structural diagram of the maximum value extraction subunit Q1 is as follows: Figure 4 As shown, the maximum value extraction subunit Q1 is composed of a comparator array, which is constructed by organizing a plurality of comparators in a tree structure. The comparator compares two input values ​​and outputs a larger value. Figure 4 A maximum value extraction subunit Q1 constructed using fifteen comparators is shown, which receives the mantissa items {d0, d1, d2…d 15}, output the maximum value of the mantissa d max .

[0068] The vector subtraction subunit Q2 calculates the mantissa term d of each first block of floating point numbers respectively. i and the first mantissa d max The difference is taken as the second mantissa d′ i .

[0069] In some optional implementation scenarios, the structural diagram of the vector subtraction subunit Q2 is as follows: Figure 5 The vector subtraction subunit Q2 is composed of a subtractor array ( Figure 5 The vector subtraction subunit is shown using sixteen subtractors. One input of the subtractor is fed with the mantissa term d of the first block of floating point numbers. i(i=0,1,2…15), the other input terminal inputs the first mantissa d max , perform subtraction on the two input values ​​and output the difference d′ between them i =d i -d max .

[0070] The vector exponent calculation subunit Q3 calculates the first exponent term p and the second mantissa term d′ based on i , construct multiple third blocks of floating point numbers z′ i =2 p ·d′ i , perform exponential operation on the third block of floating-point numbers to obtain multiple first intermediate floating-point numbers The vector exponential calculation subunit Q3 performs exponential calculation on the input block floating point number and outputs a regular floating point number. Optionally, the vector exponential calculation subunit Q3 can implement the exponential calculation function by a fitting method or a look-up table (LUT).

[0071] In some optional implementation scenarios, the structural diagram of the vector index calculation subunit Q3 is as follows: Figure 6 The vector exponent calculation subunit Q3 includes an exponent calculation module LUT. The exponent calculation module LUT splices the fixed-point number p and the fixed-point number d′ i As the lookup index, the exponential function value corresponding to the splicing result is queried by the table lookup method to obtain the floating point result x of the exponential calculation i . Assume that the first exponent term p is 4 bits and the second mantissa term d′ i For 8-bit data, the concatenation of the first exponent term and the second mantissa term is 12-bit data. For example, if p = y1y2y3y4, the second mantissa term d′1 = y5y6y7y8y9y 10 y 11 y 12 , concatenate p and d′1, the concatenation result is y1y2y3y4y5y6y7y8y9y 10 y 11 y 12 y1, y2, …, y 12 Represents the binary value (0 or 1) of different bit positions.

[0072] In other optional embodiments, the vector exponent calculation unit Q3 may be composed of a fitting unit, and the floating point result x of the exponential calculation is obtained by the fitting unit. i .

[0073] In the embodiment of the present application, to prevent Overflow, in the first calculation unit, it is mathematically equivalent. Before performing exponential calculation, the mantissa of each first block of floating-point numbers is subtracted from the maximum mantissa, and then substituted into z. i The expression can be obtained:

[0074] The conversion unit H2 receives a plurality of first intermediate floating-point numbers, and normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items.

[0075] After receiving multiple first intermediate floating-point numbers, the conversion unit H2 first obtains the exponent term from the binary code of the first intermediate floating-point number, determines the largest exponent term, and then normalizes the multiple first intermediate floating-point numbers based on the maximum value of the exponent term, and uses the normalized result as the mantissa term of the block floating-point number (i.e., the normalized mantissa term).

[0076] In an optional embodiment, if Figure 7 As shown, the conversion unit H2 includes a shared index extraction subunit U1 and a normalization subunit U2.

[0077] The shared exponent extraction subunit U1 receives a plurality of first intermediate floating-point numbers, obtains the largest exponent term among the exponent terms of the first intermediate floating-point numbers, and uses the largest exponent term as the third exponent term p′.

[0078] In some optional implementation scenarios, the structural diagram of the shared index extraction subunit U1 is as follows: Figure 8 As shown. The shared exponent extraction subunit includes an exponent extraction unit and a comparator array. The exponent extraction unit extracts the exponent term from the input floating point binary code and inputs it to the comparator array. Optionally, the comparator array in the shared exponent extraction subunit U1 can be connected to Figure 4 The comparator arrays shown have the same structure, which compare the input exponential terms and output the maximum exponential term.

[0079] The normalization subunit U2 normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items according to the third exponent item p′. For example, the normalization subunit U2 first separates the sign bit, the exponent item, and the mantissa item of the first intermediate floating-point number from the binary code, and subtracts the maximum value of the exponent item from the exponent item, and then uses this value to shift and align the mantissa item, and splices it with the sign bit to obtain the normalized mantissa item.

[0080] In some optional implementation scenarios, the structural diagram of the normalization subunit U2 is as follows: Fig. 9 The normalization subunit is composed of a plurality of parallel basic circuits G, and the basic circuit G includes an exponent extractor, a mantissa extractor, a subtractor and a shift circuit.

[0081] The exponent extractor obtains the exponent term of the first intermediate floating-point number.

[0082] The mantissa extractor obtains the mantissa term of the first intermediate floating-point number.

[0083] The subtractor obtains a difference between the exponent term of the first intermediate floating-point number and the third exponent term.

[0084] The shift circuit shifts and aligns the mantissa term of the first intermediate floating point number according to the difference between the exponent term of the first intermediate floating point number and the third exponent term to obtain a normalized mantissa term d″ i .

[0085] The second calculation unit H3 calculates the multiple normalized mantissa items to obtain multiple second intermediate floating-point numbers, and normalizes the multiple second intermediate floating-point numbers into multiple second block floating-point numbers based on the normalized mantissa items.

[0086] In some optional embodiments, such as Fig.10 As shown, the second calculation unit H3 includes a summation subunit W1 and a multiplication subunit W2.

[0087] The summation subunit W1 calculates the normalized mantissa term d″ i The sum of the normalized mantissa terms is taken as the reciprocal of the sum of the normalized mantissa terms as the intermediate value, and then the exponential term p″ and the mantissa term of the intermediate value are output. The output of the summation subunit W1 is shown as follows:

[0088]

[0089] In some optional implementation scenarios, the structural diagram of the summing subunit W1 is as follows: Fig.11 As shown. The summation subunit includes an adder array and a reciprocal module. The adder array is constructed by multiple adders through a tree structure. The adder performs an addition operation on two input values ​​and outputs the sum of the two. The reciprocal module calculates the reciprocal of the input value and outputs its exponent term and mantissa term respectively.

[0090] The multiplication subunit W2 calculates the mantissa term s of the intermediate value and the normalized mantissa term d″ i The product of is normalized and aligned with the exponent term of the intermediate value to obtain a target mantissa term and a target exponent term, the target mantissa term is the mantissa term of the second block of floating-point numbers, and the target exponent term is the second exponent term shared by the second block of floating-point numbers.

[0091] In some optional implementation scenarios, the structural diagram of the multiplication subunit W2 is as follows: Fig.12 The multiplication subunit W2 includes a multiplier and a normalization subunit.

[0092] The multiplier U calculates the product of the mantissa term of the intermediate value and the normalized mantissa term to obtain the third mantissa term d″′ i =(d″ i ·s).

[0093] The normalization subunit converts the third mantissa term d″′ i Normalize and align with the exponential term p″ of the intermediate value to obtain the target mantissa term d″′ i and the target index term p″′.

[0094] Target mantissa d″′ i The target exponent term p″′ is the mantissa term m of the block floating point number output by the calculation circuit. i and the exponential term q.

[0095] The function implemented by the calculation circuit of the embodiment of the present application is to perform normalized exponential calculation on the block floating point number, and the calculation formula is as follows:

[0096]

[0097] In the above calculation formula, the common factor between the numerator and the denominator is 2 p′ is eliminated and the hardware circuit is simplified.

[0098] Fig.13 FIG. 2 shows a data flow diagram of a computing circuit according to an embodiment of the present application. Fig.13 As shown, the calculation circuit includes: a maximum value extraction subunit Q1, a vector subtraction subunit Q2, a vector index calculation subunit Q3, a shared index extraction subunit U1, a normalization subunit U2, a summation subunit W1 and a multiplication subunit W2.

[0099] Input the block floating point number z to the calculation circuit i =2 p ·d i (i=0,1,2,…15), where p, d i All are fixed-point values, p is the floating-point number z of each block i The shared index term is used as the first index term. i Floating point number z for each block i The last digit of .

[0100] The maximum value extraction subunit Q1 compares the floating point numbers z of each block i The last digit d i , output the maximum value of the mantissa d max (like Fig.13 process (1)).

[0101] The vector subtraction subunit Q2 calculates the floating point numbers z of each block respectively. i The last digit d i and the maximum value of the mantissa d max The difference is taken as the second mantissa d′ i =d i -d max (like Fig.13 process (2)).

[0102] The vector index calculation subunit Q3 is based on the floating point number z of each block i The shared exponent term p and the second mantissa term d′ i Construct multiple blocks of floating point numbers z i ′=2 p ·d i ′, block floating point number z i ′=2 p ·d i 'Perform exponential operation to obtain multiple floating point numbers (like Fig.13 process (3)).

[0103] The shared exponent extraction subunit U1 receives multiple floating point numbers x i , get the floating point number x i The largest exponential term among the exponential terms of the , the largest exponential term is used as the third exponential term p' (such as Fig.13 process (4)).

[0104] Normalization subunit U2 converts multiple floating point numbers x into i Normalized to multiple normalized mantissa terms d″ i (like Fig.13 process (5)).

[0105] The summation subunit W1 calculates the normalized mantissa term d″ i The reciprocal of the sum of the normalized mantissa terms is taken as the intermediate value, and then the exponential term p" and the mantissa term s (such as Fig.13 process (6)).

[0106] The multiplication subunit W2 calculates the mantissa term s of the intermediate value and the normalized mantissa term d″ i The product of the product and the exponential term of the intermediate value are normalized and aligned to obtain the target mantissa term and the target exponential term, the target mantissa term d″′ i is the mantissa term of the second block of floating-point numbers, and the target exponent term p″′ is the second exponent term shared by the second block of floating-point numbers (such as Fig.13 process (7)).

[0107] Target mantissa d″′ i The target exponent term p″′ is the mantissa term and exponent term of the block floating point number output by the calculation circuit.

[0108] The computing circuit of the embodiment of the present application uses block floating-point numbers as input and output, which ensures data accuracy and validity, and is easily compatible with other block floating-point operators in the artificial intelligence model algorithm (such as matrix multiplication, etc.), without the need for numerical conversion. At the same time, each subunit uses fixed-point format for calculation, which is fast, has low overhead, and has low storage and bandwidth requirements. Moreover, the computing circuit supports parallel computing based on the characteristics of block floating-point numbers sharing exponents, which can effectively improve computing processing capabilities.

[0109] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A computing circuit, characterized in that: The calculation circuit receives a plurality of first blocks of floating-point numbers, calculates the plurality of first blocks of floating-point numbers, and obtains a plurality of second blocks of floating-point numbers; the plurality of first blocks of floating-point numbers share a first exponent term, and the plurality of second blocks of floating-point numbers share a second exponent term; The computing circuit comprises: a first computing unit, a conversion unit, and a second computing unit; the first computing unit is coupled to the conversion unit, and the conversion unit is coupled to the second computing unit; The first calculation unit receives the plurality of first blocks of floating-point numbers, calculates the mantissa items of the plurality of first blocks of floating-point numbers, and obtains a plurality of first intermediate floating-point numbers; the first calculation unit includes a maximum value extraction subunit, a vector subtraction subunit, and a vector exponent calculation subunit; the maximum value extraction subunit receives the plurality of first blocks of floating-point numbers, obtains the largest mantissa item among the mantissa items of the plurality of first blocks of floating-point numbers, and uses the largest mantissa item as the first mantissa item; the vector subtraction subunit calculates the difference between the mantissa items of each of the first blocks of floating-point numbers and the first mantissa item, respectively, and uses the difference as the second mantissa item; the vector exponent calculation subunit constructs a plurality of third blocks of floating-point numbers based on the first exponential item and the second mantissa item, performs exponential operation on the third block of floating-point numbers, and obtains the first intermediate floating-point numbers; The conversion unit receives the plurality of first intermediate floating-point numbers, and normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items; the conversion unit comprises a shared exponent extraction subunit and a normalization subunit; the shared exponent extraction subunit receives the plurality of first intermediate floating-point numbers, obtains the largest exponent item among the exponent items of the first intermediate floating-point numbers, and uses the largest exponent item as the third exponent item; the normalization subunit normalizes the plurality of first intermediate floating-point numbers into a plurality of normalized mantissa items according to the third exponent item; The second calculation unit calculates the multiple normalized mantissa items to obtain multiple second intermediate floating-point numbers, and normalizes the multiple second intermediate floating-point numbers into the multiple second block floating-point numbers based on the normalized mantissa items.

2. The calculation circuit according to claim 1, characterized in that: The maximum value extraction subunit includes a comparator array, and the comparator array compares the mantissa items of the plurality of first-block floating-point numbers to obtain the largest mantissa item among the mantissa items of the plurality of first-block floating-point numbers.

3. The calculation circuit according to claim 1, characterized in that: The vector subtraction subunit comprises a subtractor array, and the subtractor array calculates the difference between the mantissa items of each of the first block floating point numbers and the first mantissa item.

4. The calculation circuit according to claim 1, characterized in that: The vector index calculation subunit includes an index calculation module; The exponential calculation module concatenates the first exponential term and the second mantissa term, and searches for an exponential function value corresponding to the concatenation result by a table lookup method, wherein the exponential function value is the first intermediate floating-point number; Alternatively, the exponential calculation module obtains the first intermediate floating-point number by a fitting method.

5. The calculation circuit according to claim 1, characterized in that: The normalization subunit includes a plurality of basic circuits, wherein the basic circuits include an exponent extractor, a mantissa extractor, a subtractor, and a shift circuit; The exponent extractor obtains the exponent term of the first intermediate floating-point number; The mantissa extractor obtains the mantissa term of the first intermediate floating-point number; The subtractor obtains a difference between the exponential term of the first intermediate floating-point number and the third exponential term; The shift circuit shifts and aligns the mantissa term of the first intermediate floating-point number according to the difference between the exponent term of the first intermediate floating-point number and the third exponent term to obtain the normalized mantissa term.

6. The calculation circuit according to claim 1, characterized in that: The second calculation unit includes a summation subunit and a multiplication subunit; The summing subunit calculates the sum of the normalized mantissa items, and takes the reciprocal of the sum of the normalized mantissa items as an intermediate value; The multiplication subunit calculates the product of the mantissa term of the intermediate value and the normalized mantissa term, normalizes and aligns the product with the exponent term of the intermediate value, and obtains a target mantissa term and a target exponent term, wherein the target mantissa term is the mantissa term of the second block of floating-point numbers, and the target exponent term is a second exponent term shared by the second block of floating-point numbers.

7. The calculation circuit according to claim 6, characterized in that: The multiplication subunit includes a multiplier and the normalization subunit; The multiplier calculates the product of the mantissa term of the intermediate value and the normalized mantissa term to obtain a third mantissa term; The normalization subunit normalizes and aligns the third mantissa term with the exponent term of the intermediate value to obtain a target mantissa term and a target exponent term.

8. An artificial intelligence accelerator, characterized in that: The artificial intelligence accelerator comprises a computing circuit as described in any one of claims 1-7.

Images