Artificial Intelligence Computing Circuit

Abstract

The present invention provides an artificial intelligence operation circuit. The artificial intelligence operation circuit can achieve multiple operations in integer format and floating point format by adjusting the circuit configuration. Integer multiplication and floating point mantissa multiplication share a multiplication operation unit, integer comparison and floating point comparison share a comparison operation unit, and integer addition and floating point addition share an addition operation unit.

Description

Technical Field

[0001] The invention relates to a mixed-precision artificial intelligence computing circuit. Background Art

[0002] Processors used to perform artificial intelligence operations often use one of the data formats of Int8, BF16, and TF32. In terms of calculation accuracy, TF32 is the highest, BF16 is the second highest, and Int8 is the lowest. In terms of calculation speed (or computing power), Int8 is the highest, BF16 is the second highest, and TF32 is the lowest. In other words, it is difficult for artificial intelligence operation circuits using a single data format to balance calculation accuracy and calculation speed. Summary of the invention

[0003] An embodiment of the present invention provides an artificial intelligence operation circuit. The artificial intelligence operation circuit includes a plurality of operation circuits. Each operation circuit includes a floating-point multiplication module, a floating-point addition module and a storage module. The floating-point multiplication module includes a first selection unit, a second selection unit and a multiplication operation unit. A first input end of the first selection unit is used to receive a first numerical data, a second input end of the first selection unit is used to receive a plurality of mantissa bits of the first numerical data, a first input end of the second selection unit is used to receive a second numerical data, a second input end of the second selection unit is used to receive a plurality of mantissa bits of the second numerical data, and the multiplication operation unit is coupled to the first selection unit and the second selection unit. The floating-point addition module includes a comparison operation unit, a third selection unit, a fourth selection unit and an addition operation unit. The comparison operation unit is used to receive the first numerical data and the second numerical data, a first input end of the third selection unit is used to receive the first numerical data, a second input end of the third selection unit is used to receive the mantissa bits of the first numerical data, a first input end of the fourth selection unit is used to receive the second numerical data, a second input end of the fourth selection unit is used to receive the mantissa bits of the second numerical data, and the addition operation unit is coupled to the first selection unit and the second selection unit. The storage module is coupled to the floating-point multiplication module and the floating-point addition module. When the first numerical data is in an integer format, the first selection unit, the second selection unit, the third selection unit and the fourth selection unit select the first input ends as outputs, and when the first numerical data is in a floating-point format, the first selection unit, the second selection unit, the third selection unit and the fourth selection unit select the second input ends as outputs.

[0004] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 FIG. 4 is a block diagram of a mixed-precision artificial intelligence computing circuit according to an embodiment of the present invention.

[0006] Figure 2 FIG. 4 is a circuit block diagram of an artificial intelligence operation circuit according to an embodiment of the present invention.

[0007] Figure 3 FIG. 4 is a circuit block diagram of an artificial intelligence operation circuit according to another embodiment of the present invention.

[0008] Reference numerals

[0009] 10, 20, 30: Artificial intelligence computing circuit

[0010] M1: floating point multiplication module

[0011] M2: Floating point addition module

[0012] M3: Storage Module

[0013] OPU1: Multiplication Unit

[0014] OPU2: Comparison Unit

[0015] OPU3: Addition Unit

[0016] SLU1~SLU4, 201~212: Select unit

[0017] 251~263: Operation unit DETAILED DESCRIPTION

[0018] The structural principle and working principle of the present invention are described in detail below in conjunction with the accompanying drawings:

[0019] In order to more clearly understand the technical content of this case, the numerical format of binary operations is first explained. In Int8, the format of the numerical value is a signed integer. The eighth bit from the right in the format is the sign bit representing positive or negative, and the first to seventh bits represent the value, so Int8 can represent integers from -128 to 127. Binary floating-point numbers can be expressed as [(+ / -)a×b^c], where (+ / -) is the positive and negative sign representing positive or negative values, × is a multiplication operation, a is the mantissa, b is the base (i.e. 2), and c is the exponent. In BF16, the sixteenth bit from the right (the most significant bit) is the sign bit representing positive or negative, the eighth to fifteenth bits are the exponent bits, and the first to seventh bits are the mantissa bits, for a total of sixteen bits. In TF32, the nineteenth bit from the right (the most significant bit) is the sign bit representing positive or negative, the eleventh to eighteenth bits are the exponent bits, and the first to tenth bits are the mantissa bits, for a total of nineteen bits.

[0020] Please refer to Figure 1 , Figure 1 A block diagram of a mixed-precision artificial intelligence operation circuit according to an embodiment of the present invention is shown. The artificial intelligence operation circuit 10 can be used to perform various mathematical operations, such as addition, subtraction, multiplication, and maximum / minimum value, etc., according to a first numerical data D1 and a second numerical data D2. In this embodiment, the format of the first numerical data and the second numerical data can be an integer format, such as Int8, or a floating point format, such as BF16, TF32.

[0021] The artificial intelligence operation circuit 10 includes a floating point multiplication module M1, a floating point addition module M2 and one or more storage modules M3. Generally speaking, an artificial intelligence processor may include multiple artificial intelligence operation circuits 10 to cope with the huge amount of artificial intelligence operations.

[0022] The floating point multiplication module M1 includes a multiplication operation unit OPU1 and two selection units SLU1 and SLU2. A first input terminal of the selection unit SLU1 is used to receive the first numerical data D1, and a second input terminal of the selection unit SLU1 is used to receive the mantissa bit D1_M (hereinafter represented as [1^b, D1_M]) of the first numerical data with one bit "1" added as the most significant bit. A first input terminal of the selection unit SLU2 is used to receive the second numerical data D2, and a second input terminal of the selection unit SLU2 is used to receive the mantissa bit D2_M (hereinafter represented as [1^b, D2_M]) of the second numerical data with one bit "1" added as the most significant bit. It should be noted that in the floating point format, the integer part of the mantissa of the normalized floating point number is "1". In order to save one bit of space, the mantissa bit usually only includes the decimal part after the decimal point, but does not include the "1" before the decimal point, and before performing the operation, the leftmost part of the mantissa bit is padded with "1" as the most significant bit.

[0023] When the first numerical data and the second numerical data are in integer format, the selection units SLU1 and SLU2 will be controlled by a control signal output by a control unit to select the first input end for output; when the first numerical data and the second numerical data are in floating point format, the selection units SLU1 and SLU2 will be controlled by a control signal output by a control unit to select the second input end for output. In detail, when the first numerical data and the second numerical data are in integer format, the second input end of the selection units SLU1 and SLU2 will receive the first numerical data and the second numerical data after the mantissa processing, but because the first numerical data and the second numerical data are in integer format, the first input end of the selection units SLU1 and SLU2 will receive the first numerical data and the second numerical data after the mantissa processing will be erroneous data. However, these errors received by the first input end of the selection units SLU1 and SLU2 will not be selected because the control unit controls the second input end of the selection units SLU1 and SLU2 to output, and vice versa. The multiplication unit OPU1 is used to receive the outputs of the selection units SLU1 and SLU2, and perform multiplication operations according to the outputs of the selection units SLU1 and SLU2. That is, when the first numerical data and the second numerical data are in integer format, the multiplication unit OPU1 performs a multiplication operation based on the first numerical data and the second numerical data, and obtains the product of the first numerical data and the second numerical data; when the first numerical data and the second numerical data are in floating point format, the multiplication unit OPU1 performs a multiplication operation based on [1^b, the mantissa bit of the first numerical data] and [1^b, the mantissa bit of the second numerical data], and obtains the product of [1^b, the mantissa bit of the first numerical data] and [1^b, the mantissa bit of the second numerical data], that is, a part of the floating point operation, mantissa multiplication.

[0024] Furthermore, the floating-point multiplication module M1 further includes an addition operation unit (not shown) for calculating the addition of the exponent bits of the first numerical data and the second numerical data when the first numerical data and the second numerical data are in floating-point format, and an exclusive OR (XOR) operation unit (not shown) for performing sign processing according to the sign bits of the first numerical data and the second numerical data. The outputs of these two operation units will not be selected when the first numerical data and the second numerical data are in integer format (the operation result is incorrect). The product of the first numerical data and the second numerical data can be obtained by integrating the output of the multiplication operation unit OPU1, the sum of the exponent bits of the first numerical data and the second numerical data, and the result of the sign processing.

[0025] The floating point addition module M2 includes a comparison operation unit OPU2, an addition operation unit OPU3 and a plurality of selection units SLU3 and SLU4. The comparison operation unit OPU2 can be used to receive the first numerical data and the second numerical data, and compare the two to determine the size relationship between the two. A first input end of the selection unit SLU3 is used to receive the first numerical data, and a second input end of the selection unit SLU3 is used to receive the larger of the first numerical data and the second numerical data. A first input end of the selection unit SLU4 is used to receive the second numerical data, and a second input end of the selection unit SLU4 is used to receive the smaller of the first numerical data and the second numerical data. When the first numerical data and the second numerical data are in integer format, the selection units SLU3 and SLU4 are controlled by the control signal of the control unit to select the first input end for output; when the first numerical data and the second numerical data are in floating point format, the selection units SLU3 and SLU4 are controlled by the control signal of the control unit to select the second input end for output. The addition operation unit OPU3 performs addition operation according to the output of the selection units SLU3 and SLU4. In detail, since floating-point operations require alignment of decimal points, before performing addition operations, it is necessary to compare the size relationship between the first numerical data and the second numerical data, while integer operations do not require this. That is, when the first numerical data and the second numerical data are in integer format, the addition unit OPU3 performs an addition operation based on the first numerical data and the second numerical data, and obtains the sum of the first numerical data and the second numerical data; when the first numerical data and the second numerical data are in floating-point format, the addition unit OPU3 performs an addition operation based on the first numerical data and the second numerical data, and obtains a preliminary operation result of adding the first numerical data to the second numerical data.

[0026] Furthermore, the floating-point addition operation module M2 further includes a prediction operation unit (not shown) and an error correction unit (not shown). In one embodiment, the prediction operation unit uses Designware LZA (leading zero anticipator) to predict the position of the first 0 after the addition is performed. When the first numerical data and the second numerical data are in floating-point format, the prediction operation unit will synchronously calculate a sum prediction value based on the first numerical data and the second numerical data with the addition operation unit OPU3, and the error correction unit will compare the preliminary operation result output by the addition operation unit OPU3 with the sum prediction value to determine whether to correct the preliminary operation result. This correction is, for example, to shift the preliminary operation result to the left by one bit. In other words, if no correction is required, the preliminary operation result output by the addition operation unit OPU3 is the sum of the first numerical data and the second numerical data (floating point number); if correction is required, the corrected preliminary operation result is the sum of the first numerical data and the second numerical data (floating point number).

[0027] The storage module M3 may include a plurality of trigger units (not shown). The storage module M3 may be used to store various calculation results according to the first numerical data and the second numerical data.

[0028] The control unit may be included in the artificial intelligence operation circuit 10 or may be external to the artificial intelligence operation circuit 10 , which is not limited in the present invention.

[0029] In one embodiment, the artificial intelligence operation circuit 10 can further perform product term sum operation. For example, in the first round of operation, the first numerical data and the second numerical data are input to the floating-point multiplication module M1, and the calculated first result is stored in the storage module M3. In the second round of operation, the new first numerical data and the new second numerical data are input to the floating-point multiplication module M1, and the calculated second result and the first result stored in the storage module M3 are input to the floating-point addition module M2, and the calculated third result is stored in the storage module M3, and so on. In this way, the artificial intelligence operation circuit 10 can sum up the products of multiple groups of first numerical data and second numerical data to obtain the product term sum. It is worth mentioning that since the precision of the multiplication operation is input intact and completely for accumulation, only a carry processing error (rounding error) will be performed after the last accumulation. Compared with the method of separating multiplication and addition independently, a carry processing error is performed after the multiplication operation, and a carry processing error is also performed after the addition operation, a total of two carry processing errors, and the method of this embodiment has a smaller final error.

[0030] Artificial intelligence processors require a higher computing speed. In practice, an artificial intelligence processor usually includes a large number of artificial intelligence computing circuits 10, such as tens of thousands. The multiplication unit, addition unit, and comparison unit are basic, critical, and occupy a large area in integer operations and floating-point operations. By selecting the input of the selection unit, integer operations and floating-point operations can share the same multiplication unit, addition unit, and comparison unit, which can save a lot of area.

[0031] Please refer to Figure 2 , Figure 2 FIG. 2 is a block diagram of an artificial intelligence computing circuit according to an embodiment of the present invention. The artificial intelligence computing circuit 20 shows further details of the artificial intelligence computing circuit 10, and Figure 2 The artificial intelligence operation circuit 20 is configured to perform a product term and operation. For simplicity of description, Figure 2Some selection units are omitted. This embodiment is described with the integer format being Int8 and the floating point format being BF16, that is, the artificial intelligence operation circuit 20 is an operation circuit with mixed precision of Int8 and BF16. Figure 2 The text prompts in the text are explained. <x>It means that the total number of bits transmitted by this path is X bits, for example <8> Indicates that the path transmits a total of eight bits. [Y] means taking the Yth bit of the data transmitted by the path, for example,

[15] means taking the 15th bit of the data transmitted by the path (the rightmost bit is the 0th bit, i.e. the least significant bit). [A:B] means taking (A-B+1) bits from the Bth bit to the Ath bit of the data transmitted by the path. For example, [14:0] means taking fifteen bits from the 0th bit to the 14th bit from the right of the data transmitted by the path. Int8 or BF16 marked on the path means that the path will be selected in the corresponding format. For example, a path marked with Int8 means that the path will be selected when the format of the first numerical data and the second numerical data is an integer format (Int8), and a path marked with BF16 means that the path will be selected when the format of the first numerical data and the second numerical data is a floating point format (BF16).

[0032] The artificial intelligence operation circuit 20 includes a floating point multiplication module M1, a floating point addition module M2 and a storage module M3. The floating point multiplication module M1 includes a plurality of selection units 201-205 and a plurality of operation units 251-255. The selection units 201 and 202 are equivalent to Figure 1 The selection units SLU1 and SLU2 in FIG. 251 are eight-bit multipliers, which are equivalent to Figure 1 The multiplication unit OPU1. The operation unit 251 is coupled to the selection units 201 and 202. The operation unit 252 is coupled to the operation unit 251. The operation unit 252 is used to normalize the floating point number and output it. As mentioned above, in the floating point operation, the normalized floating point representation is 1.xxx, that is, the integer part before the decimal point is 1, and the output of the operation unit 251 may make the integer part 11 or 10 due to the carry in the multiplication operation, for example, 10.xxx or 11.xxx. At this time, normalization is to shift the decimal point one bit to the left to become 1.0xx or 1.1xxx, so as to meet the normalized floating point representation. Therefore, the operation unit 252 takes out the 0th bit to the 14th bit of the output of the operation unit 251, a total of fifteen bits, and fills a 0 as the most significant bit to become a total of sixteen bits before outputting. The selection unit 203 is coupled to the operation units 201 and 202. The selection unit 203 is controlled by the control signal. When the first numerical data and the second numerical data are in integer format, the output of the operation unit 201 is selected as the output. When the first numerical data and the second numerical data are in floating point format, the output of the operation unit 202 is selected as the output. The selection units 201-203 and the operation units 251-252 can realize the multiplication operation in integer format and the mantissa multiplication part in the multiplication operation in floating point format.

[0033] The operation unit 253 is an adder, which receives the exponent bit D1_E of the first numerical data, the exponent bit D2_E of the second numerical data, and the 15th bit (the carry in of the mantissa multiplication) in the output of the operation unit 251, and performs an addition operation accordingly. The operation unit 254 is a subtractor, which is coupled to the operation unit 253. The operation unit 254 is used to subtract 127 (decimal) from the output of the operation unit 253 to comply with the IEEE 754 standard established by the Institute of Electrical and Electronics Engineers (IEEE). The selection unit 204 is coupled to the operation unit 254. The selection unit 204 is controlled by the control signal, and does not output when the first numerical data and the second numerical data are in integer format, and outputs the output of the operation unit 254 when the first numerical data and the second numerical data are in floating point format. The exponent addition in the floating point format multiplication operation can be implemented by the selection unit 204 and the operation units 253 and 254 .

[0034] The operation unit 255 is an exclusive OR gate (XOR) for receiving the sign bit D1_S of the first numerical data and the sign bit D2_S of the second numerical data, and performing an exclusive OR operation accordingly. The selection unit 205 is coupled to the operation unit 255. The selection unit 205 is controlled by a control signal, and does not output when the first numerical data and the second numerical data are in integer format, and outputs the output of the operation unit 255 when the first numerical data and the second numerical data are in floating point format. The selection unit 205 and the operation unit 255 can realize the sign multiplication in the floating point format multiplication operation.

[0035] When the first numerical data and the second numerical data are in integer format, the output of the selection unit 203 (the product of the first numerical data and the second numerical data) will be input and stored in the storage module M3. When the first numerical data and the second numerical data are in floating point format, the output of the selection units 203-205 (the result of multiplying the mantissa of the first numerical data and the second numerical data, adding the exponents and multiplying the signs) will be input and stored in the storage module M3. When storing, the outputs of the selection units 203-205 will be arranged in an order that conforms to the floating point format. It is worth mentioning that the product in the floating point format obtained in this way can maintain the accuracy of the original data.

[0036] The floating point addition module M2 includes a plurality of selection units 206 ˜ 211 and a plurality of operation units 256 ˜ 263 .

[0037] The selection unit 206 is coupled to the storage module M3. The selection unit 206 can receive the output of the selection unit 203 stored in the storage module M3. The operation unit 256 is equivalent to Figure 1 The operation unit OPU2 in the storage module M3. The operation unit 256 is coupled to the selection unit 206 and the storage module M3. When the first numerical data and the second numerical data are in floating point format, the operation unit 256 compares the part of the product of the first numerical data and the second numerical data excluding the sign bit (hereinafter referred to as V1) with the part of a product term and value excluding the sign bit stored in the storage module M3 (hereinafter referred to as V2) to obtain the size relationship between V1 and V2. When V1 is greater than V2, the operation unit 256 outputs 1, and if V1 is not greater than V2, the operation unit 256 outputs 0. In addition, the operation unit 256 can also obtain the size relationship between the exponent in the product of the first numerical data and the second numerical data and the exponent of the product term and value in the process of comparing V1 and V2, and output it to the operation unit 257. The selection unit 207 is coupled to the storage module M3. When the first numerical data and the second numerical data are in integer format, the selection unit 207 is used to receive the product item and value from the storage module M3. When the first numerical data and the second numerical data are in floating point format, the selection unit 207 is used to receive the mantissa bits of the product item and value from the storage module M3. The selection unit 208 is coupled to the selection units 206, 207 and the operation unit 256. The selection unit 209 is coupled to the selection units 206, 207 and the operation unit 256. When the first numerical data and the second numerical data are in floating point format, the selection units 208 and 209 receive the outputs from the selection units 206 and 207 and are controlled by the output of the operation unit 256. When the operation unit 256 outputs 1, the selection unit 208 selects the output of the selection unit 206 for output, and the selection unit 209 selects the output of the selection unit 207 for output. When the operation unit 256 outputs 0, the selection unit 208 selects the output of the selection unit 207 for output, and the selection unit 209 selects the output of the selection unit 206 for output. The operation unit 257 is coupled to the operation unit 256. The operation unit 257 determines whether to align the decimal points of V1 and V2 according to the relationship between the exponent in the product of the first numerical data and the second numerical data and the exponent of the sum of the product terms output by the operation unit 256, and transmits the decision to the operation unit 258. The operation unit 258 is coupled to the operation unit 257. The operation unit 258 shifts the smaller of V1 and V2 to the right by one or more bits according to the decision output by the operation unit 257, when the decision output by the operation unit 257 indicates that the decimal points of V1 and V2 are to be aligned, and the number of bits shifted is determined according to the difference in the exponent bits of the two. The operation unit 259 is coupled to the storage module M3. The operation unit 259 is an XOR gate, which is used to perform an XOR operation according to the sign bit of the product of the first numerical data and the second numerical data and the sign bit of the sum of the product terms, that is, to determine whether the two have the same sign. The operation unit 260 is coupled to the operation unit 259.The operation unit 260 determines whether to perform two's complement coding on the smaller of V1 and V2 (i.e., reverse and add 1) according to the result of the XOR operation output by the operation unit 259. When the result of the XOR operation output by the operation unit 259 is 1 (i.e., the product of the first numerical data and the second numerical data and the product term sum value have different signs, one positive and one negative), the operation unit 260 performs two's complement coding on the smaller of V1 and V2. When the result of the XOR operation output by the operation unit 259 is 0 (i.e., the product of the first numerical data and the second numerical data and the product term sum value have the same sign, both positive or both negative), the operation unit 260 does not act, i.e., does not perform two's complement coding on the smaller of V1 and V2. The operation unit 261 is an adder, equivalent to. Figure 1 In addition, due to the operation unit OPU3 Figure 2 The product terms and operations are plotted, so Figure 1 The selection units SLU3 and SLU4 in the operation unit 260 are omitted. The operation unit 261 is an adder, coupled to the selection units 206-208, the operation unit 259 (as the carry value of the operation unit 261) and the operation unit 260. When the first numerical data and the second numerical data are in integer format, the two inputs of the operation unit 261 are from the selection units 206 and 207. When the first numerical data and the second numerical data are in floating point format, the two inputs of the operation unit 261 are from the selection unit 208 and the operation unit 260. The operation unit 262 is coupled to the selection unit 208 and the operation units 259-261. The operation unit 262 is a floating point addition acceleration circuit. The operation unit 262 performs operations simultaneously with the operation unit 261 according to the outputs of the selection unit 208 and the operation units 259 and 260. The operation unit 262 (for example, using Designware LZA as described above) predicts the position of the leading zero after the output of the selection unit 208 and the operation unit 260 are subtracted to obtain a prediction result, and transmits the prediction result to the operation unit 261 to reduce the critical path of floating point addition and subtraction. The input of the operation unit 261 is simultaneously input to the operation unit 262, and the operation unit 262 compares the prediction result with the output of the operation unit 261 (the result of the actual addition operation) to determine whether the prediction result is correct, and outputs a correction indication and a modified exponent bit. The operation unit 263 is coupled to the operation units 261 and 262. The operation unit 263 determines whether to calibrate the output of the operation unit 261 according to the correction indication from the operation unit 262. When the correction indication indicates that the correction is to be performed, the operation unit 263 shifts the result output by the operation unit 261 to the left by one bit. It should be noted that the operation unit 263 takes the least significant twenty-three bits of the output of the operation unit 261 for output. The selection unit 210 is coupled to the operation units 261, 263 and the storage module M3, and is controlled by a control signal. When the first numerical data and the second numerical data are in integer format, the output of the operation unit 261 is selected to be output to the storage module M3, and when the first numerical data and the second numerical data are in floating point format, the output of the operation unit 263 is selected to be output to the selection module M3. The selection unit 211 is coupled to the operation unit 259 and the storage module M3, and the selection unit 212 is coupled to the operation unit 262 and the storage module M3. The selection units 211 and 212 are controlled by a control signal. When the first numerical data and the second numerical data are in floating point format, the output of the operation unit 259 and the exponent bit corrected by the operation unit 262 are output to the storage module M3. In this way, the sum of the product of the first numerical data and the second numerical data and the sum of the product terms previously accumulated can be obtained.

[0038] In this embodiment, the operations that can actually be performed include addition / subtraction / maximum / minimum in FP32 format, multiplication / sum of products in BF16 format, addition / subtraction / multiplication / maximum / minimum / sum of products in Int8 format, and addition / subtraction / maximum / minimum in Int16 format. In addition, in another embodiment, the operation unit 261 is set as a thirty-two-bit adder, and the addition / subtraction / maximum / minimum in Int16 format supported by the above embodiment can be further supported to addition / subtraction / maximum / minimum in Int24 or even Int32 format. In other words, all numerical data formats within the upper limit of the number of bits allowed for this type of operation can be supported.

[0039] Compared with the traditional Synopsys Designware design, the area of ​​the traditional Synopsys Designware design is 100%, while the present embodiment is only 56%, saving 44% area; the operation speed of the traditional Synopsys Designware design is 91%, while the present embodiment is 100%. In addition, in terms of operation types, the traditional Synopsys Designware design can only perform addition in FP32 format and multiplication in Int8 format.

[0040] Please refer to Figure 3 , Figure 3 The circuit block diagram of the artificial intelligence operation circuit according to another embodiment of the present invention is shown. The artificial intelligence operation circuit 30 is similar to the artificial intelligence operation circuit 20, except that the floating point format of the artificial intelligence operation circuit 30 supports TF32, so adaptive modifications are made to the bit number of some paths and operation units.

[0041] In this embodiment, the operations that can actually be performed include addition / subtraction / maximum / minimum in FP32 format, multiplication / sum of products in TF32 format, multiplication / sum of products in BF16 format, multiplication / sum of products in Int16 format, addition / subtraction / multiplication / maximum / minimum / sum of products in Int8 format, and addition / subtraction / maximum / minimum in Int16 (or Int24, or Int32) format. In other words, all numerical data formats within the upper limit of the number of bits allowed for this type of operation can be supported.

[0042] Compared with the traditional Synopsys Designware design, the area of ​​the traditional Synopsys Designware design is 100%, while the present embodiment is only 68%, saving 32% area; the operation speed of the traditional Synopsys Designware design is 91%, while the present embodiment is 100%. In addition, in terms of operation types, the traditional Synopsys Designware design can only perform addition in FP32 format and multiplication in Int8 format.

[0043] The artificial intelligence computing circuit proposed in the present invention can be switched to corresponding circuit configurations corresponding to different numerical data formats, and has the advantages of high computing speed, high performance, high accuracy and low area.

[0044] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.< / x>

Claims

1. An artificial intelligence computing circuit, characterized in that: include: A plurality of operation circuits, each of the operation circuits comprising: A floating point multiplication module comprises a first selection unit, a second selection unit and a multiplication unit, wherein a first input terminal of the first selection unit is used to receive a first numerical data, a second input terminal of the first selection unit is used to receive a plurality of mantissa bits of the first numerical data, a first input terminal of the second selection unit is used to receive a second numerical data, a second input terminal of the second selection unit is used to receive a plurality of mantissa bits of the second numerical data, and the multiplication unit is coupled to the first selection unit and the second selection unit; A floating point addition module, comprising a comparison operation unit, a third selection unit, a fourth selection unit and an addition operation unit, wherein the comparison operation unit is used to receive the first numerical data and the second numerical data, a first input end of the third selection unit is used to receive the first numerical data, a second input end of the third selection unit is used to receive the plurality of mantissa bits of the first numerical data, a first input end of the fourth selection unit is used to receive the second numerical data, a second input end of the fourth selection unit is used to receive the plurality of mantissa bits of the second numerical data, and the addition operation unit is coupled to the third selection unit and the fourth selection unit; as well as a storage module coupled to the floating-point multiplication module and the floating-point addition module, When the first numerical data is in an integer format, the first selection unit, the second selection unit, the third selection unit and the fourth selection unit respectively select their corresponding first input terminals as outputs; when the first numerical data is in a floating point format, the first selection unit, the second selection unit, the third selection unit and the fourth selection unit respectively select their corresponding second input terminals as outputs.

2. The artificial intelligence operation circuit according to claim 1, characterized in that: The integer format is Int8, and the floating point format is BF16 or TF32.

3. The artificial intelligence operation circuit according to claim 1, characterized in that: When the first numerical data is in the integer format, the multiplication operation unit performs an integer multiplication operation according to the first numerical data and the second numerical data; when the first numerical data is in the floating point format, the multiplication operation unit performs a floating point multiplication operation according to the multiple mantissa bits of the first numerical data and the multiple mantissa bits of the second numerical data.

4. The artificial intelligence operation circuit according to claim 1, wherein: The comparison operation unit is used to compare the size relationship between the first numerical data and the second numerical data, and when the first numerical data and the second numerical data are in the floating point format, compare the size relationship between the first numerical data not including a sign bit and the second numerical data not including a sign bit.

5. The artificial intelligence operation circuit according to claim 1, wherein: When the first numerical data is in the integer format, the addition operation unit performs an integer addition operation according to the first numerical data and the second numerical data; when the first numerical data is in the floating point format, the addition operation unit performs a floating point addition operation according to the multiple mantissa bits of the first numerical data and the multiple mantissa bits of the second numerical data.

6. The artificial intelligence operation circuit according to claim 1, wherein: The floating point multiplication module further includes an addition operation unit for receiving a plurality of exponent bits of the first numerical data and a plurality of exponent bits of the second numerical data, and performing an addition operation according to the plurality of exponent bits of the first numerical data and the plurality of exponent bits of the second numerical data.

7. The artificial intelligence operation circuit according to claim 1, characterized in that: The floating point multiplication module further includes an XOR operation unit for receiving a sign bit of the first numerical data and a sign bit of the second numerical data, and performing an XOR operation according to the sign bit of the first numerical data and the sign bit of the second numerical data.

8. The artificial intelligence operation circuit according to claim 1, wherein: The floating-point addition module further comprises a fifth selection unit and a sixth selection unit, wherein a first input terminal of the fifth selection unit is used to receive a product of the first numerical data and the second numerical data from the storage module, a second input terminal of the fifth selection unit is used to receive a product term and a value from the storage module, a first input terminal of the sixth selection unit is used to receive a plurality of mantissa bits of the product of the first numerical data and the second numerical data from the storage module, and a second input terminal of the sixth selection unit is used to receive a plurality of mantissa bits of the product term and a value from the storage module; When the first numerical data and the second numerical data are in the integer format and the addition operation circuit is configured to perform a product-sum operation, the fifth selection unit and the sixth selection unit are controlled by a control unit to respectively select the input of the corresponding first input terminal as output to the addition operation unit; when the first numerical data and the second numerical data are in the floating-point format and the addition operation circuit is configured to perform a product-sum operation, the fifth selection unit and the sixth selection unit are controlled by the control unit to respectively select the input of the corresponding second input terminal as output to the addition operation unit.

9. The artificial intelligence operation circuit according to claim 8, characterized in that: When the first numerical data and the second numerical data are in the integer format and the addition operation circuit is configured to perform a product term sum operation, the addition operation unit performs an integer addition operation according to the product of the first numerical data and the second numerical data and the product term sum value; when the first numerical data and the second numerical data are in the floating point format and the addition operation circuit is configured to perform a product term sum operation, the addition operation unit performs a floating point addition operation according to the multiple mantissa bits of the product of the first numerical data and the second numerical data and the multiple mantissa bits of the product term sum value.

Images