Computing apparatus, method and integrated circuit chip for vector inner product

Abstract

This invention relates to a computing device, method, and integrated circuit chip for vector inner product calculations, wherein the computing device may be included in a combined processing device, which may further include a general interconnect interface and other processing devices. The computing device interacts with the other processing devices to jointly perform user-specified computational operations. The combined processing device may also include a storage device connected to both the computing device and the other processing devices for data storage of the computing device and the other processing devices.

Description

Technical Field

[0001] This disclosure generally relates to the field of floating-point vector inner product operations. More specifically, this disclosure relates to computing devices, methods, integrated circuit chips, and integrated circuit devices for floating-point vector inner product operations. Background Technology

[0002] Vector dot product operations are widely used in computer science. Taking mainstream machine learning algorithms in artificial intelligence, a popular application area, as an example, common algorithms utilize a large number of vector dot product operations. These operations involve numerous multiplication and addition operations, and the arrangement of these multiplication and addition mechanisms or methods directly affects the computation speed. Although existing technologies have achieved significant improvements in execution efficiency, there is still room for improvement in handling floating-point dot products. Therefore, how to obtain a highly efficient and low-cost module to perform floating-point vector dot products has become a problem that needs to be solved in current technology. Summary of the Invention

[0003] To at least partially address the technical problems mentioned in the background art, the present disclosure provides a method, integrated circuit chip, and apparatus for performing floating-point vector inner products.

[0004] In one aspect, this disclosure provides a computing apparatus for performing vector inner product operations, including a multiplication unit and an addition module. The multiplication unit includes one or more floating-point multipliers configured to perform element-wise multiplication on received first and second vectors to obtain a product of corresponding vector elements for each pair, wherein each of the first and second vectors includes one or more of the vector elements. The addition module is configured to perform an addition operation on the product of corresponding vector elements of the first and second vectors to obtain a summation result.

[0005] The aforementioned computing device further includes an update module configured to perform multiple addition operations on a plurality of the intermediate results generated in response to the summation result being an intermediate result of the inner product operation, so as to output the final result of the inner product operation.

[0006] The aforementioned update module includes a second adder and a register. The second adder is configured to repeatedly perform the following operations until the addition operation on all of the plurality of intermediate results is completed: receiving intermediate results from the addition module and a previous summation result from the register; adding the intermediate results and the previous summation result to obtain the summation result of the current addition operation; and updating the previous summation result stored in the register using the result of the current addition operation.

[0007] In another aspect, this disclosure provides a method for performing vector inner product operations using the aforementioned computing device, the steps of which include: using the floating-point multiplier to perform multiplication operations on corresponding vector elements of the first vector and the second vector to obtain the product result of the corresponding vector elements of each pair; and performing an addition operation on the product result of the corresponding vector elements of the first vector and the second vector to obtain a summation result.

[0008] In another aspect, this disclosure provides an integrated circuit chip or integrated circuit device, including the aforementioned computing device. In one or more embodiments, the computing device disclosed herein can be configured as a standalone integrated circuit chip or disposed on an integrated circuit chip, device, or board to perform floating-point vector inner product operations on various data formats.

[0009] By utilizing the computing device, corresponding arithmetic method, integrated circuit chip, and integrated circuit device disclosed herein, floating-point vector inner product operations can be performed more efficiently without requiring excessive hardware expansion, thereby reducing the layout area of ​​the integrated circuit. Attached Figure Description

[0010] The above and other objects, features, and advantages of exemplary embodiments of this disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0011] Figure 1 This is a schematic diagram illustrating a floating-point data format according to an embodiment of this disclosure;

[0012] Figure 2 This is a schematic structural block diagram illustrating a computing device according to an embodiment of the present disclosure;

[0013] Figure 3 This is a schematic structural block diagram illustrating a floating-point multiplier according to an embodiment of this disclosure;

[0014] Figure 4 This is a schematic structural block diagram illustrating further details of a floating-point multiplier according to an embodiment of this disclosure;

[0015] Figure 5 This is a schematic block diagram illustrating a partial product operation unit and a partial product summation unit according to embodiments of the present disclosure;

[0016] Figure 6 This is a schematic diagram illustrating a partial product operation according to an embodiment of this disclosure;

[0017] Figure 7 This is a schematic block diagram illustrating the operation flow and schematic block diagram of the Wallace tree compressor according to an embodiment of this disclosure;

[0018] Figure 8 This is a schematic block diagram showing the overall structure of a floating-point multiplier according to an embodiment of this disclosure;

[0019] Figure 9 This is a flowchart illustrating a method for performing floating-point multiplication using a floating-point multiplier according to an embodiment of this disclosure;

[0020] Figure 10 This is a schematic structural block diagram illustrating a computing device according to another embodiment of the present disclosure;

[0021] Figure 11 This is a schematic structural block diagram illustrating an addition module according to an embodiment of this disclosure;

[0022] Figure 12 This is a schematic structural block diagram illustrating an addition module according to another embodiment of this disclosure;

[0023] Figure 13 This is a flowchart illustrating the operation of the update module according to an embodiment of this disclosure;

[0024] Figure 14 This is a flowchart illustrating a computing device performing a vector inner product operation according to an embodiment of the present disclosure;

[0025] Figure 15 This is a schematic structural block diagram illustrating a combined processing apparatus according to embodiments of the present disclosure; and

[0026] Figure 16 This is a schematic structural block diagram illustrating a board according to an embodiment of this disclosure. Detailed Implementation

[0027] The technical solution disclosed herein provides a method, integrated circuit chip, and device for floating-point vector inner product operations. Unlike existing vector inner product methods, this disclosure offers an efficient calculation scheme that effectively reduces hardware area and supports data of varying widths, making it suitable for a wider range of vector inner product calculation scenarios.

[0028] The vector referred to in this disclosure can be one-dimensional vector data, or one-dimensional data in a high-dimensional data storage format, such as a row or column of a matrix, one-dimensional data of a multidimensional tensor, or scalar data in vector form.

[0029] The technical solutions and multiple embodiments thereof disclosed herein will now be described in detail with reference to the accompanying drawings. It should be understood that many specific details regarding vector inner products will be set forth in order to provide a thorough understanding of the multiple embodiments described herein. However, those skilled in the art, guided by the teachings of this disclosure, can practice the multiple embodiments described herein without these specific details. In other instances, this disclosure does not describe well-known methods, processes, and components in detail to avoid unnecessarily obscuring the embodiments described herein. Furthermore, this description should not be construed as limiting the scope of the multiple embodiments disclosed herein.

[0030] Figure 1 This is a schematic diagram illustrating a floating-point data format 100 according to an embodiment of this disclosure. Figure 1 As shown, the floating-point number to which the present disclosure can be applied may include three parts, such as a sign (or sign bit) 102, an exponent (or exponent bit) 104, and a mantissa (or mantissa bit) 106, wherein for unsigned floating-point numbers, the sign or sign bit 102 may be absent. In some embodiments, the floating-point number applicable to the computing device disclosed herein may include at least one of half-precision floating-point numbers, single-precision floating-point numbers, double-precision floating-point numbers, and custom floating-point numbers. Specifically, in some embodiments, the floating-point number format to which the present disclosure can be applied may be a floating-point format conforming to the IEEE 754 standard, such as a double-precision floating-point number (float64, abbreviated as "FP64"), a single-precision floating-point number (float32, abbreviated as "FP32"), or a half-precision floating-point number (float16, abbreviated as "FP16"). In other embodiments, the floating-point format can be an existing 16-bit floating-point number (bfloat16, abbreviated "BF16"), or a custom floating-point format, such as an 8-bit floating-point number (bfloat8, abbreviated "BF8"), an unsigned half-precision floating-point number (unsigned float16, abbreviated "UFP16"), or an unsigned 16-bit floating-point number (unsigned bfloat16, abbreviated "UBF16"). For ease of understanding, Table 1 below shows some of the above data formats, where the sign bit width, exponent bit width, and mantissa bit width are for illustrative purposes only.

[0031] Table 1

[0032] Data types Sign bit width Exponent width Mantissa width FP16 1 5 10 BF16 1 8 7 FP32 1 8 23 BF8 1 5 3 UFP16 0 5 (or 6) 11 (or 10) UBF16 0 8 8

[0033] For the various floating-point number formats mentioned above, the computing device disclosed herein can support, during operation, at least two floating-point numbers having any of the above formats, wherein the two floating-point numbers may have the same or different floating-point data formats. For example, the multiplication operation between two floating-point numbers can be FP16*FP16, BF16*BF16, FP32*FP32, FP32*BF16, FP16*BF16, FP32*FP16, BF8*BF16, UBF16*UFP16, or UBF16*FP16, etc.

[0034] Figure 2 A schematic structural block diagram of a computing device 200 according to an embodiment of this disclosure is shown. Figure 2 As shown, the computing device 200 includes a multiplication unit 202 and an addition module 204. In one embodiment, the multiplication unit 202 may include a plurality of floating-point multipliers 206, used to perform element-wise multiplication on the received first floating-point vector 208 and second vector 210 to obtain the product result 212 of each pair of corresponding vector elements. In this embodiment, the number of floating-point multipliers 206 can be arranged according to actual conditions, and Figure 2 The three floating-point multipliers 206 shown are for illustrative purposes only and not for limiting purposes. In this embodiment, the first vector 208 and the second vector 210 can be two vectors of the form k*n, where k is an integer multiple of the minimum bit width of the data type, such as 16 or 32, and n is the number of input data, which is a positive integer. Taking k as 32 and n as 16 as an example, the input data bit width is 512 bits. Based on this, the first vector 208 and the second vector 210 can be a data vector containing 16 FP32 data elements, a data vector containing 32 FP16 data elements, or a data vector containing 32 BF16 data elements. In other embodiments, the input bit widths of the first vector 208 and the second vector 210 can be different. For example, the input bit width of the first vector 208 can be 1024 bits, such as 32 FP32, while the second vector 210 can be 512 bits, such as 32 FP16. The number and bit width of the first vector 208 do not have a direct and necessary correspondence with the number and bit width of the second vector 210, and they do not affect each other.

[0035] The addition module 204 can receive the product result 212 output by the multiplication unit 202, perform an addition operation to obtain the inner product result 216, and complete the inner product operation. The addition module 204 can be an adder group formed by multiple adders, which can form a tree structure. For example, the adders include multi-level adder groups arranged in a multi-level tree structure, with each level of adder group including one or more first adders 218. The first adder 218 can be, for example, a floating-point adder. Depending on different application scenarios and implementation methods, the first adder 218 can be implemented as a full adder, a half adder, a ripple carry adder, or a carry-lookahead adder. Furthermore, since the floating-point multiplier 206 disclosed herein is a multiplier that supports multi-mode operations, the adders in the first adder 218 disclosed herein can also be adders that support multiple addition operation modes. For example, when the output of the floating-point multiplier 206 is a data format of half-precision floating-point number, single-precision floating-point number, double-precision floating-point number, or a custom floating-point number, the first adder 218 can also be a floating-point adder that supports any of the above data formats.

[0036] In this embodiment, the floating-point multiplier 206 of the multiplication unit 202 can have multiple operation modes so as to perform multi-mode multiplication operations on the multiple vector elements included in the first vector 208 and the corresponding multiple vector elements included in the second vector 210. Figure 3 This is a schematic block diagram illustrating a floating-point multiplier 206 according to an embodiment of the present disclosure. As previously described, the floating-point multiplier 206 of the present disclosure supports multiplication operations of floating-point vectors in various data formats, which can be indicated by the operation modes of the present disclosure, so that the floating-point multiplier 206 operates in one of multiple operation modes.

[0037] like Figure 3 As shown, the floating-point multiplier 206 disclosed herein generally includes an exponent processing unit 302 and a mantissa processing unit 304, wherein the exponent processing unit 302 is used to process the exponent bits of the floating-point number, and the mantissa processing unit 304 is used to process the mantissa bits of the floating-point number. Optionally or additionally, in some embodiments, when the floating-point number processed by the floating-point multiplier 206 has a sign bit, a sign processing unit 306 may also be included, which can be used to process floating-point numbers including the sign bit.

[0038] In operation, the floating-point multiplier 206 can perform a vector inner product on the received, input, or buffered first vector 208 and second vector 210 according to one of the operation modes, where the corresponding vector elements of the first vector 208 and second vector 210 have one of the floating-point data formats discussed above. For example, when the floating-point multiplier 206 is in the first operation mode, it can support the multiplication of two floating-point numbers FP16*FP16, and when the floating-point multiplier 206 is in the second operation mode, it can support the multiplication of two floating-point numbers BF16*BF16. Similarly, when the floating-point multiplier 206 is in the third operation mode, it can support the multiplication of two floating-point numbers FP32*FP32, and when the floating-point multiplier 206 is in the fourth operation mode, it can support the multiplication of two floating-point numbers FP32*BF16. Here, the example operation modes and floating-point number correspondences are shown in Table 2 below.

[0039] Table 2

[0040]

[0041] In one embodiment, Table 2 described above can be stored in a memory of the floating-point multiplier 206, and the floating-point multiplier 206 selects one of the operation modes in the table according to instructions received from an external device, such as... Figure 16 The external device 1612 is shown in the figure. In another embodiment, the input to this operation mode can also be via, as shown in the figure. Figure 4 The mode selection unit 418 shown automatically implements this. For example, when two FP16 floating-point vectors are input to the floating-point multiplier 206 disclosed herein, the mode selection unit 418 can select the floating-point multiplier 206 to operate in the first operation mode according to the data format of the two floating-point numbers. As another example, when an FP32 floating-point number and a BF16 floating-point number are input to the floating-point multiplier 206 disclosed herein, the mode selection unit 418 can select the floating-point multiplier 206 to operate in the fourth operation mode according to the data format of the two floating-point numbers.

[0042] It can be seen that the different operation modes disclosed herein are associated with corresponding floating-point data. That is, the operation modes disclosed herein can be used to indicate the data format of the vector elements of the first vector 208 and the corresponding data format of the vector elements of the second vector 210. In another embodiment, the operation modes disclosed herein can not only indicate the data format of the corresponding vector elements of the first vector 208 and the second vector 210, but also the data format after a multiplication operation. The extended operation modes, combined with Table 2, are shown in Table 3 below.

[0043] Table 3

[0044]

[0045] Unlike the operation mode numbers shown in Table 2, the operation modes in Table 3 are extended by one bit to indicate the data format after a floating-point vector multiplication operation. For example, when the floating-point multiplier 206 operates in operation mode 21, it performs a vector inner product on two input floating-point numbers of BF16*BF16 and outputs the result of the floating-point multiplication operation in FP16 data format.

[0046] The above-described numbered operation patterns indicating floating-point data formats are merely exemplary and not limiting. Based on the teachings of this disclosure, it is also conceivable to establish indexes based on the operation patterns to determine the formats of the multipliers and multiplicands. For example, an operation pattern may include two indices: the first index indicates the type of vector elements in the first vector 208, and the second index indicates the type of vector elements in the second vector 210. For instance, in operation pattern 13, the first index "1" indicates that the vector elements (or multiplicands) of the first vector 208 are in a first floating-point format, i.e., FP16, while the second index "3" indicates that the vector elements (or multipliers) of the second vector 210 are in a second floating-point format, i.e., FP32. Furthermore, a third index can be added to the operation pattern, indicating the data format of the output result. For example, for operation pattern 131, the third index "1" can indicate that the data format of the output result is a first floating-point format, i.e., FP16. As the number of operation patterns increases, corresponding indices or index levels can be added as needed to facilitate the establishment of the relationship between operation patterns and data formats.

[0047] Furthermore, although numerical designations are used here as an example to represent the operation mode, in other examples, other symbols or codes can be used to represent the operation mode as needed for the application, such as letters, symbols, or numbers and combinations thereof. Such expressions using letters, numbers, symbols, or combinations thereof can then represent the operation mode and identify the vector elements of the first vector 208, the vector elements of the second vector 210, and the data format of the output result. Additionally, when these expressions are formed in the form of instructions, the instructions can include three fields: a first field indicating the data format of the vector elements of the first vector 208, a second field indicating the data format of the vector elements of the second vector 210, and a third field indicating the data format of the output result. Of course, these fields can also be combined into one field, or new fields can be added to indicate more content related to floating-point data formats. It can be seen that the operation mode disclosed herein can not only be associated with the input floating-point data format but can also be used to normalize the output result to obtain a product result with the desired data format.

[0048] Figure 4 This is a more detailed structural block diagram illustrating a floating-point multiplier 206 according to an embodiment of this disclosure. From Figure 4As can be seen from the content shown, it includes not only Figure 3 The exponent processing unit 302, mantissa processing unit 304, and optional sign processing unit 306 shown also illustrate the internal components these units may include and the units related to their operation, which are described below in conjunction with... Figure 4 The exemplary operation of these units will be described in detail below.

[0049] To perform floating-point vector multiplication, the exponent processing unit 302 can be used to obtain the exponent after multiplication based on the aforementioned operation mode, the exponents of the vector elements of the first vector 208, and the exponents of the corresponding vector elements of the second vector 210. In one embodiment, the exponent processing unit 302 can be implemented using an addition and subtraction circuit. For example, the exponent processing unit 302 can be used to add the exponents of the vector elements of the first vector 208, the exponents of the corresponding vector elements of the second vector 210, and the offset values ​​of their respective input floating-point data formats, and then subtract the offset values ​​of the output floating-point data formats to obtain the exponent after multiplication of the vector elements of the first vector 208 and the vector elements of the second vector 210.

[0050] Furthermore, the mantissa processing unit 304 of the floating-point multiplier 206 can be used to obtain the mantissa after multiplication based on the aforementioned operation mode, the vector elements of the first vector 208, and the corresponding vector elements of the second vector 210. In one embodiment, the mantissa processing unit 304 may include a partial product operation unit 402 and a partial product summation unit 404, wherein the partial product operation unit 402 is used to obtain an intermediate result based on the mantissa of the vector elements of the first vector 208 and the mantissa of the corresponding vector elements of the second vector 210. In some embodiments, the intermediate result may be multiple partial products (e.g., ...) obtained during the multiplication operation of the vector elements of the first vector 208 and the corresponding vector elements of the second vector 210. Figure 6 and Figure 7 (Illustrated schematically). The partial product summation unit 404 is used to perform a summation operation on the intermediate results to obtain a summed result, and uses the summed result as the mantissa after the multiplication operation.

[0051] To obtain intermediate results, in one embodiment, this disclosure utilizes a Booth encoding circuit to pad the mantissas of the corresponding vector elements of the second vector 210 (e.g., acting as multipliers in floating-point operations) with leading and trailing zeros (where padding with leading zeros converts the mantissas from unsigned to signed numbers). It should be understood that, depending on the encoding method, the mantissas of the vector elements of the first vector 208 (e.g., acting as multiplicands in floating-point operations) may also be encoded (e.g., padding with leading and trailing zeros), or both, to obtain multiple partial products. Further description of the partial products will be provided later with reference to the accompanying drawings.

[0052] In another embodiment, the partial product summation unit 404 may include an adder for summing the intermediate results to obtain the summed result. In yet another embodiment, the partial product summation unit 404 includes a Wallace tree and an adder, wherein the Wallace tree is used to sum the intermediate results to obtain a second intermediate result, and the adder is used to sum the second intermediate result to obtain the summed result. In these embodiments, the adder may include at least one of a full adder, a serial adder, and a carry-lookahead adder.

[0053] In one embodiment, the mantissa processing unit may further include a control circuit 406, configured to call the mantissa processing unit 304 multiple times according to the operation mode when the operation module indicates that the mantissa bit width of at least one of the vector elements of the first vector 208 or the corresponding vector elements of the second vector 210 is greater than the data bit width that the mantissa processing unit 304 can process at one time. In one embodiment, the control circuit 406 may be implemented as a control signal, such as a counter or a control flag. To achieve multiple calls, the partial product summation unit 404 may further include a shifter. When the control circuit 406 calls the mantissa processing unit 304 multiple times according to the operation mode, the shifter shifts the existing summation result in each call and adds it to the summation result obtained in the current call to obtain a new summation result. The new summation result obtained in the last call is used as the mantissa after the multiplication operation.

[0054] In one embodiment, the floating-point multiplier 206 disclosed herein further includes a regularization unit 408 and a rounding unit 410. The regularization unit 408 can be used to perform floating-point number regularization processing on the mantissa and exponent after multiplication to obtain a regularized exponent result and a regularized mantissa result, and use the regularized exponent result and the regularized mantissa result as the exponent and mantissa after multiplication. For example, according to the data format indicated by the arithmetic module, the regularization unit 408 can adjust the bit width of the exponent and mantissa to conform to the requirements of the aforementioned indicated data format. In addition, the regularization unit 408 can also make other adjustments to the exponent or mantissa. For example, in some application scenarios, when the mantissa value is not 0, the most significant bit of the mantissa should be 1; otherwise, the exponent bit can be modified and the mantissa bit shifted simultaneously to make it a normalized number. In another embodiment, the regularization unit 408 can also adjust the exponent after multiplication based on the mantissa after multiplication. For example, when the highest bit of the mantissa after multiplication is 1, the exponent obtained after multiplication can be incremented by 1. Correspondingly, the rounding unit 410 can perform a rounding operation on the regularized mantissa result according to a rounding mode, and use the mantissa after rounding as the mantissa after the multiplication operation. Depending on the application scenario, the rounding unit 410 can perform rounding operations such as rounding down, rounding up, and rounding to the nearest significant number. In some application scenarios, the rounding unit 410 can also round the 1 shifted out during the right shift of the mantissa.

[0055] In addition to the exponent processing unit 302 and the mantissa processing unit 304, the floating-point multiplier 206 disclosed herein may optionally include a sign processing unit 306. When the input vector is a floating-point number with a sign bit, the sign processing unit 306 can be used to obtain the sign of the multiplication operation based on the signs of the vector elements of the first vector 208 and the signs of the corresponding vector elements of the second vector 210. For example, in one embodiment, the sign processing unit 306 may include an XOR logic circuit 412, which is used to perform an XOR operation based on the signs of the vector elements of the first vector 208 and the signs of the corresponding vector elements of the second vector 210 to obtain the sign of the multiplication operation. In another embodiment, the sign processing unit 306 can also be implemented using a truth table or logical judgment.

[0056] In addition, to ensure that the vector elements of the input or received first and second vectors conform to a specified format, in one embodiment, the floating-point multiplier 206 disclosed herein may further include a normalization processing unit 414, used to normalize the vector elements of the first vector 208 or the second vector 210 according to the operation mode when, for example, the vector elements of the first vector 208 or the vector elements of the second vector 210 are unnormalized non-zero floating-point numbers, to obtain the corresponding exponent and mantissa. For example, when the selected operation mode is the second operation mode shown in Table 2, and the vector elements of the input first and second vectors 208 and 210 are FP16 type data, the normalization processing unit 414 can be used to normalize the FP16 type data to BF16 type data so that the floating-point multiplier 206 can operate in the second operation mode. In one or more embodiments, the normalization processing unit 414 may also be used to preprocess (e.g., mantissa expansion) the mantissa of normalized floating-point numbers with implicit 1s and unnormalized floating-point numbers without implicit 1s, to facilitate the operation of the subsequent mantissa processing unit 304. Based on the above description, it can be understood that the normalization processing unit 414 and the aforementioned regularization unit 408 may perform the same or similar operations in some embodiments. The difference is that the normalization processing unit 414 performs normalization processing on the input floating-point data, while the regularization unit 408 performs normalization processing on the mantissa and exponent to be output.

[0057] The above combination Figure 4 The floating-point multiplier 206 and several embodiments thereof disclosed herein have been described. Based on the above description, those skilled in the art will understand that the solution disclosed herein obtains the result of a multiplication operation (including the exponent, mantissa, and optional sign) through the execution of the floating-point multiplier 206. Depending on the application scenario, for example, when the aforementioned regularization and rounding processes are not required, the result obtained by the mantissa processing unit 304 and the exponent processing unit 302 can be regarded as the final operation result 212. Furthermore, when the aforementioned regularization and rounding processes are required, the exponent and mantissa obtained after the regularization and rounding processes can be regarded as the final operation result 212, or a part of the final operation result (when considering the final sign). Furthermore, the solution disclosed herein enables the floating-point multiplier 206 to support operations on floating-point numbers of different types or data formats through multiple operation modes, thereby achieving the reuse of the floating-point multiplier 206, thereby saving chip design overhead and reducing computational costs. Furthermore, through a multiple-call mechanism, the computing device disclosed herein also supports calculations of high-bit-width floating-point numbers. Given that in floating-point multiplication operations, the multiplication of the mantissa (or mantissa bits or mantissa portion) is crucial to the performance of the entire vector inner product, the following will combine... Figure 5 To describe the tails operation disclosed herein.

[0058] Figure 5 This is a schematic block diagram illustrating the operation of the mantissa processing unit 500 according to an embodiment of this disclosure. Figure 5 As shown, the mantissa processing operation 500 disclosed herein may primarily involve two units, namely, the aforementioned combination as shown below. Figure 4 The partial product operation unit 402 and the partial product summation unit 404 are discussed. From the perspective of operation timing, the mantissa processing operation 500 can be roughly divided into a first stage and a second stage. In the first stage, the mantissa processing operation 500 will obtain an intermediate result, while in the second stage, the mantissa processing operation 500 will obtain the mantissa result output from the adder 508.

[0059] In an exemplary specific operation, the vector elements of the first vector 208 and the corresponding vector elements of the second vector 210 received by the floating-point multiplier 206 can be divided into multiple parts, namely the aforementioned sign (optional), exponent, and mantissa. Optionally, after normalization, the mantissa portions of the two floating-point numbers will be used as input to the mantissa processing unit (e.g., ...). Figure 3 or Figure 4 The mantissa processing unit 304 in the middle, and specifically enters the partial product operation unit 402. For example Figure 5 As shown, this disclosure utilizes the Booth encoding circuit 502 to pad the mantissas of the corresponding vector elements of the second vector 210 (i.e., the multiplier in floating-point arithmetic) with 0s at both high and low bits, and performs Booth encoding processing to obtain the intermediate result in the partial product generation circuit 504. Of course, in some application scenarios, the vector elements of the first vector 208 can be multipliers, and the corresponding vector elements of the second vector 210 can be multiplicands. Accordingly, in some encoding processes, encoding operations can also be performed on the floating-point number acting as the multiplicand.

[0060] To better understand the technical solution disclosed herein, a brief introduction to Booth encoding is provided below. Generally, when two binary numbers are multiplied, the multiplication operation produces a large number of intermediate results called partial products. These partial products are then accumulated to obtain the final result of multiplying the two binary numbers. The more partial products there are, the larger the area and power consumption of the array floating-point multiplier 206 will be, the slower the execution speed will be, and the more difficult its implementation circuit will be. The purpose of Booth encoding is to effectively reduce the number of summation terms in the partial products, thereby reducing the circuit area. Its algorithm involves first encoding the input multipliers according to corresponding rules. In one embodiment, the encoding rules can be, for example, the rules shown in Table 4 below:

[0061] Table 4

[0062]

[0063]

[0064] y in Table 4 2i+1 y 2i and y 2i-1 X can represent the numerical value corresponding to each group of sub-data to be encoded (i.e., the multiplier), and X can represent the mantissa of the vector elements (i.e., the multiplicand) of the first vector 208. After performing Booth encoding on each group of corresponding sub-data to be encoded, the corresponding encoded signal PPi (i = 0, 1, 2, ..., n) is obtained. As illustrated in Table 4, the encoded signal obtained after Booth encoding can include five categories, namely -2X, 2X, -X, X, and 0. For example, based on the above encoding rules, if the received multiplicand is 8-bit data "X7X6X5X4X3X2X1X0", the following partial product can be obtained:

[0065] 1) When the multiplier includes three consecutive data "001" in the table above, the partial product is X, which can be represented as "X7X6X5X4X3X2X1X0". The 9th bit is the sign bit, i.e. PPi = {X[7], X};

[0066] 2) When the multiplier includes three consecutive digits “011” from the table above, the partial product is 2X, which can be represented as X shifted left by one digit, resulting in “X7X6X5X4X3X2X1X00”, i.e., PPi = {X, 0}.

[0067] 3) When the multiplier includes three consecutive digits "101" from the table above, the partial product is -X, which can be expressed as: This means that “X7X6X5X4X3X2X1X0” is inverted bitwise and then 1 is added, i.e., PPi = ~{X[7],X}+1;

[0068] 4) When the multiplier includes three consecutive digits "100" from the table above, the partial product is -2X, which can be expressed as: This means shifting "X7X6X5X4X3X2X1X0" one bit to the left, inverting the result, and then adding 1, i.e., PPi = ~{X, 0} + 1;

[0069] 5) When the multiplier includes three consecutive digits “111” or “000” from the table above, the partial product is 0, i.e., PPi = {9′b0}.

[0070] It should be understood that the above description of the process for obtaining partial products in conjunction with Table 4 is merely exemplary and not restrictive. Those skilled in the art, under the teachings of this disclosure, can modify the rules in Table 4 to obtain partial products different from those shown in Table 4. For example, when there are multiple consecutive digits (e.g., 3 or more) of a specific number in the multiplier, the obtained partial product can be the two's complement of the multiplicand, or, for example, the "add 1" operation in items 3) and 4) above can be performed after summing the partial products.

[0071] As can be understood from the above introductory description, by encoding the mantissas of the corresponding vector elements of the second vector 210 using the Booth encoding circuit 502, and using the mantissas of the vector elements of the first vector 208, multiple partial products can be generated from the partial product generation circuit 504 as intermediate results, and the intermediate results are fed into the Wallace Tree compressor 506 in the partial product summing unit 404. It should be understood that obtaining partial products using Booth encoding is only a preferred method of obtaining partial products in this disclosure, and those skilled in the art can obtain such partial products in other ways. For example, it can also be obtained through a shift operation, i.e., selecting whether to shift and add the multiplicand or add 0 based on whether the multiplier's bit value is 1 or 0 to obtain the corresponding partial product. Similarly, using the Wallace Tree compressor 506 to implement the partial product addition operation is merely exemplary and not limiting; those skilled in the art can also conceive of using other types of adders to implement such partial product addition operations, such as one or more full adders, half adders, or various combinations thereof.

[0072] The Wallace tree compressor 506 (or simply Wallace tree) is primarily used to sum the intermediate results (i.e., multiple partial products) mentioned above, reducing the number of partial product summations (i.e., compression). Typically, the Wallace tree compressor 506 employs a carry-save (CAS) architecture and the Wallace tree algorithm, which utilizes a Wallace tree array for computation much faster than traditional carry-pass addition.

[0073] Specifically, the Wallace tree compressor 506 can compute the sum of partial products of each row in parallel. For example, it can reduce the number of times to accumulate N partial products from N-1 times to Log2N times, thereby improving the speed of the floating-point multiplier 206 and is of great significance for the efficient use of resources. Depending on different application needs, the Wallace tree compressor 506 can be designed in various types, such as 7-2 Wallace trees, 4-2 Wallace trees, and 3-2 Wallace trees. In one or more embodiments, this disclosure uses a 7-2 Wallace tree as an example of implementing various vector inner products of this disclosure, which will be discussed later in conjunction with... Figure 6 and Figure 7 It is described in detail.

[0074] In some embodiments, the Wallace tree compression operation disclosed herein can be arranged to have M inputs and N outputs, the number of which can be no less than K, where N is a preset positive integer less than M, and K is a positive integer no less than the maximum bit width of the intermediate results. For example, M can be 7, and N can be 2, i.e., a 7-2 Wallace tree as described in detail below. When the maximum bit width of the intermediate results is 48, K can be a positive integer of 48, meaning the number of Wallace trees can be 48.

[0075] In some embodiments, depending on the operation mode, one or more groups of Wallace trees can be selected to sum the intermediate results, wherein each group has X Wallace trees, and X is the number of bits in the intermediate result. Furthermore, there can be a carry relationship between Wallace trees within each group, but no carry relationship between groups. In an exemplary connection, the Wallace tree compressor 506 can be connected via carry, for example, the carry output from the lower-order Wallace tree compressor 506 (e.g., ...). Figure 7 C in The carry output (C) of the high-order Wallace tree compressor 506 is sent to the high-order Wallace tree compressor. out It can also become a higher-order Wallace tree compressor 506 receiving carry input from a lower-order Wallace tree compressor 506. Furthermore, when selecting one or more Wallace tree compressors 506 from a plurality of Wallace tree compressors 506, any selection can be made; for example, they can be selected in the order of numbers 0, 1, 2, and 3, or connected in the order of numbers 0, 2, 4, and 6, as long as the selected Wallace tree compressors 506 are selected according to the aforementioned carry relationship.

[0076] The Wallace tree and its operations described above will be illustrated with an illustrative example below. Assume that the vector elements of the first vector 208 and the corresponding vector elements of the second vector 210 are 16-bit data, the computing device supports a 32-bit input width (thus supporting parallel multiplication of two sets of 16-bit data), and the Wallace tree is a 7-2 Wallace tree compressor 506 with 7 inputs (i.e., an example value of M mentioned above) and 2 outputs (i.e., an example value of N mentioned above). In this example scenario, 48 Wallace trees (i.e., an example value of K mentioned above) can be used to perform the multiplication of the two sets of data in parallel.

[0077] Of the 48 Wallace trees mentioned above, trees 0 through 23 (i.e., the 24 trees in the first group) can perform partial product summation of the first group of multiplications, and the Wallace trees within this group can be connected sequentially via carry. Furthermore, trees 24 through 47 (i.e., the 24 trees in the second group) can perform partial product summation of the second group of multiplications, where the Wallace trees within this group are connected sequentially via carry. Additionally, there is no carry relationship between the 23rd Wallace tree in the first group and the 24th Wallace tree in the second group; that is, there is no carry relationship between Wallace trees in different groups.

[0078] Return to Figure 5 After the partial products are summed and compressed by the Wallace tree compressor 506, the compressed partial products are summed by the adder 508 to obtain the result of the mantissa multiplication operation. Regarding the adder 508, in one or more embodiments disclosed herein, it may include one of a full adder, a serial adder, and a carry-lookahead adder, used to sum the last two rows of partial products obtained by the Wallace tree compressor 506 to obtain the result of the mantissa multiplication operation.

[0079] Understandable, through Figure 5 The mantissa multiplication operation illustrated, particularly using Booth coding and Wallace trees, can efficiently obtain the result of the mantissa multiplication operation. Specifically, Booth coding effectively reduces the number of partial product summation terms, thereby reducing circuit area, while Wallace compressed trees can compute the sum of partial products of each row in parallel, thereby improving the speed of the computing device.

[0080] The following will combine Figure 6 and Figure 7 The example operations for partial product sums and 7-2 Wallace trees are described in detail. It is understood that the descriptions herein are merely illustrative and not restrictive, and are intended only to provide a better understanding of this disclosure.

[0081] Figure 6 This shows that after the aforementioned combination Figures 3 to 5 The partial product 600 obtained after the partial product generation circuit 504 in the described mantissa processing unit 304 is represented by four rows of white dots between the two dashed lines in the figure, where each row of white dots identifies a partial product. To facilitate the subsequent execution of the Wallace tree compressor 506, the bit depth can be pre-expanded. For example, Figure 6The black dots represent the highest bit value of each 9-bit partial product that has been copied. It can be seen that the partial product is expanded and aligned to 16 (8+8) bits (i.e., the mantissa of the multiplicand is 8 bits wide + the mantissa of the multiplier is 8 bits wide). In another embodiment, for example, for the partial product of a 25*13 binary multiplication, the partial product is expanded to 38 (25+13) bits (i.e., the mantissa of the multiplicand is 25 bits wide + the mantissa of the multiplier is 13 bits wide).

[0082] Figure 7 This is a schematic block diagram 700 illustrating the operation flow and schematic block diagram of the Wallace tree compressor 506 according to an embodiment of this disclosure.

[0083] like Figure 7 As shown, after performing a multiplication operation on the mantissas of two floating-point numbers, as previously described, the result can be obtained by encoding the multiplier with Booth code and using the multiplicand. Figure 7 The diagram shows seven partial products. The number of partial products generated is reduced due to the use of the Booth coding algorithm. For ease of understanding, a Wallace tree with seven elements is marked with a dashed box in the partial product section of the diagram, and the process of compressing it from seven elements to two elements is further illustrated with arrows. In one embodiment, this compression process (or summation process) can be implemented using a full adder, i.e., inputting three elements and outputting two elements (i.e., a sum and a carry to the higher bit). A schematic block diagram of the Wallace tree compressor 506 is shown in 7-2. Figure 7 As shown on the right, it can be understood that the Wallace tree compressor 506 includes 7 inputs from a column of partial products (such as...). Figure 7 (The seven elements are marked in the dashed box on the left). In operation, the carry input of the 0th column of the Wallace tree is 0, and the carry output Cout of each column of the Wallace tree serves as the carry input Cin of the next column of the Wallace tree.

[0084] from Figure 7 As can be seen in the left-hand section, after four compressions, a Wallace tree containing 7 elements can be compressed into one containing 2 elements. As mentioned earlier, this disclosure uses a 7-2 Wallace tree compressor 506 to finally compress the partial product of 7 rows into a partial product with two rows (i.e., the second intermediate result of this disclosure), and uses an adder (e.g., a carry-lookahead adder) to obtain the mantissa result.

[0085] To further illustrate the principle of this disclosure, the following will exemplarily describe how the floating-point multiplier 206 of this disclosure completes the first-stage operation under the four operation modes of FP16*FP16, FP16*FP16, FP32*FP32, and FP32*BF16, that is, until the Wallace tree compressor 506 completes the summation of intermediate results to obtain the second intermediate result:

[0086] (1)FP16*FP16

[0087] In the operation mode of the floating-point multiplier 206, the mantissa of the floating-point number is 10 bits. Considering the denormalized non-zero number under the IEEE 754 standard, it can be extended by 1 bit, resulting in a mantissa of 11 bits. Furthermore, since the mantissa is an unsigned number, when using the Booth encoding algorithm, 1 bit of 0 can be added to the high-order bits, thus the total mantissa is 12 bits. When Booth encoding is performed on the corresponding vector element (multiplier) of the second vector 210, and referring to the vector elements of the first vector 208, seven partial products can be obtained in both the high and low parts through the partial product generation circuit. The seventh partial product is 0, and the bit width of each partial product is 24 bits. This can be compressed using 48 7-2 Wallace trees, with the carry from the 23rd to 24th Wallace trees being 0.

[0088] (2)BF16*BF16

[0089] In this operation mode of the floating-point multiplier 206, the mantissa of the floating-point number is 7 bits. Considering the denormalized non-zero number and the extension to a signed number under the IEEE 754 standard, the mantissa can be extended to 9 bits. When the corresponding vector element of the second vector 210, i.e. the multiplier, is encoded using Booz and referenced to the vector element of the first vector 208, the partial product generation circuit 504 can obtain 7 valid partial products in the high and low parts respectively. Among them, the 6th and 7th partial products are 0. Each partial product has a bit width of 18 bits. It is compressed by using two groups of 7-2 Wallace trees, namely the 0th to 17th and the 24th to 41st, where the carry of the 23rd to 24th Wallace trees is 0.

[0090] (3)FP32*FP32

[0091] In this operation mode of the floating-point multiplier 206, the mantissa of the floating-point number can be 23 bits. Considering the denormalized non-zero number under the IEEE 754 standard, the mantissa can be extended to 24 bits. To save the area of ​​the multiplication unit, the floating-point multiplier 206 disclosed herein can be called twice in this operation mode to complete one operation. For this purpose, the multiplication of the mantissa each time is 25 bits * 13 bits, that is, the vector element ina of the first vector 208 is extended by 1 bit 0 to become a 25-bit signed number, and the 24-bit mantissa of the corresponding vector element inb of the second vector 210 is divided into two 12-bit parts, high and low, and extended by 1 bit 0 to obtain two 13-bit multipliers, represented as inb_high13 and inb_low13. In specific operation, the floating-point multiplier 206 disclosed herein is called for the first time to calculate ina * inb_low13, and the floating-point multiplier 206 is called for the second time to calculate ina * inb_high13. In each calculation, seven valid partial products are generated through Booth encoding, each with a bit width of 38 bits, and compressed using a 7-2 Wallace tree from the 0th to the 37th product.

[0092] (4)FP32*BF16

[0093] In this operation mode of the floating-point multiplier 206, the mantissa of the vector element ina of the first vector 208 is 23 bits, and the mantissa of the corresponding vector element inb of the second vector 210 is 7 bits. Considering the denormalized non-zero number and the extended signed number under the IEEE 754 standard, the mantissa can be extended to 25 bits and 9 bits respectively. A 25-bit × 9-bit multiplication is performed to obtain 7 valid partial products, of which the 6th and 7th partial products are 0. The bit width of each partial product is 34 bits, which is compressed by the 0th to 33rd Wallace trees.

[0094] The above examples illustrate how the floating-point multiplier 206 of this disclosure completes the first-stage operation in four operational modes, with Booth coding and a 7-2 Wallace tree being preferredly used. Based on the above description, those skilled in the art will understand that this disclosure uses seven partial products, allowing the 7-2 Wallace tree to be reused in different operational modes.

[0095] In some operation modes, the aforementioned mantissa processing unit 304 may further include a control circuit 406, which can be used to call the mantissa processing unit 304 multiple times according to the operation mode when the mantissa bit width of the vector elements of the first vector 208 and / or the mantissa bit width of the corresponding vector elements of the second vector 210 is greater than the data bit width that the mantissa processing unit 304 can process at one time. Further, for the case of multiple calls, the partial product summation circuit may further include a shifter, which is used to shift the existing summation result when the mantissa processing unit 304 is called multiple times according to the operation mode, and add it to the summation result obtained in the current call to obtain a new summation result, and use the new summation result as the mantissa after the multiplication operation.

[0096] For example, as mentioned above, the mantissa processing unit 304 can be called twice in the FP32*FP32 operation mode. Specifically, in the first call to the mantissa processing unit 304, the mantissa bits (i.e., ina*inb_low13) are added in the second stage using a carry-lookahead adder to obtain the second low-order intermediate result. In the second call to the mantissa processing unit 304, the mantissa bits (i.e., ina*inb_high13) are added in the second stage using a carry-lookahead adder to obtain the second high-order intermediate result. Subsequently, in one embodiment, the second low-order intermediate result and the second high-order intermediate result can be accumulated using a shift operation of a shifter to obtain the mantissa after the multiplication operation. This shift operation can be expressed as follows:

[0097] r fp32xfp32 =sum h [37:0] << 12+sum l [37:0]

[0098] The second highest intermediate result sum h [37:0] Shift left by 12 bits and sum with the middle result of the second least significant bit. l [37:0]Accumulate.

[0099] The above text combined Figures 5 to 7 The disclosure describes in detail the operation performed by the floating-point multiplier 206 when performing a vector dot product, specifically multiplying the mantissas of the vector elements of the first vector 208 and the corresponding vector elements of the second vector 210. Of course, Figure 5 To focus on describing the operation of the mantissa processing unit 304 of the floating-point multiplier 206 disclosed herein, other units, such as the exponent processing unit 302 and the sign processing unit 306, are not shown or described. The following will combine... Figure 8 The description of the floating-point multiplier 206 disclosed herein, as well as the description of the mantissa processing unit 304 given above, also applies. Figure 8 The scene depicted.

[0100] Figure 8 This is a general schematic block diagram illustrating a floating-point multiplier 206 according to an embodiment of this disclosure. It should be understood that the positions, presence, and connections of the various units depicted in the figure are merely exemplary and not limiting. For example, some units may be integrated, while others may be separated or omitted or replaced depending on the application scenario.

[0101] The floating-point multiplier 206 disclosed herein can be exemplarily divided into a first stage and a second stage according to the operation flow in each operation mode, as shown by the dotted lines in the figure. In summary, in the first stage: the calculation result of the sign bit is output, the intermediate result calculation of the exponent bit is output, and the intermediate result calculation of the mantissa bit is output (e.g., including the aforementioned encoding process of the fixed-point multiplication Booth algorithm for the input mantissa bit and the Wallace tree compression process). In the second stage: regularization and rounding operations are performed on the exponent and mantissa to output the calculation result of the exponent and the calculation result of the mantissa.

[0102] like Figure 8 As shown, the floating-point multiplier 206 disclosed herein may include a mode selection unit 802 and a normalization processing unit 804, wherein the mode selection unit 802 can select the operation mode according to the input mode signal (in_mode). In one embodiment, the input mode signal may correspond to the operation mode number in Table 2. For example, when the input mode signal indicates operation mode number "1" in Table 2, the floating-point multiplier 206 can be made to operate in the FP16*FP16 operation mode, and when the input mode signal indicates operation mode number "3" in Table 2, the floating-point multiplier 206 can be made to operate in the FP32*FP32 operation mode. For illustrative purposes, Figure 8 Only four exemplary operation modes are shown: FP16*FP16, BF16*BF16, FP32*FP32, and FP32*BP16. However, as previously stated, the floating-point multiplier 206 disclosed herein also supports many other different operation modes.

[0103] The normalization processing unit 804 can be configured to normalize the vector elements of the first vector 208 or the corresponding vector elements of the second vector 210 according to the operation mode when the vector elements of the first vector 208 or the corresponding vector elements of the second vector 210 are unnormalized non-zero floating-point numbers, so as to obtain the corresponding exponent and mantissa, for example, to normalize the floating-point numbers in the data format indicated by the operation mode according to the IEEE 754 standard.

[0104] Furthermore, the floating-point multiplier 206 includes a mantissa processing unit to perform a multiplication operation of the mantissas of the vector elements of the first vector 208 and the corresponding mantissas of the vector elements of the second vector 210. To this end, in one or more embodiments, the mantissa processing unit may include a bit-length expansion circuit 806, a Booth encoder 808, a partial product generation circuit 810, a Wallace tree compressor 812, and an adder 814, wherein the bit-length expansion circuit 806 can be used to expand the mantissa to accommodate denormalized non-zero numbers under the IEEE 754 standard, to suit the operation of the Booth encoder. Since the Booth encoder 808, the partial product generation circuit 810, the Wallace tree compressor 812, and the adder 814 have already been combined... Figures 5 to 7 The details have been described in detail, so I will not repeat them here.

[0105] In some embodiments, the floating-point multiplier 206 disclosed herein further includes a regularization unit 816 and a rounding unit 818, the regularization unit 816 and the rounding unit 818 having the same characteristics as... Figure 4 The units shown have the same function. Specifically, for the regularization unit 816, it can be based on, as shown in the example... Figure 8 The output mode signal "out_mode" indicates the data format used to perform floating-point regularization processing on the summation result and the exponent data from the exponent processing unit 820 to obtain a regularized exponent result and a regularized mantissa result. For example, according to the data format indicated by the output mode signal, the regularization unit 816 can adjust the bit width of the exponent and mantissa to conform to the requirements of the aforementioned indicated data format. For another example, when the highest bit of the mantissa is 0, and the mantissa is not 0, the regularization unit 816 can repeatedly shift the mantissa left by 1 bit and decrement the exponent by 1 until the highest bit value is 1. For the rounding unit 818, in one embodiment, it can be used to perform a rounding operation on the regularized mantissa result according to the rounding mode to obtain a rounded mantissa, and use the rounded mantissa as the mantissa after the multiplication operation.

[0106] In one or more embodiments, the aforementioned output mode signal "out_mode" can be part of the operation mode, used to indicate the data format after the multiplication operation. For example, as described in Table 3 above, when the operation mode number is "12", the number "1" can be equivalent to the aforementioned "in_mode" signal, used to indicate the execution of an FP16*FP16 multiplication operation, while the number "2" can be equivalent to the "out_mode" signal, used to indicate that the data type of the output result is BF16. Therefore, it can be understood that in some application scenarios, the output mode signal can be combined with the aforementioned input mode signal to provide it to the mode selection unit 802. Based on this combined mode signal, the mode selection unit 802 can determine the data format of the input data and the output result at the initial stage of the floating-point multiplier 206 operation without having to provide a separate output mode signal to the regularization, thereby further simplifying the operation.

[0107] In one or more embodiments, the aforementioned rounding operation may include, for example, the following five rounding modes.

[0108] (1) Round to the nearest value: In this mode, when two values ​​are equally close, the even number takes precedence. The result will be rounded to the nearest representable value. However, when two numbers are equally close, the even number (which is the number ending in 0 in binary) will be used as the rounding result.

[0109] (2) Rounding: See the example below for a demonstration operation;

[0110] (3) Rounding towards +∞: Under this rule, the result will be rounded towards positive infinity;

[0111] (4) Rounding towards -∞: Under this rule, the result will be rounded towards negative infinity; and

[0112] (5) Rounding to 0: Under this rule, the result will be rounded to 0.

[0113] For example, in the "rounding" mode, the mantissa rounding is as follows: Multiplying two 24-bit mantissas results in a 48-bit mantissa (47-0). After normalization, only bits 46 to 24 are taken in the output. When the 23rd bit of the mantissa is 0, the (23-0)th bit is discarded; when the 23rd bit of the mantissa is 1, 1 is carried over to the 24th bit, and the (23-0)th bit is discarded.

[0114] Return to Figure 8 The floating-point multiplier 206 disclosed herein also includes an exponent processing unit 820 and a sign processing unit 822. Figure 9This is a flowchart illustrating a method 900 for performing floating-point multiplication using a floating-point multiplier 206 according to an embodiment of this disclosure.

[0115] like Figure 9 As shown, the method 900 may include, at step S902, using an exponent processing unit 820 to obtain the exponent after the multiplication operation based on the operation mode, the exponents of the vector elements of the first vector 208, and the exponents of the corresponding vector elements of the second vector 208. As previously mentioned, the operation mode may be one of multiple operation modes and may be used to indicate the data format of the floating-point number. In one or more embodiments, the operation mode may also be used to determine the data format of the floating-point number of the output result. For example, the exponent processing unit 820 may add the exponent data of the vector elements of the first vector 208, the exponent data of the corresponding vector elements of the second vector 210, and the offset values ​​of their respective corresponding input floating-point data types, and subtract the offset values ​​of the output floating-point data types to obtain the exponent data of the product of the vector elements of the first vector 208 and the corresponding vector elements of the second vector 210. In one or more embodiments, the exponent processing unit 820 may be implemented as or include an addition / subtraction circuit for obtaining the exponent after the multiplication operation based on the operation mode, the exponents of the vector elements of the first vector 208, the exponents of the corresponding vector elements of the second vector 210, and the operation mode.

[0116] Next, in step S904, the method 900 can utilize the mantissa processing unit to obtain the mantissa after the multiplication operation based on the operation mode, the vector elements of the first vector 208, and the corresponding vector elements of the second vector 208. Regarding exemplary mantissa operations, this disclosure uses a Booth coding algorithm and a Wallace tree compressor in some preferred embodiments to improve the efficiency of mantissa processing.

[0117] Furthermore, when the vector elements of the first vector 208 and the corresponding vector elements of the second vector 208 are signed numbers, method 900 can also obtain the sign of the multiplication operation in step S906 by the sign processing unit 822 based on the signs of the vector elements of the first vector 208 and the corresponding vector elements of the second vector 208. In one embodiment, the sign processing unit 822 can be implemented as an XOR circuit, which performs an XOR operation on the sign bit data of the vector elements of the first vector 208 and the corresponding vector elements of the second vector 210 to obtain the sign bit data of the product of the vector elements of the first vector 208 and the corresponding vector elements of the second vector 210.

[0118] The above text combined Figures 2 to 9The computing device disclosed herein has been described in detail. Through this description, those skilled in the art will understand that the computing device supports operation under multiple operational modes, thereby overcoming the shortcomings of existing multipliers that only support single floating-point operations. Furthermore, since the computing device disclosed herein is reusable, it also supports high-bit-width floating-point data, reducing computational costs and overhead. In one or more embodiments, the computing device disclosed herein may also be arranged or included in an integrated circuit chip or computing device to perform floating-point multiplication operations under multiple operational modes.

[0119] Another embodiment of the vector inner product calculation device disclosed herein is, for example... Figure 10 As shown, the computing device 1000 includes a multiplication unit 1002, a first type conversion unit 1004, an addition module 1006, and an update module 1008. The multiplication unit 1002 includes at least one floating-point multiplier 1010, used to perform element-wise multiplication on the received first vector 1012 and second vector 1014 to obtain the product result 1016 of each pair of corresponding vector elements. In this embodiment, the operation mode of the multiplication unit 1002 can be... Figure 2 The multiplication unit 202 is the same, so it will not be described again.

[0120] The first type conversion unit 1004 is used to convert the data type of the product result 1016 so that the converted product result 1018 can be output to the addition module 1006 for addition operation. In some embodiments, the type of the output (product result 1016) of the multiplication unit 1002 is inconsistent with the input type that the addition module 1006 can accept, so the first type conversion unit 1004 needs to perform type conversion. For example, when the product result 1016 is an FP16 floating-point number, while the addition module 1006 supports FP32 floating-point numbers, the first type conversion unit 1004 can be configured to perform the following operations on the FP16 data to convert it to FP32 data:

[0121] S1: Left shift the sign bit by 16 bits;

[0122] S2: Add 112 to the exponent (the difference between the base of the exponent, 127, and 15), shift left by 13 bits (right alignment); and

[0123] S3: Shift the mantissa 13 bits to the left (left alignment).

[0124] In the above example, the FP32 data can also be converted to FP16 data by performing the reverse operation to meet the requirements of an adder that supports FP16 data. It is understood that the data type conversion method described here is merely exemplary, and those skilled in the art can choose appropriate methods or mechanisms to convert the data type of the multiplication result to a data type suitable for the adder based on the teachings of this disclosure.

[0125] In one embodiment, the addition module 1006 may be the first adder 1028 of a multi-level adder group arranged in a multi-level tree structure. Figure 11 The diagram illustrates one embodiment 1100 of a first adder 1028, exemplified by FP32. As can be seen from the schematic representation, it is a three-level tree-structured adder group. The first level includes four adders 1102, which exemplarily receive eight FP32 floating-point inputs, such as in0, in1, ..., in7. The second level includes two adders 1104, which exemplarily receive four FP16 floating-point inputs. The third level includes only one adder 1106, which can receive two FP16 floating-point inputs and output the sum of the aforementioned eight FP32 floating-point inputs.

[0126] In this embodiment, it is assumed that the two adders 1104 of the second stage do not support addition operations of FP32 floating-point numbers. Therefore, this disclosure proposes to provide one or more second type conversion units 1108 between the first-stage and second-stage adders. In one embodiment, the second type conversion unit 1108 may have a combination with... Figure 10 The first type conversion unit 1004 has the same or similar function, namely, converting the input floating-point data into a data type consistent with the subsequent addition operation. Specifically, the second type conversion unit 1108 can support one or more data type conversions according to different application requirements. For example, in Figure 11In the example shown, it can support one-way data type conversion from FP32 to FP16. In other examples, the second type conversion unit 1108 can be designed to support bidirectional data type conversion between FP32 and FP16 data. In other words, it can support both FP32 to FP16 and FP16 to FP32 data conversion. Additionally or optionally, the first type conversion unit 1004 or the second type conversion unit 1108 can also be configured to support bidirectional conversion between various floating-point data types. For example, it can support bidirectional conversion between various floating-point data types described in the aforementioned combined operation mode, thereby helping to maintain forward or backward compatibility of data during data processing and further expanding the application scenarios and scope of the disclosed solution. It should be emphasized that the above-mentioned type conversion unit is only an optional solution of this disclosure. When the first or second adder itself supports addition operations of multiple data formats, or when operations of multiple data formats can be reused, such a type conversion unit is not required. In addition, when the data format supported by the second adder is the same as the data format output by the first adder, there is no need to set up such a type conversion unit between the two.

[0127] Figure 12 This is a schematic block diagram illustrating an adder group 1200 according to another embodiment of the first adder 1006 disclosed herein. As can be seen from the figure, it schematically illustrates a five-level tree-structured adder group, specifically including 16 adders in the first level, 8 adders in the second level, 4 adders in the third level, 2 adders in the fourth level, and 1 adder in the fifth level. From this multi-level tree structure, it can be seen that... Figure 12 The adder group 1200 shown can be considered as a... Figure 11 The tree structure shown is an extension. Or conversely, Figure 11 The adder group 1100 shown can be regarded as Figure 12 A portion or component of the adder group 1200 shown, such as Figure 12 The part enclosed by the dashed line 1202.

[0128] In operation, the 16 adders in the first group can receive the product result 1018 from the first type conversion unit 1004. Optionally, when the aforementioned product result 1016 has the same data type supported by the first-stage adder of the adder group 1200 of the adder module 1006, it can be directly input into the adder group 1200 without going through the first type conversion unit 1004, for example... Figure 12The diagram shows 32 FP32 floating-point numbers (e.g., in0 to in31). After addition operations by the first-stage 16 adders, 16 sums are obtained, which serve as inputs to the second-stage 8 adders. This process continues, with the final sum, output by the fourth-stage 2 adders, being input to the fifth-stage 1 adder. The output of this fifth-stage adder can then be used as... Figure 10 The intermediate result 1020 is input into the second adder 1024 located in the update module 1008. Depending on the application scenario, the intermediate result 1020 may undergo one of the following operations:

[0129] When the intermediate result 1020 is the same as the intermediate result 1020 obtained from the first round of calling the multiplication unit 1002, it can be input into the second adder 1024 of the aforementioned update module 1008, and then cached in the register 1026 of the update module 1008 to wait for addition with the intermediate result 1020 obtained in the second round; or

[0130] When the intermediate result 1020 is the result of an intermediate round (e.g., when more than two rounds of operation are performed), it can be input into the second adder 1024 and then added to the sum of the previous round of addition operations input from register 1026 into the second adder 1024, so as to store the sum of this intermediate round of addition operations in register 1026; or

[0131] When the intermediate result 1020 is the intermediate result 1020 obtained by the last round of calling the multiplication unit 1002, it can be input into the second adder 1024, and then added to the summation result obtained by the previous round of addition operation input by the register 1026 into the second adder 1024, as the final result 1022 of this vector inner product operation.

[0132] Considering that the first adder 1028 of the aforementioned addition module 1006 can be a floating-point adder supporting multiple modes, correspondingly, the second adder 1024 in the update module 1008 can also have the same or similar properties, that is, it also supports floating-point addition operations in multiple modes. When the first adder 1028 or the second adder 1024 does not support addition operations for multiple floating-point data formats, this disclosure also discloses a first or second type conversion unit for performing conversions between data types or formats, thereby enabling the first or second adder to perform floating-point addition operations in multiple modes. Although Figure 12This disclosure uses a tree-like hierarchical arrangement of multiple adders to perform addition operations on multiple numbers, but the solution disclosed herein is not limited to this. Those skilled in the art, based on the teachings of this disclosure, can also arrange multiple adders in other suitable structures or methods, such as by serially or in parallel connecting multiple full adders, half adders, or other types of adders to perform addition operations on multiple input floating-point numbers. Furthermore, for the sake of simplicity, Figure 12 The addition tree structure shown does not show, for example, the ... Figure 11 The second type conversion unit 1108 is shown. However, depending on the needs of the application, those skilled in the art will realize that... Figure 12 The multi-level adder shown is provided with one or more inter-level type conversion units to enable data type conversion between different levels, thereby further expanding the applicability of the computing device disclosed herein.

[0133] Figure 13 The operation flow 1300 of the update module 1008 is further illustrated. For clarity, it is assumed here that... Figure 10 The multiplication unit 1002 has 16 multipliers 1010, while the first vector 1012 has 64 FP32s and the second vector 1014 also has 64 FP32s. Since there are 16 multipliers 1010, batch processing is performed in units of 16 FP32s. For example, the multiplication unit 1002 first receives the first to the 16th FP32s of the first vector 1012 and the second vector 1014, and after processing by the first type conversion unit 1004 and the addition module 1006, it is output to the update module 1008.

[0134] In step S1302, the second adder 1024 receives the first segment of intermediate results from the 1st to the 16th FP32s of the addition module 1006. In step S1304, the second adder 1024 transfers the first segment of intermediate results to register 1026 for storage. While the update module 1008 executes steps S1302 and S1304, the multiplication unit 1002 receives the 17th to the 32nd FP32s of the first vector 1012 and the second vector 1014. After processing by the first type conversion unit 1004 and the addition module 1006, in step S1306, the second adder 1024 receives the next segment of intermediate results from the addition module 1006 (e.g., the second segment of intermediate results from the 17th to the 32nd FP32s) and the previous segment (e.g., the first segment) of intermediate results from register 1026. In step S1308, the second adder 1024 adds the next intermediate result to the previous intermediate result, for example, adding the second intermediate result to the first intermediate result to obtain a sum. In step S1310, the second adder 1024 transfers the sum to register 1026, updating the result stored in register 1026. Steps S1306, S1308, and S1310 are then repeated until all 64 FP32 addition operations are completed.

[0135] In one embodiment, the multiplication unit 1002, the first type conversion unit 1004, the addition module 1006, and the update module 1008 can all operate independently and in parallel. For example, after the multiplication unit 1002 outputs the product result 1016, it receives the next pair of corresponding vector elements for multiplication, without waiting for the subsequent stages (first type conversion unit 1004, addition module 1006, and update module 1008) to finish before receiving and processing the data. Similarly, after the first type conversion unit 1004 outputs the converted product result 1018, it receives the next product result 1016 for type conversion; after the addition module 1006 outputs the intermediate result 1020, it receives the next converted product result 1018 from the first type conversion unit 1004 for addition. In some embodiments, vector type conversion is not required, and the computing device 1000 may not need to provide the first type conversion unit 1004. Those skilled in the art can easily deduce how each level of unit / module can operate in parallel without the first type conversion unit 1004, so it will not be described in detail here.

[0136] Figure 14 This is a flowchart illustrating a method 1400 for performing vector inner product operations using a computing device according to an embodiment of this disclosure. It will be understood that the computing device described herein can be... Figure 2 or Figure 10 Computing device.

[0137] by Figure 2Taking a computing device as an example. In step S1402, the multiplication unit 202 is used to perform multiplication operations on corresponding vector elements of the first vector 208 and the second vector 210 to obtain the product result 212 of each pair of corresponding vector elements; in step S1404, the addition module 204 is used to perform an addition operation on the product result of corresponding vector elements of the first vector 208 and the second vector 210 to obtain the floating-point vector inner product result 216. Although in Figure 14 Not shown, but as previously described, in some embodiments, the method can be executed cyclically when the bit width of the input vector or its vector elements exceeds the bit width of the computing device input port.

[0138] Although the above method illustrates the use of the computing device disclosed herein to perform floating-point vector inner product operations in the form of steps, the order of these steps does not imply that the steps of this method must be performed in the stated order, but can be processed in other orders or in parallel. Furthermore, other steps of this disclosure are not described herein for the sake of brevity, but those skilled in the art will understand from the content of this disclosure that the method can also perform the various operations described above in conjunction with the accompanying drawings using a computing device.

[0139] In the above embodiments disclosed herein, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant descriptions in other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as these combinations of technical features do not contradict each other, they should be considered within the scope of this specification.

[0140] Figure 15 This is a structural diagram illustrating a combined processing apparatus 1500 according to an embodiment of the present disclosure. As shown, the combined processing apparatus 1500 includes a computing device 1502, which may be... Figure 2 or Figure 10 The computing device. Additionally, the combined processing device 1500 includes a general-purpose interconnect interface 1504 and other processing devices 1506. The computing device disclosed herein interacts with other processing devices to jointly perform user-specified operations.

[0141] According to the scheme disclosed herein, the other processing device 1506 may include one or more types of processors such as a central processing unit (“CPU”), a graphics processing unit (“GPU”), and an artificial intelligence processor, and the number of such processors is not limited but determined according to actual needs. In one or more embodiments, the other processing device 1506 may serve as an interface between the computing device 1502 disclosed herein (which may be specifically embodied as an artificial intelligence computing device) and external data and control, performing tasks including but not limited to data transfer, and completing basic control such as starting and stopping the machine learning computing device; the other processing device may also cooperate with the machine learning computing device to jointly complete computing tasks.

[0142] According to the disclosed scheme, the general interconnect interface 1504 can be used to transfer data and control commands between the computing device 1502 and other processing devices 1506. For example, the computing device 1502 can obtain the required input data from other processing devices 1506 via the general interconnect interface 1504 and write it to the on-chip storage device of the computing device 1502. Further, the computing device 1502 can obtain control commands from other processing devices 1506 via the general interconnect interface 1504 and write them to the on-chip control cache of the computing device 1502. Alternatively or optionally, the general interconnect interface 1504 can also read data from the storage module of the computing device 1502 and transmit it to other processing devices 1506.

[0143] Optionally, the combined processing device 1500 may further include a storage device 1508, which may be connected to the computing device 1502 and the other processing device 1506, respectively. In one or more embodiments, the storage device 1508 may be used to store data of the computing device 1502 and the other processing device 1506, particularly suitable for data that cannot be fully stored in the internal storage of the computing device 1502 or other processing device 1506, which requires computation.

[0144] Depending on the application scenario, the combined processing device 1500 disclosed herein can serve as a System-on-a-Chip (SoC) for devices such as mobile phones, robots, drones, video capture equipment, and video surveillance equipment. This effectively reduces the core area of ​​the control unit, increases processing speed, and lowers overall power consumption. In this case, the general-purpose interconnect interface 1504 of the combined processing device 1500 connects to certain components of the device. These components may include, for example, a camera, monitor, mouse, keyboard, network card, or Wi-Fi interface.

[0145] In some embodiments, this disclosure also discloses a chip or integrated circuit chip that includes a combined processing device 1500. In other embodiments, this disclosure also discloses a chip package structure that includes the above-described chip.

[0146] In some embodiments, this disclosure also discloses a circuit board that includes the above-described chip packaging structure. See also... Figure 16 The aforementioned exemplary board 1600 is provided. In addition to the aforementioned chip 1602, the board 1600 may also include other supporting components, which may include, but are not limited to, a storage device 1604, an interface device 1606, and a controller 1608.

[0147] The storage device 1604 is connected to the chip 1602 within the chip package structure via a bus and is used to store data. The storage device 1604 may include multiple sets of storage cells 1610. Each set of storage cells 1610 is connected to the chip 1602 via a bus. It can be understood that each set of storage cells 1610 may be DDR SDRAM (“Double Data Rate SDRAM”).

[0148] DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read on both the rising and falling edges of the clock pulse. DDR is twice as fast as standard SDRAM. In one embodiment, the memory device 1604 may include four groups of memory cells 1610. Each group of memory cells 1610 may include multiple DDR4 chips. In one embodiment, the chip 1602 may internally include four 72-bit DDR4 controllers, of which 64 bits are used for data transmission and 8 bits are used for ECC verification.

[0149] In one embodiment, each group of memory cells 1610 may include multiple Double Data Rate (DDR) synchronous dynamic random access memories (DDRs) arranged in parallel. DDRs can transfer data twice within one clock cycle. A controller for controlling the DDRs is provided in the chip 1602 for controlling the data transmission and data storage of each memory cell 1610.

[0150] The interface device 1606 is electrically connected to the chip 1602 within the chip package structure. The interface device 1606 is used to realize data transmission between the chip 1602 and an external device 1612 (e.g., a server or computer). For example, in one embodiment, the interface device 1606 can be a standard PCIe interface. For instance, data to be processed is transferred from the server to the chip 1602 via a standard PCIe interface, realizing data transfer. In another embodiment, the interface device 1606 can also be other interfaces; this disclosure does not limit the specific form of the other interfaces mentioned above, as long as the interface unit can realize the switching function. Furthermore, the calculation results of the chip 1602 are still transmitted back to the external device (e.g., the server) by the interface device 1606.

[0151] The controller 1608 is electrically connected to the chip 1602 to monitor the status of the chip 1602. Specifically, the chip 1602 and the controller 1608 can be electrically connected via an SPI interface. The controller 1608 may include a microcontroller (“MCU”). The chip 1602 may include multiple processing chips, multiple processing cores, or multiple processing circuits, and can drive multiple loads. Thus, the chip 1602 can operate in different states, such as high load and low load. The controller 1608 can regulate the operating status of multiple processing chips, multiple processing cores, and / or multiple processing circuits in the chip 1602.

[0152] In some embodiments, this disclosure also discloses an electronic device or apparatus that includes the aforementioned board 1600. Depending on the application scenario, the electronic device or apparatus may include a data processing device, robot, computer, printer, scanner, tablet computer, smart terminal, mobile phone, dashcam, navigator, sensor, camera, server, cloud server, camera, camcorder, projector, watch, earphone, mobile storage, wearable device, vehicle, home appliance, and / or medical device. The vehicle includes airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound scanners, and / or electrocardiographs.

[0153] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this disclosure. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0154] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, optical, acoustic, magnetic, or other forms.

[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] Furthermore, the functional units in the various embodiments disclosed herein can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0158] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, when the technical solution disclosed herein can be embodied in the form of a software product, the computer software product is stored in a storage device and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage device includes various media capable of storing program code, such as a USB flash drive, read-only memory ("ROM"), random access memory ("RAM"), portable hard drive, magnetic disk, or optical disk.

[0159] The foregoing can be better understood in accordance with the following terms:

[0160] Clause A1. A computing device for performing vector inner product operations, comprising:

[0161] A multiplication unit comprising one or more floating-point multipliers configured to perform element-wise multiplication on a received first vector and a second vector to obtain a product of corresponding vector elements for each pair, wherein each of the first and second vectors comprises one or more of the vector elements; and

[0162] An addition module is configured to perform an addition operation on the product of the corresponding vector elements of the first vector and the second vector to obtain a summation result.

[0163] Clause A2, the computing device according to Clause A1, further includes:

[0164] An update module is configured to perform multiple addition operations on the resulting intermediate results in response to the summation result being an intermediate result of the inner product operation, so as to output the final result of the inner product operation.

[0165] Clause A3. A computing device according to Clause A1 or A2, wherein the update module includes a second adder and a register, the second adder being configured to repeatedly perform the following operations until the addition operation on all of the plurality of intermediate results is completed:

[0166] Receive intermediate results from the addition module and the summation result from the previous addition operation from the register;

[0167] The intermediate result is added to the previous summation result to obtain the summation result of this addition operation; and

[0168] The result of this addition operation is used to update the previous summation result stored in the register.

[0169] Clause A4. The computing device according to Clause 1, wherein: after the multiplication unit outputs the product result, it receives the next pair of corresponding vector elements for multiplication; after the addition module outputs the summation result, it receives the next product result from the multiplication unit for addition.

[0170] Clause A5. The computing device according to any one of Clauses A1-A4 further comprises:

[0171] A first type conversion unit is configured to convert the data type of the product result so that the addition module can perform the addition operation.

[0172] Clause A6. The computing device according to any one of Clauses A1-A5, wherein the addition module comprises a multi-level adder group arranged in a multi-level tree structure, each level of adder group comprising one or more first adders.

[0173] Clause A7. The computing device according to any one of Clauses A1-A6 further includes one or more second type conversion units arranged in the multi-stage adder group, configured to convert data output from the first-stage adder group into another type of data for addition operations of the subsequent-stage adder group.

[0174] Clause A8. A computing device according to any one of clauses A1-A7, wherein the floating-point multiplier is used to perform floating-point multiplication operations according to an operation mode, wherein the corresponding vector elements of the first vector and the second vector include at least an exponent and a mantissa, and the floating-point multiplier comprises:

[0175] An exponent processing unit is configured to obtain the exponent after the multiplication operation based on the operation mode and the exponents of the corresponding vector elements of the first and second vectors; and

[0176] The mantissa processing unit is used to obtain the mantissa after the multiplication operation based on the operation mode and the corresponding vector elements of the first vector and the second vector.

[0177] The operation mode is used to indicate the data format of the corresponding vector elements of the first vector and the second vector.

[0178] Clause A9. The computing device according to Clause A8, wherein the operation mode is further used to indicate the data format after the multiplication operation.

[0179] Clause A10, the computing device as described in Clause A8, wherein the data format includes at least one of half-precision floating-point number, single-precision floating-point number, double-precision floating-point number, and custom floating-point number.

[0180] Clause A11, the computing apparatus according to Clause A8, wherein the corresponding vector elements of the first vector and the second vector further include signs, and the floating-point multiplier further includes:

[0181] The symbol processing unit is used to obtain the symbol after multiplication based on the symbols of the corresponding vector elements of the first vector and the second vector.

[0182] Clause A12. The computing device according to Clause A11, wherein the symbol processing unit includes an XOR logic circuit for performing an XOR operation on the symbols of the corresponding vector elements of the first vector and the second vector to obtain the symbol after the multiplication operation.

[0183] Clause A13, the computing device according to Clause A8, further includes:

[0184] The normalization processing unit is used to normalize the corresponding vector elements of the first vector and the second vector according to the operation mode when the corresponding vector elements of the first vector and the second vector are unnormalized non-zero floating-point numbers, so as to obtain the corresponding exponent and mantissa.

[0185] Clause A14. The computing apparatus according to Clause A8, wherein the mantissa processing unit includes a partial product operation unit and a partial product summation unit, wherein the partial product operation unit is used to obtain an intermediate result based on the mantissa of the corresponding vector elements of the first vector and the second vector, and the partial product summation unit is used to perform a summation operation on the intermediate result to obtain a summation result, and use the summation result as the mantissa after the multiplication operation.

[0186] Clause A15. The computing device according to Clause A14, wherein the partial product operation unit includes a Booth coding circuit, the Booth coding circuit being used to pad the high and low bits of the mantissa of the corresponding vector elements of the first vector or the second vector with 0 and to perform Booth coding processing to obtain the intermediate result.

[0187] Clause A16. The computing apparatus according to Clause A15, wherein the partial product summation unit includes an adder for summing the intermediate results to obtain the summed result.

[0188] Clause A17. The computing apparatus according to Clause A15, wherein the partial product summation unit includes a Wallace tree and an adder, wherein the Wallace tree is used to sum the intermediate results to obtain a second intermediate result, and the adder is used to sum the second intermediate results to obtain the summed result.

[0189] Clause A18. A computing device according to any one of Clauses A16-17, wherein the adder includes at least one of a full adder, a serial adder, and a carry-lookahead adder.

[0190] Clause A19. The computing device according to Clause A17, wherein when the number of intermediate results is less than M, zero values ​​are added as intermediate results, such that the number of intermediate results is equal to M, where M is a preset positive integer.

[0191] Clause A20, the computing device according to Clause A19, wherein each of the Wallace trees has M inputs and N outputs, the number of the Wallace trees is not less than K, where N is a preset positive integer less than M, and K is a positive integer not less than the maximum bit width of the intermediate result.

[0192] Clause A21. The computing apparatus according to Clause A20, wherein the partial product summation unit is used to select one or more groups of Wallace trees to sum the intermediate result according to the operation mode, wherein each group of Wallace trees has X Wallace trees, where X is the number of bits in the intermediate result, wherein there is a carry relationship between the Wallace trees within each group, but no carry relationship between the Wallace trees between groups.

[0193] Clause A22. A computing device according to any one of clauses A19-21, wherein the mantissa processing unit further includes a control circuit for calling the mantissa processing unit multiple times according to the operation mode when the operation module indicates that the mantissa bit width of at least one of the corresponding vector elements of the first vector or the second vector is greater than the data bit width that the mantissa processing unit can process at one time.

[0194] Clause A23. The computing device according to Clause A22, wherein the partial product summation unit further includes a shifter, wherein when the control circuit calls the mantissa processing unit multiple times according to the operation mode, the shifter is used in each call to shift the existing summation result and add it to the summation result obtained in the current call to obtain a new summation result, and the new summation result obtained in the last call is used as the mantissa after the multiplication operation.

[0195] Clause A24. The computing device according to Clause A23 further includes a regularization unit for:

[0196] The mantissa and exponent after the multiplication operation are subjected to floating-point number regularization processing to obtain regularized exponent result and regularized mantissa result, and the regularized exponent result and regularized mantissa result are used as the exponent and mantissa after the multiplication operation.

[0197] Clause A25, the computing device according to Clause A24, further includes:

[0198] The rounding unit is used to perform a rounding operation on the regularized mantissa result according to the rounding mode to obtain the rounded mantissa, and use the rounded mantissa as the mantissa after the multiplication operation.

[0199] Clause A26. The computing device described in Clause A8 further includes:

[0200] The mode selection unit is used to select, from the various operation modes supported by the floating-point multiplier, an operation mode that indicates the data format of the corresponding vector elements of the first vector and the second vector.

[0201] Clause A27. A method for performing vector dot product operation using a computing device according to any one of Clauses A1-A26, comprising:

[0202] The floating-point multiplier is used to perform multiplication operations on corresponding vector elements of the first and second vectors to obtain the product of corresponding vector elements for each pair; and

[0203] An addition operation is performed on the product of the corresponding vector elements of the first vector and the second vector to obtain a summation result.

[0204] Clause A28, an integrated circuit chip comprising a computing device as described in any one of A1-A26.

[0205] Clause A29, an integrated circuit device comprising the computing device described in any one of A1-A26.

[0206] The embodiments of this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

[0207] It should be understood that the terms "first," "second," "third," and "fourth," etc., in the claims, specification, and drawings of this disclosure are used to distinguish different objects, not to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0208] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0209] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0210] The embodiments of this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this disclosure. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of this disclosure, and on the specific implementation methods and application scope of this disclosure, are all within the scope of protection of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A computing device for performing vector dot product operations, comprising: A multiplication unit includes one or more floating-point multipliers configured to perform element-wise multiplication on a received first vector and a second vector to obtain a product of the corresponding vector elements of each pair, wherein the first vector and the second vector each include one or more of the vector elements. The floating-point multiplier includes a mantissa processing unit, which includes a partial product operation unit and a partial product summation unit. The partial product operation unit is used to obtain an intermediate result based on the mantissa of the corresponding vector elements of the first vector and the second vector. The partial product summation unit is used to perform a summation operation on the intermediate result to obtain a summation result, and uses the summation result as the mantissa after the multiplication operation. The partial product operation unit includes a Booth coding circuit, wherein the Booth coding circuit is used to pad the high and low bits of the mantissa of the corresponding vector elements of the first vector or the second vector with 0 and perform Booth coding processing to obtain the intermediate result, thereby reducing the number of summation terms of the intermediate result. The partial product summation unit includes a Wallace tree and an adder, wherein the Wallace tree is used to sum the intermediate results to obtain a second intermediate result, thereby reducing the number of times the intermediate results are accumulated, and the adder is used to sum the second intermediate results to obtain the summed result, and the summed result is used as the mantissa after the multiplication operation; The computing device further includes an addition module configured to perform an addition operation on the product of the corresponding vector elements of the first vector and the second vector to obtain a summation result.

2. The computing device according to claim 1, further comprising: An update module, configured to perform multiple addition operations on the resulting intermediate results in response to the summation result being an intermediate result of the inner product operation, to output the final result of the inner product operation.

3. The computing device of claim 2, wherein the update module includes a second adder and a register, the second adder being configured to repeatedly perform the following operations until the addition operation on all of the plurality of intermediate results is completed: Receive intermediate results from the addition module and the summation result from the previous addition operation from the register; The intermediate result is added to the previous summation result to obtain the summation result of this addition operation; as well as The result of this addition operation is used to update the previous summation result stored in the register.

4. The computing device according to claim 1, wherein: After the multiplication unit outputs the product result, it receives the next pair of corresponding vector elements and performs a multiplication operation; after the addition module outputs the summation result, it receives the next product result from the multiplication unit and performs an addition operation.

5. The computing device according to claim 1, further comprising: A first type conversion unit is configured to convert the data type of the product result so that the addition module can perform the addition operation.

6. The computing device according to claim 5, wherein the addition module comprises a multi-level adder group arranged in a multi-level tree structure, each level of adder group comprising one or more first adders.

7. The computing device of claim 6, further comprising one or more second type conversion units arranged in the multi-stage adder group, configured to convert data output from the first-stage adder group into another type of data for addition operations of a subsequent-stage adder group.

8. The computing device according to any one of claims 1-7, wherein the floating-point multiplier is used to perform floating-point multiplication operations according to an operation mode, wherein the corresponding vector elements of the first vector and the second vector include at least an exponent and a mantissa, and the floating-point multiplier further comprises: An exponent processing unit is used to obtain the exponent after the multiplication operation based on the operation mode and the exponents of the corresponding vector elements of the first vector and the second vector; wherein, a mantissa processing unit is used to obtain the mantissa after the multiplication operation based on the operation mode and the corresponding vector elements of the first vector and the second vector. The operation mode is used to indicate the data format of the corresponding vector elements of the first vector and the second vector.

9. The computing device according to claim 8, wherein the operation mode is further used to indicate the data format after the multiplication operation.

10. The computing device according to claim 8, wherein the data format includes at least one of half-precision floating-point number, single-precision floating-point number, double-precision floating-point number, and custom floating-point number.

11. The computing device of claim 8, wherein the corresponding vector elements of the first vector and the second vector further include signs, and the floating-point multiplier further includes: The symbol processing unit is used to obtain the symbol after multiplication based on the symbols of the corresponding vector elements of the first vector and the second vector.

12. The computing device according to claim 11, wherein the symbol processing unit includes an XOR logic circuit, the XOR logic circuit being configured to perform an XOR operation based on the symbols of the corresponding vector elements of the first vector and the second vector to obtain the symbol after the multiplication operation.

13. The computing device according to claim 8, further comprising: The normalization processing unit is used to normalize the corresponding vector elements of the first vector and the second vector according to the operation mode when the corresponding vector elements of the first vector and the second vector are unnormalized non-zero floating-point numbers, so as to obtain the corresponding exponent and mantissa.

14. The computing device of claim 1, wherein the adder comprises at least one of a full adder, a serial adder, and a carry-lookahead adder.

15. The computing device according to claim 1, wherein when the number of intermediate results is less than M, zero values ​​are added as intermediate results, such that the number of intermediate results is equal to M, where M is a preset positive integer.

16. The computing device of claim 15, wherein each of the Wallace trees has M inputs and N outputs, the number of Wallace trees is not less than K, where N is a preset positive integer less than M, and K is a positive integer not less than the maximum bit width of the intermediate result.

17. The computing apparatus of claim 16, wherein the partial product summation unit is configured to select one or more groups of Wallace trees to sum the intermediate result according to the operation mode, wherein each group of Wallace trees has X Wallace trees, where X is the number of bits in the intermediate result, wherein there is a carry relationship between the Wallace trees within each group, while there is no carry relationship between the Wallace trees between groups.

18. The computing device of claim 17, wherein the mantissa processing unit further comprises a control circuit for calling the mantissa processing unit multiple times according to the operation mode when the operation mode indicates that the mantissa bit width of at least one of the corresponding vector elements of the first vector or the second vector is greater than the data bit width that the mantissa processing unit can process at one time.

19. The computing device of claim 18, wherein the partial product summation unit further comprises a shifter, wherein when the control circuit calls the mantissa processing unit multiple times according to the operation mode, the shifter is used in each call to shift the existing summation result and add it to the summation result obtained in the current call to obtain a new summation result, and the new summation result obtained in the last call is used as the mantissa after the multiplication operation.

20. The computing device of claim 19, further comprising a regularization unit for: The mantissa and exponent after the multiplication operation are subjected to floating-point number regularization processing to obtain regularized exponent result and regularized mantissa result, and the regularized exponent result and regularized mantissa result are used as the exponent and mantissa after the multiplication operation.

21. The computing device according to claim 20, further comprising: The rounding unit is used to perform a rounding operation on the regularized mantissa result according to the rounding mode to obtain the rounded mantissa, and use the rounded mantissa as the mantissa after the multiplication operation.

22. The computing device according to claim 8, further comprising: The mode selection unit is used to select, from the various operation modes supported by the floating-point multiplier, an operation mode that indicates the data format of the corresponding vector elements of the first vector and the second vector.

23. A method for performing a vector inner product operation using a computing device according to any one of claims 1-22, comprising: The floating-point multiplier is used to perform multiplication operations on the corresponding vector elements of the first vector and the second vector to obtain the product of the corresponding vector elements of each pair. The floating-point multiplier includes a mantissa processing unit, which further includes a partial product operation unit and a partial product summation unit. The step of using the floating-point multiplier to perform multiplication operations on corresponding vector elements of the first and second vectors includes: The intermediate result is obtained by using the partial product operation unit based on the mantissa of the corresponding vector elements of the first vector and the second vector; The intermediate results are summed using the partial product summation unit to obtain a summed result, and the summed result is used as the last digit after the multiplication operation. The partial product operation unit includes a Booth coding circuit, and the step of obtaining intermediate results using the partial product operation unit based on the mantissas of the corresponding vector elements of the first vector and the second vector includes: The intermediate result is obtained by padding the mantissa of the corresponding vector elements of the first or second vector with 0s in both high and low bits using the Booth coding circuit and then performing Booth coding. The partial product summation unit includes a Wallace tree and an adder. The step of using the partial product summation unit to perform a summation operation on the intermediate results to obtain a summed result, and using the summed result as the mantissa after the multiplication operation, includes: The intermediate results are summed using the Wallace tree to obtain a second intermediate result, thereby reducing the number of times the intermediate results are accumulated. The second intermediate result is summed using an adder to obtain the summed result, and the summed result is used as the mantissa after the multiplication operation. The method further includes performing an addition operation on the product of the corresponding vector elements of the first vector and the second vector to obtain a summation result.

24. An integrated circuit chip comprising the computing device according to any one of claims 1-22.

25. An integrated circuit device comprising a computing device according to any one of claims 1-22.

Images