A dot product calculation device

Abstract

The present application provides a dot product operation device, which includes: a decoding module, a sign and exponent operation module, an exponent comparison and difference module, a mantissa multiplication module, an adder, a significant number alignment module, a sign processing and mantissa addition module, a mantissa normalization and exponent adjustment module, and an encoding and rounding module. Due to the use of a fusion operation, there is no need to perform decoding and encoding and rounding functions of intermediate results in the adder, resulting in a reduction in the number of decoding modules and encoding and rounding modules required for the present application, thereby reducing hardware area and hardware power consumption; and, due to the reduction in decoding modules and encoding and rounding modules, the critical path of the calculation is shortened, thereby reducing the calculation delay; further, since each encoding and rounding module rounds data according to the output precision limit when outputting data, the reduction in encoding and rounding modules leads to a reduction in the number of rounding times required in the entire calculation process, thereby improving the precision of the dot product result.

Description

Technical Field

[0001] The present application relates to the field of Posit dot product operations, and in particular to a dot product operation device. Background Art

[0002] Traditional single-precision floating-point formats provide good accuracy in most neural network models. However, as neural network models become more complex, the number of parameters and the amount of computation required also increase. For dot product operations in complex neural network models, the traditional single-precision floating-point format suffers from bit-width redundancy.

[0003] In order to avoid the problem of bit width redundancy, the low-bit-width Posit format is currently used for dot product operations in neural network models, where, for example, Figure 1a As shown, the Posit format is a data format including a sign bit 04, a regime field 05, an exponent segment 06, and a mantissa segment 07, where the regime field is a field consisting of consecutive identical bits and opposite bits.

[0004] The traditional device for implementing dot product operation includes multiple devices such as multiplier 01 and adder 02, and each multiplier 01 includes a sign and exponent operation module, an exponent comparison and difference module, a mantissa multiplication module, a significant number alignment module, a mantissa normalization and exponent adjustment module, an encoding and rounding module and two decoding modules; each adder 02 includes an exponent comparison and difference module, a significant number alignment module, a sign processing and mantissa addition module, a mantissa normalization and exponent adjustment module, an encoding and rounding module and two decoding modules, and each 2-input dot product unit 03 includes a sign and exponent operation module, an exponent comparison and difference module, a mantissa multiplication module, a significant number alignment module, a sign processing and mantissa addition module, a mantissa normalization and exponent adjustment module, an encoding and rounding module and four decoding modules. After receiving data, each device decodes the received data through its internal decoding module, processes the decoded data, and encodes the processed data through its encoding and rounding module, which then converts the processed data into Posit format. In other words, in the related art, each device in an apparatus implementing a dot product operation must decode the received data through its internal decoding module before performing data calculations. After the data calculations are completed, the device's internal encoding and rounding module converts the calculated data into Posit format.

[0005] like Figure 1bAs shown, taking the calculation process of Posit format data in multiplier 01 and adder 02 as an example, at least two groups of Posit format data are decoded by the decoding module in multiplier 01 to obtain the sign, exponent value and mantissa value of each group of data, wherein each group of data includes a multiplicand and a multiplier. For example: Figure 1b In the example, a0 and b0, a1 and b1, a2 and b2, and a3 and b3 are each a group of data. At least two groups of decoded data are multiplied by multiplier 01 to obtain a product of each group of decoded data. The products of the at least two groups of decoded data are encoded by the encoding and rounding module of multiplier 01 and rounded according to the precision limit of the output encoding result to obtain an encoding result of the product of each group of decoded data. The encoding result is in Posit format and is then output to adder 02. Adder 02 decodes the encoded results transmitted by multiplier 01 through its built-in decoding module, obtaining the sign, exponent value, and mantissa value of each encoded result. The decoded encoded results are then added together through adder 02 to obtain the sum of every two decoded encoded results. This sum is then encoded through the built-in encoding and rounding module of adder 02 and rounded according to the precision limit for the output encoded result to obtain data in Posit format, which is then output from adder 02. Each device processes the data in sequence to obtain a dot product result, which is in Posit format.

[0006] like Figure 1c As shown, taking the calculation process of Posit format data in the 2-input dot product unit 03 and the adder 02 as an example, at least two groups of Posit format data are decoded by the decoding module in the 2-input dot product unit 03 to obtain the sign, exponent value and mantissa value of each group of data, wherein each group of data includes two multiplicands and two multipliers. For example: Figure 1cIn the example, a0, b0, a1, and b1, and a2, b2, a3, and b3 are each a group of data. At least two groups of decoded data are multiplied and added together by a 2-input dot product unit 03 to obtain each group of decoded multiplied and added data. The at least two decoded multiplied and added data are then encoded by the encoding and rounding module of the 2-input dot product unit 03 and rounded according to the precision limit of the output encoding result to obtain encoding results of the at least two decoded multiplied and added data. The encoding results are in Posit format and are then output to the adder 02. Adder 02 decodes the encoded results transmitted by the two-input dot product unit 03 through its built-in decoding module, obtaining the sign, exponent value, and mantissa value of each of the encoded results. The decoded encoded results are then added together through adder 02 to obtain the sum of the two decoded encoded results. This sum is then encoded through the built-in encoding and rounding module of adder 02 and rounded according to the precision limit of the output encoded result to obtain data in Posit format, which is then used as the output of adder 02. Each device processes the data in sequence to obtain a dot product result, which is in Posit format.

[0007] However, in the above-mentioned process of implementing the dot product operation, since an encoding and rounding module and multiple decoding modules need to be set in each device, the entire dot product operation device needs to be provided with multiple decoding modules and encoding and rounding modules, thereby increasing the hardware area and hardware power consumption; and, since the decoding module and the encoding and rounding module are added, the critical path of the calculation is extended, thereby increasing the calculation delay; further, since each encoding and rounding module rounds the data according to the output precision limit when outputting the data, multiple roundings are required in the entire calculation process, thereby reducing the precision of the dot product result. Summary of the Invention

[0008] The present application provides a dot product operation device to solve the technical problems of increased hardware area and hardware power consumption, increased calculation delay and reduced accuracy of dot product results.

[0009] In order to solve the above technical problems, the embodiments of the present application disclose the following technical solutions:

[0010] In a first aspect, an embodiment of the present application discloses a dot product operation device, comprising:

[0011] 2n parallel decoding modules, each decoding module is used to determine the sign, exponent value, and mantissa value including the hidden value of the input data by decoding the input data in Posit format, the input data including the multiplicand and the multiplier, and n is a positive integer greater than or equal to 2;

[0012] n parallel sign and exponent operation modules, each of which is configured to obtain a sign and an exponent value of a set of input data transmitted by two of the parallel decoding modules, determine a sign obtained by performing an OR operation on the signs of the set of input data, and determine a sum of the exponent values ​​of the set of input data, each set of input data including a multiplicand and a multiplier;

[0013] An exponential comparison and difference module is used to obtain the sum of the exponential values ​​of the input data and determine the exponential maximum and n exponential differences. The exponential maximum is the maximum value in the sum of the exponential values, and each exponential difference is the difference between the exponential maximum and the sum of the exponential values ​​of a set of input data;

[0014] n mantissa multiplication modules, each mantissa multiplication module being configured to obtain a mantissa value including a hidden value for a set of input data transmitted by two parallel decoding modules, determine a partial sum of the mantissa values ​​including the hidden value for the set of input data, sign-extend the partial sum, and determine a sum value and a carry value obtained by compressing the sign-extended partial sum;

[0015] n+1 adders, wherein n adders are used to obtain each group of sum values ​​and carry values ​​obtained by each mantissa multiplication module, and another adder is used to obtain a group of sum values ​​and carry values ​​obtained by the sign processing and mantissa addition module, and each adder adds each group of sum values ​​and carry values ​​obtained to obtain the sum of each group of sum values ​​and carry values;

[0016] The significand alignment module is used to obtain n exponent differences and determine the product of the mantissa values ​​of the n sets of aligned input data containing the hidden value;

[0017] a sign processing and mantissa addition module, configured to obtain a product of mantissa values ​​containing hidden values ​​in the n aligned sets of input data, determine a set of sum values ​​and a carry value obtained by compressing the product of the mantissa values ​​in the n aligned sets of input data in two's complement form, obtain a sum of the set of sum values ​​and the carry value in two's complement form, and determine a sign of the sum of the set of sum values ​​and the carry value in two's complement form as a final sign;

[0018] a mantissa normalization and exponent adjustment module, configured to obtain the sum of a set of sum values ​​and a carry value obtained by the sign processing and mantissa addition module, determine a final mantissa value obtained by shifting the decimal point of the sum of the set of sum values ​​and the carry value in the original code form, and determine a final exponent value obtained by adjusting the maximum value of the exponent, wherein the decimal point in the final mantissa value is located after the hidden bit 1;

[0019] The encoding and rounding module is used to determine the output value of the dot product operation in the Posit format by encoding the final sign, the final exponent value and the final mantissa value after rounding.

[0020] Optionally, the exponential comparison and difference module includes:

[0021] a comparator tree module configured to determine a tree shape of the comparator tree module that matches the number of groups of sums of exponential values ​​of the n groups of input data, and to determine a maximum value of the sums of the exponential values ​​of the n groups of input data according to the tree shape of the comparator tree module as the exponential maximum value;

[0022] The exponential difference module is used to determine n exponential differences, each exponential difference being the difference between the maximum exponential value and the sum of the exponential values ​​of a set of input data.

[0023] Optionally, the comparator tree module is a recursive structure comparator tree module, and the comparator tree module includes at least one comparator.

[0024] Optionally, the mantissa multiplication module includes:

[0025] a partial sum generation module for determining a first product based on the Booth coded value of the mantissa of the multiplier in the set of input data, the first product being the product of a shifted value of the mantissa value of the multiplicand in the set of input data and the Booth coded value of the mantissa of the multiplier, and determining the first product as a partial sum of the mantissa values ​​in the set of input data, the Booth coded value of the mantissa of the multiplier being a value obtained by radix-4 Booth recoding the mantissa value of the multiplier;

[0026] The first Wallace tree module is used to determine a sum value and a carry value obtained by compressing the partial sum after the sign bit is expanded.

[0027] Optionally, the first Wallace tree module is a recursive structure Wallace tree module, and the first Wallace tree module includes at least one 3:2 compressor or at least one 4:2 compressor.

[0028] Optionally, the mantissa multiplication module also includes:

[0029] A sign bit extension module is used to perform sign bit extension on the high bits of the partial sum of the mantissa values ​​in the input data based on the sign of the partial sum of the mantissa values ​​in the input data, which is the sign of the mantissa Booth coded value of the multiplier in the input data.

[0030] Optionally, the bit width of the mantissa value in the input data set by the decoding module is smaller than the bit width of the mantissa value in the output data set by the encoding and rounding module, the bit width of the mantissa value in the input data is the input data precision set by the decoding module, and the bit width of the mantissa value in the output data is the output data precision set by the encoding and rounding module.

[0031] Optionally, the bit width of the mantissa value set by the significand alignment module is greater than the bit width of the mantissa value in the output data set by the encoding and rounding module.

[0032] Optional, valid number alignment module includes:

[0033] The mantissa discarding module is used to discard the low-order values ​​of the product of the mantissa values ​​in the aligned n groups of input data that exceed the set mantissa value bit width range.

[0034] Optionally, the significand alignment module is a barrel shifter.

[0035] The beneficial effects of this application are:

[0036] A fusion design is adopted, that is, the input data in the Posit format is decoded into a sign, an exponent value and a mantissa value in the decoding module, and all operations are completed inside the sign and exponent operation module, the exponent comparison and difference module, the mantissa multiplication module, the adder, the valid number alignment module, the sign processing and mantissa addition module, the mantissa normalization and exponent adjustment module to obtain the final calculation result, and then the result is converted into an output format through the encoding and rounding module. Due to the adoption of the fusion operation, there is no need to perform the decoding and encoding and rounding functions of the intermediate results in the adder, resulting in a reduction in the number of decoding modules and encoding and rounding modules required for this application, thereby reducing the hardware area and hardware power consumption; and, due to the reduction of the decoding modules and encoding and rounding modules, the critical path of the calculation is shortened, thereby reducing the calculation delay; further, since each encoding and rounding module rounds according to the output precision limit when outputting data, the reduction of the encoding and rounding modules leads to a reduction in the number of rounding times required in the entire calculation process, thereby improving the accuracy of the dot product result.

[0037] Since the decoding module itself supports decoding of any Posit input format, and the encoding and rounding module supports its own arbitrary Posit output format, the present application supports any input and output format; the decoding module, the sign and exponent operation module, and the mantissa multiplication module are all set in parallel. If N data are input, 2N parallel decoding modules, N parallel sign and exponent operation modules, and N parallel mantissa multiplication modules are called, then the present application achieves the function of supporting any dot product scale N; since the present application supports any input and output format and supports any dot product scale, the range of neural network models adapted by the present application is expanded.

[0038] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0040] Figure 1a This is a schematic diagram of the components of data in Posit format in the prior art;

[0041] Figure 1b Schematic diagram of the structure of a device for implementing dot product operation of data in Posit format in the prior art;

[0042] Figure 1c Schematic diagram of another structure of a device for implementing dot product operation of data in Posit format in the prior art;

[0043] Figure 2 A schematic diagram of the structure of a dot product operation device provided in an embodiment of the present application;

[0044] Figure 3a A schematic diagram of a process for calculating the number of consecutive 0s or 1s in a regime field provided in an embodiment of the present application;

[0045] Figure 3b Provided in the embodiments of this application Figure 3a Schematic diagram of the partial process of implementing the leading 0 / 1 counting module;

[0046] Figure 4a A schematic diagram of the comparator module structure when the number of input data items provided in an embodiment of the present application is 5;

[0047] Figure 4b A schematic diagram of the comparator module structure when the number of input data items provided in an embodiment of the present application is 6;

[0048] Figure 4c A schematic diagram of the comparator module structure when the number of input data items provided in an embodiment of the present application is 8;

[0049] Figure 5 A schematic diagram illustrating the principle of implementing the accumulation of binary numbers of any scale using the first Wallace tree module provided in an embodiment of the present application;

[0050] Figure 6a A schematic diagram of the circuit structure of a 3:2 compressor provided in an embodiment of the present application;

[0051] Figure 6b A schematic diagram of the circuit structure of a 3:2 carry-save adder provided in an embodiment of the present application;

[0052] Figure 6c A schematic diagram of the circuit structure of a 4:2 compressor provided in an embodiment of the present application;

[0053] Figure 6d A schematic diagram of the circuit structure of a 5:3 counter provided in an embodiment of the present application;

[0054] Figure 7aA schematic diagram of the sign bit extension of not all bits provided in an embodiment of the present application;

[0055] Figure 7b A schematic diagram of traditional sign bit expansion provided in an embodiment of the present application;

[0056] Figure 8 This is a schematic diagram of the circuit structure of the barrel shifter provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0058] To facilitate the technical solution of the application, some concepts involved in this application are first explained below.

[0059] The Posit format is determined by the total bit width n and the exponent bit width es, and consists of four parts: the sign bit 04 (1 bit), the regime field 05 (variable bit width), the exponent segment 06 (0 to es bits), and the mantissa segment 07 (the remaining bits). Among them, the sign bit 04 is represented by s, the regime field 05 consists of consecutive identical bits r and opposite bits. The bit width changes dynamically according to the size of the real value; the exponent segment 06 and the mantissa segment 07 are encoded after the regime field 05, and the exponent segment 06 uses e1e2e 3... e es Indicates that the tail number segment 07 uses m1m2m 3... Indicates that, since the total bit width of the Posit format is n, bits exceeding the total bit width will overflow.

[0060] Data in Posit format can be expressed by the following formula:

[0061]

[0062] Among them, s is a positive integer used to determine the sign of the sign bit 04, es is the size of the theoretical exponent bit width (the actual exponent segment 06 bit width is less than or equal to es), e p is the unsigned exponent encoded by exponent segment 06, m pis the mantissa value of mantissa field 07, k is the value of regime field 05, and otherwise means that except for the two special values ​​"000…000" and "100…000" representing ±0 and ±∞ respectively, all other cases are decoded according to the above formula.

[0063] When the value of the sign bit 04 is 0, that is, s in the above formula is 0, the value is a positive number; when the value of the sign bit 04 is 1, that is, s in the above formula is 1, the value is a negative number.

[0064] If the length of consecutive identical bits in the regime field 05 is runlength, the value of k is defined as follows:

[0065]

[0066] In some embodiments, a 4-bit binary number is used as the regime field 05, and the k value of the regime field 05 is shown in Table 1 below:

[0067] Table 1 Examples of k values ​​for the regime field in the Posit format

[0068] Binary 1111 1110 110x 10xx 01xx 001x 0001 0000 Regime, k 3 2 1 0 -1 -2 -3 -4

[0069] Here, Binary represents a binary number, and x represents the value of a bit in the binary number that does not affect the size of k.

[0070] See also Figure 2 , an embodiment of the present application provides a dot product operation device, comprising:

[0071] 2n parallel decoding modules, n parallel sign and exponent operation modules, exponent comparison and difference modules, n mantissa multiplication modules, n+1 adders, significant number alignment module, sign processing and mantissa addition module, mantissa normalization and exponent adjustment module and encoding and rounding module, among which,

[0072] Every two parallel decoding modules are connected to one sign and exponent operation module and one mantissa multiplication module. The sign and exponent operation module is connected to the significand alignment module through the exponent comparison and difference module; the mantissa multiplication module is connected to the significand alignment module through the adder. The sign and exponent operation module and the significand alignment module are both connected to the sign processing and mantissa addition module. The sign processing and mantissa addition module is connected to the mantissa normalization and exponent adjustment module through the adder; the exponent comparison and difference module is connected to the mantissa normalization and exponent adjustment module. The adder close to the sign processing and mantissa addition module and the mantissa normalization and exponent adjustment module are both connected to the encoding and rounding module. Wherein, Figure 2 a0 and b0, a1 and b1, ...a N-2 and b N-2 、a N-1 and b N-1 They are respectively a set of input data in Posit format.

[0073] 2n parallel decoding modules, each decoding module is used to determine the sign, exponent value and mantissa value containing the hidden value of the input data by decoding the input data in Posit format, the input data includes a multiplicand and a multiplier, and n is a positive integer greater than or equal to 2.

[0074] In some embodiments, when the decoding module determines the exponent value and the mantissa value including the hidden value, it is necessary to first determine the bit width of the variable-width regime field 05 .

[0075] In some embodiments, as Figure 3a As shown, the leading 0 / 1 counting module outputs runlength to determine the bit width of the regime field 05. The bit width of the regime field 05 is runlength+1. The leading 0 / 1 counting module includes a leading 0 counting module and a leading 1 counting module. x is a Posit format input data, n is the total bit width of x, x[n-1] is the sign bit 04, x[n-2:0] is the remaining bits in x except the sign bit 04, and runlength is the number of consecutive 0s or 1s in the regime field 05.

[0076] In some embodiments, as Figure 3a and Figure 3b As shown, in order to avoid the increase in hardware caused by calling the leading 0 counting module and the leading 1 counting module at the same time, when the number of consecutive 1s needs to be calculated, the data to be calculated for consecutive 1s is inverted and then the number of consecutive 0s is obtained through the leading 0 counting module. The number of consecutive 0s is the same as the number of consecutive 1s. Figure 3a The output data of the 2-way selector, the Figure 3a The bit width of the output data of the 2-way selector is n-1, and y[n-2] is the Figure 3a The highest bit of the output data of the 2-way selector, y[n-2:0] is all bits of y, that is, Figure 3a The output data of the 2-way selector. When y[n-2]=0, Figure 3b The 2-way selector outputs y[n-2:0]; when y[n-2]=1, Figure 3b The 2-way selector outputs the value after all bits of y[n-2:0] are inverted, and the runlength is the number of consecutive 0s or 1s in the regime field 05.

[0077] In some embodiments, if x is an 8-bit Posit format data, that is, "10011101", then x[n-1] is 1, x[n-2:0] is "0011101", Figure 3a The output value of the 2-way selector is "1100011", the value of y is "1100011", y[n-2] is 1, and y[n-2:0] is "1100011". Figure 3b The output value of the 2-way selector is "0011100", which enters the leading zero counting module to determine the value of runlength, that is, the number of consecutive 0s or 1s in the regime field 05.

[0078] In some embodiments, after determining the bit width of the regime field 05, the remaining n-1-(runlength+1) bits store the exponent segment 06, i.e., e p 、The last digit is 07, which is m p , the calculation formula for the k value is as follows:

[0079]

[0080] Where k is the value of the regime field 05, runlength is the length of consecutive identical bits in the regime field 05, and r is the consecutive identical bits in the regime field 05.

[0081] The calculation formulas for the exponent value and the mantissa value including the hidden value are as follows:

[0082]

[0083] Wherein, exponent represents the exponential value, k represents the value of the regime field 05, es represents the theoretical bit width of the exponential segment 06, e p Indicates index segment 06, m p Represents the mantissa segment 07, and mantissa represents the mantissa value containing the hidden value.

[0084] n parallel sign and exponent operation modules, each of which is used to obtain the sign and exponent value of a group of input data transmitted by two parallel decoding modules, determine the sign obtained after performing an OR operation on the signs of the group of input data, and determine the sum of the exponent values ​​of the group of input data, each group of input data includes a multiplicand and a multiplier.

[0085] The exponential comparison and difference module is used to obtain the sum of the exponential values ​​of the input data and determine the exponential maximum and n exponential differences. The exponential maximum is the maximum value in the sum of the exponential values, and each exponential difference is the difference between the exponential maximum and the sum of the exponential values ​​of a set of input data.

[0086] In some embodiments, the exponential comparison and difference module includes:

[0087] a comparator tree module configured to determine a tree shape of the comparator tree module that matches the number of groups of sums of exponential values ​​of the n groups of input data, and to determine a maximum value of the sums of the exponential values ​​of the n groups of input data according to the tree shape of the comparator tree module as the exponential maximum value;

[0088] The exponential difference module is used to determine n exponential differences, each exponential difference being the difference between the maximum exponential value and the sum of the exponential values ​​of a set of input data.

[0089] In some embodiments, the comparator tree module is a recursive structure comparator tree module, which includes at least one comparator, wherein the comparator is used to compare two data items to determine whether the two data items are equal and the size relationship between the two data items.

[0090] In some embodiments, the comparator tree module constructs different tree structures according to the size of the input data item, such as Figure 4a 、 Figure 4b and Figure 4c As shown: when the scale N of the input data items is greater than 2, the N data items are split into two parts, N1=N / 2 and N2=N-N1, and the comparator tree module itself is re-called to obtain the maximum value of the two parts respectively, and then the larger value of the two intermediate results is output through a comparator, wherein the scale of the data items is the number of the data items.

[0091] n mantissa multiplication modules, each mantissa multiplication module is used to obtain a mantissa value containing a hidden value of a group of input data transmitted by two parallel decoding modules, determine a partial sum of the mantissa values ​​containing the hidden value of the group of input data, perform sign bit extension on the partial sum, and determine a sum value and a carry value obtained after compressing the partial sum after the sign bit extension.

[0092] In some embodiments, the mantissa multiplication module includes:

[0093] a partial sum generation module for determining a first product based on the Booth coded value of the mantissa of the multiplier in the set of input data, the first product being the product of a shifted value of the mantissa value of the multiplicand in the set of input data and the Booth coded value of the mantissa of the multiplier, and determining the first product as a partial sum of the mantissa values ​​in the set of input data, the Booth coded value of the mantissa of the multiplier being a value obtained by radix-4 Booth recoding the mantissa value of the multiplier;

[0094] The first Wallace tree module is used to determine a sum value and a carry value obtained by compressing the partial sum after the sign bit is expanded.

[0095] In some embodiments, the mantissa of the multiplier in a set of input data is re-encoded according to a radix-4 Booth encoding table to obtain a Booth encoding value of the mantissa of the multiplier. The radix-4 Booth encoding table is as follows:

[0096] Table 2 Examples of k values ​​for the regime field in the Posit format

[0097] <![CDATA[B i+1 B i B i-1 ]]> <![CDATA[-2B i+1 +B i +B i-1 ]]> 000 0 001 +1 010 +1 011 +2 100 -2 101 -1 110 -1 111 0

[0098] Among them, B i+1 B i B i-1 The adjacent 3-bit number in the mantissa of the multiplier, -2B i+1 +B i +B i-1 The Booth coded value of the mantissa corresponding to the re-encoded adjacent 3-bit number in the mantissa of the multiplier.

[0099] In some embodiments, taking the mantissa of the multiplier as 1100101100110110 as an example, in order to perform radix-4 Booth recoding on the mantissa of the multiplier, the mantissa of the multiplier is expanded into 9 groups of adjacent 3-bit numbers, which are 001, 110, 001, 101, 110, 001, 110, 011, and 100 from high to low, and the 9 Booth code values ​​of the mantissa corresponding to the 9 groups of adjacent 3-bit numbers are 1, -1, 1, -1, -1, 1, -1, 2, and -2.

[0100] In some embodiments, performing a shift process on a mantissa value of a multiplicand in a set of input data includes:

[0101] If the absolute value of the mantissa Booth coded values ​​corresponding to the adjacent 3 bits in the mantissa of the multiplier is 2, the multiplicand is shifted left by 1 bit. If the absolute value of the mantissa Booth coded values ​​corresponding to the adjacent 3 bits in the mantissa of the multiplier is 1, the multiplicand does not need to be shifted. If the absolute value of the mantissa Booth coded values ​​corresponding to the adjacent 3 bits in the mantissa of the multiplier is 0, the product of the mantissa Booth coded values ​​corresponding to the adjacent 3 bits in the mantissa of the multiplier and the multiplicand is 0, that is, the partial sum of the group is 0. If the mantissa Booth coded value is a negative number, the product of the mantissa Booth coded values ​​corresponding to the adjacent 3 bits in the mantissa of the multiplier and the multiplicand is obtained by taking the two's complement (i.e., inverting +1) of the product of the mantissa Booth coded values ​​corresponding to the adjacent 3 bits in the mantissa of the multiplier and the multiplicand to obtain a partial sum.

[0102] In some embodiments, the first Wallace tree module is a recursive structure Wallace tree module, and the first Wallace tree module includes at least one 3:2 compressor or at least one 4:2 compressor.

[0103] In some embodiments, the first Wallace tree module constructs different tree structures according to the scale of the input data, and compresses the input data into a sum value (s, sum) and a carry value (c, carry) as output, and can complete the accumulation of binary numbers of arbitrary bit width and scale, wherein the scale of the input data is the number of the input data.

[0104] The first Wallace tree module implements the principle of binary number accumulation of any size, such as Figure 5 As shown, when the scale N of the input data is less than 3, the input is directly assigned to the output; when N = 3 or 4, a 3:2 compressor or a 4:2 compressor will be called respectively to complete the input compression function; when N ≥ 5, the input data of scale N is split into two smaller parts: N1 = N / 2 and N2 = N-N1, each of which will recursively call the first Wallace tree itself. The input data of scale N1 will produce the corresponding sum value s1 and carry value c1, and the input data of scale N2 will produce the corresponding sum value s2 and carry value c2, and finally the input compression function is completed by the 4:2 compressor. Wherein, N is the scale of the input data, that is, the number of input data, x0, x1, ... x N-1 Both are input data, s is the output sum value, and c is the output carry value.

[0105] In some embodiments, as Figure 6a As shown, the 3:2 compressor is composed of multiple Figure 6b The 3:2 carry-save adder (CSA) shown in the figure is constructed. The circuit implementation of the 3:2 carry-save adder is similar to that of a full adder. When the bit width of the input data is n, the 3:2 compressor uses n 3:2 carry-save adders in parallel to complete the accumulation of binary numbers of any bit width. n-1 are the values ​​of the corresponding bits of the input data x, y0, y1, ...y n-1 are the values ​​of the corresponding bits of the input data y, z0, z1, ...z n-1 are the values ​​of the corresponding bits of the input data z; c0, c1, ... c n are the values ​​of the corresponding bits of the output carry value c, s0, s1, ...s n-1 are the values ​​of the corresponding bits of the output and value s respectively.

[0106] like Figure 6c As shown, the 4:2 compressor is composed of multiple 5:3 counters, and the circuit implementation of the 5:3 counter is as follows Figure 6dAs shown, when the input data bit width is n, the 4:2 compressor cascades n 5:3 counters to complete the accumulation of binary numbers of any bit width. Figure 6c a0, a1, ...a n-1 are the values ​​of the corresponding bits of the input data a, b0, b1, ...b n-1 are the values ​​of the corresponding bits of the input data b, c0, c1, ...c n-1 are the values ​​of the corresponding bits of the input data c, d0, d1, ...d n-1 are the corresponding bit values ​​of the input data d, c o0 ,c o1 ,...c o(n-1) are the values ​​of the bits corresponding to the internally generated carry signals, c0, c1, ...c output from the 4:2 compressor. n ,c out They are the values ​​of the corresponding bits of the output carry value carry, s0, s1, ...s n-1 They are the values ​​of the corresponding bits of the output and the value sum respectively.

[0107] In some embodiments, the mantissa multiplication module further comprises:

[0108] A sign bit extension module is used to perform sign bit extension on the high bits of the partial sum of the mantissa values ​​in the input data based on the sign of the partial sum of the mantissa values ​​in the input data, which is the sign of the mantissa Booth coded value of the multiplier in the input data.

[0109] In some embodiments, as Figure 7a As shown in the figure, the high bits of the partial sum of the mantissa values ​​in the input data are extended by the sign bit of non-all bits. The black dots represent the multiplicands, s0, s1, ... s7 represent the signs of the partial sums of each group, and the horizontal line above indicates that the bit is inverted. Figure 7b Compared with the traditional sign bit extension method shown, a finer-grained control is performed, that is, the corresponding sign bit extension is performed through the sign of each group of partial sums (s0, s1, ... s7), rather than the sign bit extension of all bits; since the number of sign bits extended by the sign bit extension of not all bits is small, the addition between the partial sums is reduced when the partial sums after sign bit expansion are compressed. Since for each bit, it is flipped once every time 1 is added, the flipping is reduced, thereby reducing the power consumption caused by the flipping.

[0110] n+1 adders, n of which are used to obtain each group of sum values ​​and carry values ​​obtained by each mantissa multiplication module, and another adder is used to obtain a group of sum values ​​and carry values ​​obtained by the sign processing and mantissa addition module. Each adder adds each group of sum values ​​and carry values ​​obtained to obtain the sum of each group of sum values ​​and carry values.

[0111] The significant number alignment module is used to obtain n exponent differences and determine the product of the mantissa values ​​containing the hidden value in the aligned n groups of input data.

[0112] In some embodiments, the bit width of the mantissa value set by the significand alignment module is greater than the bit width of the mantissa value in the output data set by the encoding and rounding module.

[0113] In some embodiments, the significand alignment module includes:

[0114] The mantissa discarding module is used to discard the low-order values ​​of the product of the mantissa values ​​in the aligned n groups of input data that exceed the set mantissa value bit width range.

[0115] In some embodiments, the significand alignment module is a barrel shifter.

[0116] In some embodiments, as Figure 8 As shown, input is the output of the mantissa multiplication module, s is the value that needs to be shifted, n is the binary bit width of s, s[0] represents the value of the 0th bit of s, s[1] represents the value of the 1st bit of s, s[n-1] represents the value of the n-1th bit of s, and output is the value of the shifted output of the mantissa multiplication module. Taking s=110 as an example, that is, s[2]=1, s[1]=1, s[0]=0, then n=3, according to Figure 8 As shown from left to right in the figure, input is first shifted right by 4 bits, then by 2 bits, and s[0]=0 does not continue to shift right, for a total of 6 bits shifted right, that is, output=input>>6.

[0117] A sign processing and mantissa addition module is used to obtain the product of the mantissa values ​​of the n aligned groups of input data containing hidden values, determine a set of sum values ​​and carry values ​​obtained by compressing the product of the mantissa values ​​in the n aligned groups of input data in complement form, obtain the sum of the group of sum values ​​and carry values ​​in complement form, and determine the sign of the sum of the group of sum values ​​and carry values ​​in complement form as the final symbol, wherein the sum value and carry value are a set of the sum value and carry value obtained by compressing the product of the mantissa values ​​in the n aligned groups of input data in complement form.

[0118] In some embodiments, the sign processing and mantissa addition module obtains the product of the mantissa values ​​containing hidden values ​​in the aligned n groups of input data, converts the product of the mantissa values ​​containing hidden values ​​in the aligned n groups of input data from binary original code form to binary complement form, and determines a set of sum values ​​and carry values ​​obtained after compressing the product of the mantissa values ​​in the aligned n groups of input data in binary complement form.

[0119] The mantissa normalization and exponent adjustment module is used to obtain the sum of a set of sum values ​​and carry values ​​obtained by the sign processing and mantissa addition module, determine the final mantissa value obtained after shifting the decimal point in the sum of a set of sum values ​​and carry values ​​in the original code form, and determine the final exponent value obtained after adjusting the maximum value of the exponent, and the decimal point in the final mantissa value is located after the hidden bit 1.

[0120] In some embodiments, the mantissa normalization and exponent adjustment module obtains a set of sum values ​​and carry values ​​obtained by the sign processing and mantissa addition module, converts the obtained set of sum values ​​and carry values ​​from the binary complement form to the binary original code form, and determines the final mantissa value obtained after shifting the decimal point in the sum of the set of sum values ​​and carry values ​​in the original code form.

[0121] The encoding and rounding module is used to determine the output value of the dot product operation in the Posit format by encoding the final sign, the final exponent value and the final mantissa value after rounding.

[0122] In some embodiments, the encoding and rounding module uses the rounding to nearest (RNE) mode to obtain the final mantissa value after rounding. Since the regime field 05 has a variable bit width, the encoding and rounding module must determine the actual bit width of the regime field 05 based on the final exponent value. According to the Posit format encoding rules, except for two special values ​​(0 and infinity), in other cases, the final exponent value exponent and the value k of the regime field 05 and the encoded exponent e p The relationship is as follows:

[0123]

[0124] Among them, exponent is the final exponent value, k is the value of regime field 05, e p is the encoded exponent, and es0 is the exponent width.

[0125] Therefore, k and e can be obtained p as follows:

[0126] k=exponent>>>es o

[0127]

[0128] Among them, es0 is the exponent bit width, >>> represents arithmetic shift, e p Divide the final index value by The remainder, exponent[es0-1:0] represents the low es0 bit of exponent in binary representation.

[0129] After obtaining k, we can infer from Table 1 that the length of the regime field 05 is runlength+1. Therefore, the total bit width of the sign bit 04, the regime field 05, and the exponent segment 06 is n1=2+runlength+es0. If no overflow occurs, the bit width of the remaining mantissa is n-n1.

[0130] In some embodiments, assuming es0=2, exponent=18, its binary complement representation is "010010", then according to the formula, k=010010>>>2=000100=4, and e p =exponent[1:0]="10"=2.

[0131] In some embodiments, the bit width of the mantissa value in the input data set by the decoding module is smaller than the bit width of the mantissa value in the output data set by the encoding and rounding module. The bit width of the mantissa value in the input data is the input data precision set by the decoding module, and the bit width of the mantissa value in the output data is the output data precision set by the encoding and rounding module. Because the decoding module itself supports any Posit input format, and the encoding and rounding module itself supports any Posit output format, when the precision of the output data is higher than the precision of the input data, the dot product operation device provided in this application supports mixed-precision calculations of input and output formats, effectively improving the efficiency of the dot product operation.

[0132] In some embodiments, the calculation process of a dot product operation device provided by the present application is divided into N different stages, which is equivalent to dividing the critical path into N segments, and can reduce the clock cycle to 1 / N of the original, thereby greatly improving the calculation frequency, increasing the processing speed of the operation unit for input data, and achieving the purpose of high throughput. N is a positive integer greater than 1, and N can be optionally 5.

[0133] Since the above embodiments are all described by reference in combination with other embodiments, different embodiments have the same parts, and the same and similar parts between the various embodiments in this specification can be referred to each other. No further detailed explanation is given here.

[0134] It should be noted that, in this specification, relational terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such circuit structure, article or device. In the absence of further restrictions, the presence of an element defined by the phrase "includes a..." does not exclude the presence of other identical elements in the circuit structure, article or device comprising the element.

[0135] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the disclosure of the invention herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0136] The above-described embodiments of the present application do not constitute a limitation on the scope of protection of the present application.

Claims

1. A dot product operation device, characterized in that: include: 2n parallel decoding modules, each of the decoding modules being configured to determine a sign, an exponent value, and a mantissa value including a hidden value of the input data by decoding input data in a Posit format, the input data including a multiplicand and a multiplier, where n is a positive integer greater than or equal to 2; n parallel sign and exponent operation modules, each of the sign and exponent operation modules being configured to obtain a sign and an exponent value of a set of input data transmitted by two of the parallel decoding modules, determine a sign obtained by performing an OR operation on the signs of the set of input data, and determine a sum of the exponent values ​​of the set of input data, each set of input data including a multiplicand and a multiplier; an exponential comparison and difference module, configured to obtain the sum of the exponential values ​​of the input data, and determine an exponential maximum value and n exponential differences, wherein the exponential maximum value is the maximum value in the sum of the exponential values, and each exponential difference is the difference between the exponential maximum value and the sum of the exponential values ​​of a group of the input data; n mantissa multiplication modules, each mantissa multiplication module being configured to obtain a mantissa value including a hidden value from a set of input data transmitted by two of the parallel decoding modules, determine a partial sum of the mantissa values ​​including the hidden value from the set of input data, sign-extend the partial sum, and determine a sum value and a carry value obtained by compressing the sign-extended partial sum; n+1 adders, wherein n adders are used to obtain each group of the sum values ​​and the carry values ​​obtained by each mantissa multiplication module, and another adder is used to obtain a group of the sum values ​​and the carry values ​​obtained by the sign processing and mantissa addition module, and each adder adds the sum values ​​and the carry values ​​in each group obtained to obtain the sum of the sum values ​​and the carry values ​​in each group; a significant number alignment module, configured to obtain n exponent differences and determine the product of mantissa values ​​of n aligned groups of input data containing hidden values; a sign processing and mantissa addition module, configured to obtain a product of mantissa values ​​of the n aligned groups of input data containing hidden values, determine a set of sum values ​​and a carry value obtained by compressing the product of the mantissa values ​​of the n aligned groups of input data in two's complement form, obtain a sum of the set of sum values ​​and the carry value in two's complement form, and determine a sign of the sum of the set of sum values ​​and the carry value in two's complement form as a final sign; a mantissa normalization and exponent adjustment module, configured to obtain a sum of a set of sum values ​​and a carry value obtained by the sign processing and mantissa addition module, determine a final mantissa value obtained by shifting a decimal point in the sum of the set of sum values ​​and the carry value in original code form, and determine a final exponent value obtained by adjusting the maximum value of the exponent, wherein the decimal point in the final mantissa value is located after the hidden bit 1; The encoding and rounding module is used to determine the output value of the dot product operation in the Posit format by encoding the final sign, the final exponent value and the final mantissa value after rounding.

2. The dot product operation device according to claim 1, wherein: The exponential comparison and difference module includes: a comparator tree module configured to determine a tree shape of the comparator tree module that matches the number of groups of sums of the exponential values ​​of the n groups of input data, and to determine, based on the tree shape of the comparator tree module, a maximum value among the sums of the exponential values ​​of the n groups of input data as the exponential maximum value; The exponential difference module is used to determine n exponential differences, each of which is the difference between the maximum exponential value and the sum of the exponential values ​​of a set of input data.

3. The dot product operation device according to claim 2, wherein: The comparator tree module is a recursive structure comparator tree module, and the comparator tree module includes at least one comparator.

4. The dot product operation device according to claim 1, wherein: The mantissa multiplication module comprises: a partial sum generation module, configured to determine a first product based on Booth-coded values ​​of mantissas of a multiplier in a set of input data, the first product being the product of a shifted value of a mantissa value of a multiplicand in the set of input data and the Booth-coded value of the mantissa of the multiplier, and determining the first product as a partial sum of the mantissa values ​​in the set of input data, the Booth-coded value of the mantissa of the multiplier being a value obtained by radix-4 Booth recoding the mantissa value of the multiplier; The first Wallace tree module is used to determine a sum value and a carry value obtained by compressing the partial sum after the sign bit is expanded.

5. The dot product operation device according to claim 4, wherein: The first Wallace tree module is a recursive structure Wallace tree module, and the first Wallace tree module includes at least one 3:2 compressor or at least one 4:2 compressor.

6. The dot product operation device according to claim 4, wherein: The mantissa multiplication module also includes: A sign bit extension module is used to perform sign bit extension on the upper bits of the partial sum of the mantissa values ​​in the input data based on the sign of the partial sum of the mantissa values ​​in the input data, which is the sign of the mantissa Booth coded value of the multiplier in the input data.

7. The dot product operation device according to claim 1, wherein: The bit width of the mantissa value in the input data set by the decoding module is smaller than the bit width of the mantissa value in the output data set by the encoding and rounding module. The bit width of the mantissa value in the input data is the input data precision set by the decoding module, and the bit width of the mantissa value in the output data is the output data precision set by the encoding and rounding module.

8. The dot product operation device according to claim 7, wherein: The bit width of the mantissa value set by the significand alignment module is greater than the bit width of the mantissa value in the output data set by the encoding and rounding module.

9. The dot product operation device according to claim 8, wherein: The significant number alignment module includes: The mantissa discarding module is used to discard the low-order values ​​of the product of the mantissa values ​​in the aligned n groups of input data that exceed the set mantissa value bit width range.

10. The dot product operation device according to claim 1, wherein: The significant number alignment module is a barrel shifter.

Images