Computing units, methods and computer programs for multiplication

Abstract

The invention relates to a computing unit (4) for multiplying a first value x with a first multiplicand w or for multiplying the first value x with a second multiplicand and a third multiplicand, respectively. The computing unit receives the multiplicands in logarithmic number format, such that the multiplicands exist as at least one exponent to a predefinable base number, respectively. The computing unit (4) comprises a first register (43) in which either two exponents of the first multiplicand, or an exponent of the second multiplicand and an exponent of the third multiplicand are stored. A configuration bit (40) is set which characterizes whether the two exponents of the first multiplicand or the exponent of the second multiplicand and the exponent of the third multiplicand are stored in the first register (43). The computing unit comprises at least two shift operators (31, 32). Furthermore, the invention relates to a method and a computer program for multiplying a value x with a multiplicand.

Description

Technical Field

[0001] This invention relates to a computing unit for performing multiplication using a bitshift operator. Similarly, this invention relates to a method and a computer program, respectively configured to perform multiplication using a bitshift operator. Background Technology

[0002] To enable real-time implementation of neural networks, dedicated hardware accelerators have been developed. These hardware accelerators employ hardware-implemented multiply-accumulate units (MACs) for multiplication. M ultiply- A (ccumulate-Einheiten). It has been recognized that the workload in neural networks consists almost entirely of multiplication and addition. Therefore, hardware accelerators achieve high computational throughput by employing MAC units in massive parallelism. In general, networks in graphics processing units (GPUs, G raphical P rocessing U The network is trained on nits, and the representation format of the network weights and intermediate results corresponds to Float32. This digital format is not well-suited for use in embedded systems, as the corresponding embedded systems may have to use Float32-MAC units, which come with high space and energy requirements. Summary of the Invention

[0003] Advantages of the present invention

[0004] The following proposes a computing unit that allows the use of a number format with a reduced word width (e.g., 4 bits), thereby advantageously eliminating the need for multipliers in hardware and allowing the use of bitshifts instead. Bitshifts require a significantly (10-20x) smaller chip area in ASIC implementations, making them more cost-effective. Furthermore, bitshifts result in reduced energy requirements compared to multiplication. In addition, the proposed computing unit offers flexibility given its resolution accuracy and high computational throughput. Neural networks, in particular, can be part of an image classifier. Correspondingly, the computing unit can be particularly suitable for implementing image classifiers.

[0005] Furthermore, the computation unit can manipulate the weights of different layers of the neural network with varying degrees of quantization. This is because the computation unit can operate differently depending on the word width used for the weights.

[0006] The known MAC units of the accelerators mentioned above use linearly quantized weights and therefore cannot benefit from the cost, energy, and chip area savings of simple displacements to the extent that the proposed computational units would.

[0007] Therefore, using the proposed computing unit, an accelerator is proposed that can be flexibly configured according to requirements for the accuracy of digital formats, energy consumption, and computational throughput.

[0008] Disclosure of the present invention

[0009] In a first aspect, a calculation unit is proposed for multiplying a first value x by a first multiplicand w, or for multiplying the first value x by a second multiplicand and a third multiplicand, respectively. The calculation unit receives the value x and the exponents of the multiplicands in logarithmic number format. The multiplicands are represented in logarithmic number format by at least one exponent with respect to a pre-given base. The calculation unit receives at least two exponents of the first multiplicand in a first logarithmic number format (e.g., in "two-hot" number format). Alternatively, the exponents of the second and third multiplicands can be received in a second logarithmic number format (e.g., in "power-of-two" format). In other words, these exponents can be assigned either to the first multiplicand or to the second and third multiplicands. The exponents and the first value x are quantized, for example, using 2, 4, or 8 bits, or more than 8 bits, and can exist as binary numbers.

[0010] The computation unit includes a first register, in which either the two exponents of the first multiplicand are registered. Either the exponents of the second and third multiplicands are stored. There is a configuration bit that indicates the two exponents of the first multiplicand. The exponents of the second and third multiplicands are still registered in the first register. Alternatively, a configuration bit can indicate whether one or more multiplicands are assigned an exponent registered in the first register, or whether an exponent in a first logarithmic number format or a second logarithmic number format is used. The configuration bit can be a bit set in a register of the computation unit or in memory, preferably a bit set in the first register.

[0011] Furthermore, the calculation unit includes a first shift operator and a second shift operator. The shift operators are configured to shift a binary number to the left (preferably in hardware) by a predetermined number of bits, based on a predetermined (binary) value. The first shift operator shifts a first value x by a number of bits corresponding to the value of a first exponent in the exponent stored in the first register. The second shift operator shifts the first value x by a number of bits corresponding to the value of a second exponent in the exponent of the first register. The calculation unit also includes an adder that sums the results of the first and second shift operators. Furthermore, the calculation unit is configured to output the result (B) of the adder when the configuration bit indicates that a first multiplicand is stored in the first register; and to output the result (A) of the second shift operator and the result (C) of the first shift operator when the configuration bit indicates that a second and a third multiplicand are stored in the first register. After the adder has performed its calculation, the result (B) is present at the output of the adder. Result (B) corresponds to the result of multiplying the value x with the first multiplicand. Result (A) corresponds to the result of multiplying the value x with a multiplicand that has been assigned an exponent with a displacement that has been used for the second shifter. The corresponding content applies to result (C).

[0012] On the one hand, the computing unit has the advantage that the proposed hardware is prepared in a first mode (e.g., a "two-hot" digital format): this first mode preferably approaches linear quantization due to the resolution / digital representation, and thus allows for very precise computation, while having lower area and power requirements due to the shifter. On the other hand, the computing unit provides other modes (e.g., a "power-of-two" digital format), where efficiency or computational throughput is important. If the total workload remains constant, the circuit consumes particularly little power in this mode. If the total workload is variable, the computational throughput is twice as high in this mode. Thus, the computing unit not only achieves energy savings but can also be used for higher data throughput. The computing unit also has the advantage that resolution accuracy can be adjusted through two modes, thereby supporting different word widths for different layers of the neural network.

[0013] It is suggested that the computation unit include additional shift operators that shift the result of the first shift operator to the left by a predefined number of bits. This predefined number corresponds to a predefined value ζ, which represents the different resolutions of the two exponents of the first multiplicand. An adder sums the results of the second and other shift operators.

[0014] Furthermore, it is suggested that each exponent in the first register be assigned a sign bit, and that the adder use this sign bit to accumulate or subtract the results from the shift operator. The sign bit indicates whether the corresponding multiplicand, and especially the exponent assigned to that multiplicand, is positive or negative.

[0015] Furthermore, it is suggested that the calculation unit includes a first accumulator and a second accumulator. When the configuration bit indicates that the first multiplicand is registered in the first register, then one of the accumulators only accumulates the result (B) of the adder. When the configuration bit indicates that the exponents of the second and third multiplicands are registered in the first register, then the first accumulator accumulates the result (A) of the second shift operator, and the second accumulator accumulates the result (C) of the first shift operator.

[0016] Accumulation can be understood as performing multiple multiplications and accumulating the results of each multiplication, for example: a←a+(w*x).

[0017] The accumulator can consist of two registers, an accumulator register, and an adder. One of the results, A, B, or C, is stored in the first register. The sign bit of the corresponding exponent is stored in the second register; this exponent is assigned to the multiplicand to which the result from the first register belongs. The adder adds or subtracts the contents of the first register from the accumulator register based on the sign bit from the second register, and stores the result in the accumulator register. Advantageously, this allows for very efficient vector-matrix multiplication.

[0018] Furthermore, it is suggested that the first accumulator accumulates or subtracts the result (A) based on the sign bit allocated to the exponent used in the second shifter; and the second accumulator accumulates or subtracts the result (C) based on the sign bit allocated to the exponent used in the first shifter. Preferably, the sign bit is also stored in the first register.

[0019] Furthermore, it is suggested that the calculation unit be configured such that when the configuration bit changes, the first and second accumulators are reset. Reset can be understood as setting the stored value or the contents of the accumulator (register) to a pre-given initial value, preferably to zero.

[0020] Furthermore, it is suggested that the base of the multiplicand in both the first and second logarithmic number formats corresponds to the value two. The total word width of the two exponents of the first multiplicand can correspond to the total word width of the two exponents of the first and second multiplicands, or vice versa.

[0021] In a second aspect, a method (particularly computer-implemented) for operating a computational unit according to the first aspect is proposed. The method includes the steps of: providing two exponents and a value x and a configuration bit, wherein the configuration bit indicates whether the exponent is assigned to exactly one multiplicand or to each multiplicand separately. Then, the value x is shifted a first time by the number of bits of the first exponent and a second time by the number of bits of the second exponent. Then, the result of the shift operator is output according to the configuration bit; wherein when the configuration bit indicates that both exponents are assigned to one multiplicand, the result of the shift operator is added and output as the result of multiplication; wherein when the configuration bit indicates that both exponents are assigned to one multiplicand separately, the result of the shift operator is output as the result of multiplication of that value with one of the multiplicands respectively.

[0022] In other respects, a computer program is proposed. This computer program is configured to implement the method of the second aspect. The computer program includes instructions that, when run on a computer, cause the computer to perform one of the mentioned methods along with all the steps of that method. Furthermore, a machine-readable storage module is proposed on which the computer program is stored. Attached Figure Description

[0023] Embodiments of the aspects mentioned above are shown in the accompanying drawings and will be described in more detail in the following description. Herein:

[0024] Figure 1 Two schematic graphs are shown, illustrating the distribution of linearly quantized and unquantized values ​​of filters in a neural network layer.

[0025] Figure 2 shows two schematic graphs illustrating the distribution of different pairs of quantized and unquantized values ​​of filters in a neural network layer;

[0026] Figure 3 A schematic diagram of a multiplier is shown;

[0027] Figure 4 A schematic diagram of other multipliers is shown;

[0028] Figure 5 An implementation of a method for multiplying different logarithmic values ​​is shown. Detailed Implementation

[0029] Figure 1A first schematic graph (1) is shown, which illustrates the distribution (10) of the filter values ​​(conv3_3) of a trained neural network layer. The distribution (10) is the frequency distribution of the quantized values ​​represented as Float32 numbers. It should be noted that in the first schematic graph (1), the values ​​follow a monomodal distribution. Furthermore, the first schematic graph (1) shows quantized values ​​(11) with a resolution of 4 bits. These quantized values ​​(11) with a resolution of 4 bits are linearly quantized in this case and are arranged around the value 0.

[0030] However, it has been observed that the values ​​of the filters, especially the weights of the neural network, follow a non-uniform frequency distribution (see the second schematic diagram (2)). Therefore, linear quantization is not suitable for low bit widths in order to quantize the filter values ​​as well as possible in the case of non-uniform distribution. As shown in the second schematic diagram (2), with a small bit width, linear quantization may only reasonably quantize the first mode (12a), while the filter values ​​along the second mode (12b) are completely ignored in the case of linear quantization. This can lead to high quantization errors.

[0031] The following suggests manipulating the distribution (10, 12a, 12b) of the values ​​of neural network layers using a non-uniform quantization format. Preferably, logarithmic representation is used for quantization. Logarithmic representation is understood as using only the value of the exponent with respect to a pre-given base, instead of the value of the number. For example, the value 64 is stored as 2 in logarithmic representation. 6 =64, therefore it is stored as the value 6.

[0032] Preferably, the number representation of the base 2 is used below, and this number representation is called the "Power-of-two" number format.

[0033] If the weights w or filter values ​​of the neural network are quantized as "Power-of-two" values The multiplication of activation x and weights of neurons in the corresponding layer or layer can be achieved by shifting activation x to the left. To proceed in positions:

[0034]

[0035] In the following text, the index is used. This indicates that the value x and the exponent are... They are quantized separately, and preferably exist as binary numbers.

[0036] It has been recognized that the logarithmic representation of the weights of a neural network is only conditionally suitable, especially for the case of a bimodal distribution, where only a small number of sampling points are available for the second mode (see the second schematic plot (2)), see Figure 2(b).

[0037] Therefore, it is recommended to use a numerical format or quantization, where the weights w of the neural network are represented by two "Power-of-two" values:

[0038]

[0039] The number format obtained by using the two “Power-of-two” values ​​according to equation (2) should be referred to below as the “two-hot” number format.

[0040] For multiplication, in "two-hot" number format:

[0041]

[0042] Figure 2 exemplarily illustrates the corresponding quantization values ​​in “Power-of-two” and “two-hot” digital formats for the first and second schematic graphs (1, 2).

[0043] As becomes clear in Figure 2, the “two-hot” format for the bimodal distribution provides more reasonable sample values ​​for the second mode (21).

[0044] To integrate degrees of freedom into the "two-hot" digital representation without incurring any additional cost or significant extra expense to the hardware, a parameter ζ can be inserted. The parameter ζ characterizes the different resolutions of the two values ​​w1 and w2:

[0045]

[0046] Figure 3 A computational unit (3) for multiplying at least two numbers in “two-hot” format is schematically shown.

[0047] In this embodiment, the computing unit (3) includes a multiplier (30) and optionally includes an accumulator (35).

[0048] As input, the multiplier (30) receives either the value x or the activation x, and receives the weights in "two-hot" numerical format, i.e., the exponent of the weights. and In addition, give the index and Two sign bits, s1 and s2, are assigned to the exponent. and Preferably, the sign bit is stored in the first register.

[0049] The input value is preferably an 8-bit resolution. Any other resolution is conceivable, but 16-bit or less than 4-bit resolution is preferred.

[0050] Next, according to equation (4), based on the exponent and And according to the activation x, the multiplier (30) calculates the result of the multiplication through the shift operators (31, 32). For the case where the parameter ζ is given, as in equation (4), the result of the shift operator (31) can be shifted to the left by the following number of bits by the other shift operator (33): the number of bits is described by the parameter ζ.

[0051] According to the first index If the sign bit s1 is obtained, then the results of the two shift operators (31, 32) are accumulated or subtracted, preferably by means of an adder (34), and output as an intermediate result p.

[0052] For cases where the activation x should be multiplied multiple times with different weights w and then all results of each multiplication should be summed, an accumulator (35) can be used. Once the multiplier (30) has determined the intermediate result (p), it can be further processed by the accumulator (35). In this case, based on the intermediate result p and the exponent... The sign bit s2 and the result stored in the accumulator register (acc_reg) determine the final result q.

[0053] Preferably, the accumulator (35) stores the intermediate result p in the register (reg). Then, the sign bit (sign_2) is read from another register (37), and using the adder (36), the intermediate result p in the register (reg) and the contents of the accumulator register (acc_reg) are summed or subtracted according to the sign bit (sign_2). The result of the adder (36) is then stored in the accumulator register (acc_reg) and optionally output as the result q.

[0054] Figure 4 A schematic diagram of another computing unit (4) is shown, which is advantageously capable of performing multiplication in both "two-hot" and "power-of-two" number formats.

[0055] By configuring bit (40), the calculation unit (4) can either be placed in the first mode, i.e., in the weight w according to the “two-hot” number format, or it can be configured to process two weights in the “power-of-two” number format in the second mode.

[0056] Here, in the "two-hot" mode (first mode), the results of the two shifts are added (or subtracted, depending on the first sign bit (sign_l)). The multiplication result is prepared at output B and accumulated in the first accumulator (35a) according to the second sign bit (sign_2). The output of the second accumulator (35b) is 0 and / or not used. In this mode, the resolution of the numerical representation of the weights is higher than that in the second mode.

[0057] The forwarding of the results of the multiplier (30) (at outputs A, B, and C) is performed via switches (41, 42). These switches direct the results of the multiplier (30) (at outputs A, B, and C) to the corresponding accumulators (35a, 35b) according to the configuration bit (40). These switches can be multiplexers.

[0058] Alternatively, for the first mode, the configuration bit (40) can disable the second accumulator (35b) or discard the results at outputs A and C.

[0059] In the "power-of-two" mode (second mode), two distinct weights are multiplied in parallel with the input value x using two shifts. The results of these two multiplications are then prepared at outputs A and C.

[0060] Based on the corresponding sign bits (sign_1, sign_2), each result is added or subtracted into the accumulator registers of the first and second accumulators (35a, 35b). The two results are then ready at the outputs of the two accumulators. In this mode, the number format coarser distinguishes the weights. In return, the computation unit (4) achieves twice the computational throughput.

[0061] Preferably, in the "power-of-two" mode (second mode), by means of not in Figure 4 The other adders shown sum the results of the first and second accumulators (35a, 35b). Preferably, this is done when all results in the accumulators have been calculated and the sum of all multiplication results is to be output.

[0062] Figure 5A flowchart (5) is schematically shown for running a calculation unit (4), which is specifically used to multiply the value x with a multiplicand in “two-hot” number format, or to multiply the value x with two multiplicands in “power-of-two” number format.

[0063] In the first step 500, a value x is provided, along with the exponent of the multiplicand in "two-hot" number format or the exponents of the two multiplicands in "power-of-two" number format. Additionally, configuration bits can be set to indicate whether the two exponents are assigned to a single multiplicand in "two-hot" number format or to two multiplicands in "power-of-two" number format.

[0064] In step 500, the multiplication is then performed either according to the configuration bit or according to the number format of the multiplicand, determining whether to perform the multiplication according to the first mode or the second mode.

[0065] If it has been determined that multiplication should be performed according to the first mode, then step 510a is performed after step 500. If it has been determined in step 500 that multiplication should be performed according to the second mode, then step 510b follows.

[0066] In step 510a, according to equation (3), the result of the multiplication is determined using the exponent of a multiplicand in “two-hot” number format, with the aid of a shift operator and addition.

[0067] After step 510a has ended, step 520a follows. In this case, the result of step 510a is output. Alternatively or additionally, in step 520a, the result of step 510a can be accumulated.

[0068] In step 510b, the exponents of the two multiplicands, in a “power-of-two” number format, are used to perform a shift operation, for example, using two shift operators: and The value x was shifted to the left by the number of exponent bits.

[0069] After step 510b has ended, step 520b follows. In this case, the results of the corresponding move operations in step 510b are output separately from each other. Alternatively, in step 520b, the two results from step 510b can be accumulated separately.

Claims

1. A computing device (4) for multiplying a first value x with at least one multiplicand, wherein said computing device (4) comprises a first register in which either two exponents of a first logarithmic number format of a first multiplicand or one exponent of a second logarithmic number format of a second and a third multiplicand, respectively, can be stored, wherein said computing device is arranged to set a configuration bit (40) representing whether said two exponents of said first multiplicand or said exponent of said second multiplicand and said exponent of said third multiplicand are registered in said first register, wherein the computing device (4) comprises a first and a second shift operator (31, 32), wherein the first shift operator (31) is set up to shift the first value x by a first number of bits depending on the value of a first one of the stored exponents of the first register, wherein the second shift operator (32) is set up to shift the first value x by a second number of bits depending on the value of a second one of the exponents of the first register, wherein the computing device (4) furthermore comprises an adder (34) which is set up to sum up the results of the first and second shift operators, wherein the computing device (4) is set up to output the result (B) of the adder (34) when the configuration bit characterizes that the first multiplicand is registered in the first register and to output the result (A) of the second shift operator and the result (C) of the first shift operator when the configuration bit characterizes that the second and third multiplicands are registered in the first register.

2. The computing device (4) according to claim 1, wherein The computing device (4) comprises a further shift operator, wherein the further shift operator is set up to shift the result of the first shift operator to the left by a third number of bits, wherein the computing device is set up to receive a predefinable value ζ, wherein the third number corresponds to the predefinable value ζ, which characterizes a different resolution of the two exponents of the first multiplicand, wherein the adder (34) is set up to sum up the results of the second shift operator (32) and the further shift operator.

3. The computing device (4) according to claim 1 or claim 2, wherein, Additionally, a sign bit assigned to one of the stored exponents can be registered in the first register, wherein the adder (34) is set up to add or subtract the results of the shift operators depending on the sign bit.

4. The computing device (4) according to claim 1 or 2, wherein, The computing device furthermore comprises a first and a second accumulator (35a, 35b), wherein the computing device is furthermore set up such that, when the configuration bit (40) characterizes that the first multiplicand is registered in the first register, then one of the accumulators (35a, 35b) only adds the result (B) of the adder (34) and wherein the computing device is set up such that, when the configuration bit characterizes that the exponents of the second and third multiplicands are registered in the first register, then the first accumulator (35a) adds the result (A) of the second shift operator and the second accumulator (35b) adds the result (C) of the first shift operator.

5. The computing device (4) according to claim 4, wherein, The first accumulator (35a) is set up to accumulate or subtract the result (A) depending on a sign bit sign2 assigned to the exponent for the second shift operator; and the second accumulator (35b) is set up to accumulate or subtract the result (C) depending on a sign bit signl assigned to the exponent for the first shift operator.

6. The computing device (4) according to claim 4, wherein The computing device (4) is furthermore set up to reset the first and second accumulators (35a, 35b) when the configuration bit (40) changes.

7. A method for operating a computing device (4) according to any one of claims 1 to 6, the method comprising the steps of: providing two exponents and values x and a configuration bit, wherein the configuration bit characterizes whether the exponents are both assigned to exactly one multiplicand or are assigned to one multiplicand each; performing a first shift of the value x by a first number of bits of a first exponent and performing a second shift of the value x by a second number of bits of a second exponent, outputting a result of a shift operator depending on the configuration bit, wherein adding the result of the shift operator when the configuration bit characterizes that the two exponents are assigned to one multiplicand and outputting the result as a result of a multiplication, wherein the result of the shift operator is outputted as a result of a multiplication of the value with one of the multiplicands each when the configuration bit characterizes that the two exponents are assigned to one multiplicand each.

8. The method of claim 7, wherein, The base of the multiplicands in the first and second logarithmic number format corresponds to the value 2.

9. A computer program product comprising instructions which, when implemented on a computer, cause each step of the method according to claim 7 or according to claim 8 to be performed.

10. A machine-readable storage medium having stored thereon the computer program product according to claim 9.

Images