Multipliers and adders in systolic arrays.

Abstract

The subject matter described herein provides systems and techniques for the design and use of multiply-accumulate (MAC) units that perform matrix multiplication with systolic arrays, such as those used in accelerators for deep neural networks (DNNs). These MAC units can take advantage of the particular way in which matrix multiplication is performed in systolic arrays. For example, when matrix A is multiplied with matrix B, the scalar value a of matrix A can be reused many times, and the scalar value b of matrix B can be streamed into the systolic array and forwarded to a series of MAC units in the systolic array, such that only the final value of the dot product calculated for the matrix multiplication, not the intermediate values, can be correct. MAC unit hardware that is specialized to take advantage of these observations is described herein.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Patent Application No. 17 / 377,743, filed July 16, 2021, the disclosure of which is incorporated herein by reference. [Background technology]

[0002] background Recently, accelerators for neural networks such as deep neural networks (DNNs) have been utilizing systolic arrays for high density computation. A systolic array can be a 2D array of multiply-accumulate (MAC) units using a weight-stationary technique used for dense matrix multiplication. A systolic array may alternatively be a 2D array of MAC units using an output-stationary structure or some other structure. A commonly used hardware design used in the multipliers in the MAC units is the Booth (or modified Booth) multiplier. Such a multiplier multiplies two scalar numbers, such as a and b for matrices A and B, respectively, by preparing a partial product from a, Booth encoding b, accumulating the partial products into two terms, such as by using a carry-save adder (CSA) tree or tree reduction, and outputting the result to a final carry-propagate adder (CPA) to derive the result of the multiplication. The number of partial products, which may be approximately n / 2 when using a radix-4 Booth2 multiplier, n / 3 when using a radix-8 Booth3 multiplier, and n / 4 when using a radix-16 Booth4 multiplier, may be important in determining the complexity of the tree reduction. In practice, Booth2 multipliers are used in most cases, and Booth3 multipliers in other cases. Higher radix multipliers may be used rarely due to "hard multiples" that are difficult to compute. In a Booth2 multiplier, the partial products may include 0, ±a, and ±2a. However, in a Booth3 multiplier, the partial products may additionally include ±3a and ±4a. Here, 3a is not a power of 2, and therefore, unlike the other partial products, it may be known as a hard multiple that must be computed before other steps in the multiplication of a and b are performed. Higher radix multiplier designs may have more hard multiples that must be computed. The need to compute hard multiples makes existing high radix Booth multiplier designs impractical and inefficient.In addition, conventional MAC unit adders may add the multiplier products to partial sums and pass the result to the next MAC unit in the systolic array in an inefficient and suboptimal manner. For example, the design of these conventional adders may have inefficiencies that do not allow for their implementation in a systolic array for matrix multiplication. Summary of the Invention

[0003] overview The efficiency of matrix multiplication based on systolic arrays can be important in the design of accelerators such as DNN accelerators. This paper describes the design of more efficient and practical multipliers and adders used in MAC units for systolic arrays. By examining a conventional MAC unit used for matrix multiplication of matrix A with matrix B in a systolic array, the following three observations can be made:

[0004] First, once a scalar value a in matrix A is loaded and latched by flip / flops in a MAC unit in a systolic array, it may be reused many times, for example as many times as the width of matrix B, which may typically be wide. Scalar value a may be reused several times before it is reloaded.

[0005] Second, a scalar value b in matrix B may be streamed to a systolic array. In particular, the same scalar value b may be forwarded to a set of MAC units in a row of the systolic array.

[0006] Third, the MAC units in a column of the systolic array may jointly compute a dot product using a row of matrix A and a column of matrix B. Only the final value of the dot product computation may be used to derive the result. In particular, the intermediate values, or partial sums, at each MAC unit in the systolic array may not need to be correct as long as the final dot product computation is correct.

[0007] Based on the above observations regarding the operation of conventional MAC units in systolic arrays, extended multipliers and adders for MAC units in systolic arrays can be designed. In particular, the multipliers and adders traditionally found in each MAC unit can be fused, and possibly using additional components, to generate an extended MAC unit that can take advantage of the above observations. The extended MAC unit can take advantage of higher radix multiplication, thereby simplifying CSA tree contraction. The extended MAC unit can be faster, more efficient, and optimized to perform matrix multiplication, requiring less hardware, and more energy efficient, when compared to conventional MAC units. The extended MAC unit can include these and other advantages when used for matrix multiplication of matrix A with matrix B in systolic arrays such as those used in accelerators for DNNs.

[0008] Some structural examples of such an enhanced MAC unit are provided herein. In general, one aspect of the subject matter described herein includes a multiply-accumulate (MAC) unit that multiplies two numbers to generate a result. The MAC unit can include a first flip / flop, a second flip / flop, a multiplexer, at least one carry-save adder, and a plurality of parallel segmented adders. The first flip / flop can be configured to latch the first number and output the first number and a multiplier value based on the first number. The second flip / flop can be configured to load the second number and output the second number. The multiplexer can be in communication with the first flip / flop and the second flip / flop and can be configured to receive the first number and the multiplier value from the first flip / flop and receive the second number from the second flip / flop. The multiplexer may be configured to output a plurality of partial products based on the first number, the multiplied value, and the second number. The at least one carry-save adder may be in communication with the multiplexer. The at least one carry-save adder may be configured to receive the plurality of partial products and the partial sums and output at least two partially summed numbers based on the plurality of partial products and the partial sums. The plurality of parallel split adders may be in communication with the at least one carry-save adder. The plurality of parallel split adders may be configured to receive the at least two partially summed numbers, perform an addition operation on the at least two partially summed numbers, and output a result. The second number may be encoded using Booth encoding. The MAC unit may include at least one hard multiple calculator. The at least one hard multiple calculator may be in communication with the first flip / flop and may be configured to receive a preloaded number and output a multiplied value to the first flip / flop. The at least one carry-save adder may include only multiple-input, two-output carry-save adders. The at least one carry-save adder may include carry-save adders and multiple-input, two-output carry-save adders.The MAC unit may include a third flip / flop. The third flip / flop may be in communication with the multiple-input, two-output, carry-save adder and may be configured to load the partial sum and output the partial sum to the multiple-input, two-output, carry-save adder. The third flip / flop may be configured to load the partial sum from a partial sum output from another MAC unit. The multiple parallel split adders are configured to operate in parallel on segments of the number that are in partially redundant form. The first flip / flop may be configured to latch the first number at twice the normal clock speed. The MAC unit may be an enhanced MAC unit that uses a fused version of a multiplier and an adder. The MAC unit may be in a systolic array.

[0009] Another aspect of the subject matter includes a multiply-accumulate (MAC) unit. The MAC unit can include a first flip / flop, a second flip / flop, a third flip / flop, a multiplier, and an adder. The first flip / flop can be configured to latch a first numerical value and output the first numerical value. The second flip / flop can be configured to latch a multiplier value based on the first numerical value and output the multiplier value. The third flip / flop can be configured to load a second numerical value and output the second numerical value. The multiplier can be in communication with the first, second, and third flip / flops. The multiplier can be configured to receive the first numerical value from the first flip / flop, the multiplier value from the second flip / flop, and the second numerical value from the third flip / flop. The multiplier can be configured to generate a partial product based on the first numerical value, the multiplier value, and the second numerical value. The multiplier can be configured to output the partial product. The adder can be in communication with the multiplier. The adder can be configured to receive the partial product from the multiplier. The adder can be configured to add the partial product with the partial sum value to generate a result. The MAC unit can include a double data rate flip / flop configured to latch the first number and the multiple value at twice the normal clock speed. The MAC unit can include a demultiplexer in communication with the double data rate flip / flop and the first and second flip / flops. The demultiplexer can be configured to receive the first number and the multiple value from the double data rate flip / flop, output the first number to the first flip / flop and output the multiple value to the second flip / flop. The MAC unit can include a fourth flip / flop. The fourth flip / flop can be in communication with the adder and can be configured to load the partial sum value and output the partial sum value to the adder. The MAC unit can be in a systolic array.

[0010] Yet another aspect of the subject matter includes a method of computing a result of at least one multiply-accumulate operation using a first flip-flop. , No. The number 1 and Based on the first number The multiple value may be latched. A second flip / flop may be used to load a second number. A multiplexer may be used to generate a plurality of partial products based on the first number, the multiple value, and the second number. The plurality of partial products may be received from the multiplexer. At least one carry-save adder may be used to receive the partial sums. At least one carry-save adder may be used to generate at least two partially summed numbers based on the plurality of partial products and the partial sums. A plurality of parallel split adders may be used to receive the at least two partially summed numbers. An addition operation may be performed on the at least two partially summed numbers to calculate a result. The second number may be encoded using Booth encoding. The multiple value may be calculated using at least one hard multiple calculator. The process may include loading the partial sums and outputting the partial sums to the at least one carry-save adder. [Brief description of the drawings]

[0011] [Figure 1A] FIG. 2 illustrates an example systolic array used to multiply matrix A and matrix B to generate output matrix C. [Figure 1B] FIG. 2 illustrates an example MAC unit that may be used in a systolic array used to multiply matrix A and matrix B. [Diagram 2] FIG. 2 illustrates a partially redundant format for numbers that can be several bits long. [Diagram 3] FIG. 1 illustrates an example of an operation performed using a parallel split adder in each MAC unit in a systolic array. [Figure 4A] FIG. 2 illustrates another example systolic array used to multiply matrix A and matrix B to generate output matrix C. [Figure 4B]FIG. 4B illustrates a MAC unit that can be used in a systolic array used to multiply matrix A and matrix B, such as the systolic array described in connection with FIG. 4A. [Diagram 5] FIG. 2 illustrates yet another example systolic array used to multiply matrix A and matrix B to generate output matrix C. [Figure 6A] FIG. 6 illustrates a MAC unit that can be used in a systolic array used to multiply matrix A and matrix B, such as the systolic array described in connection with FIG. 4A and / or FIG. 5. [Figure 6B] FIG. 6 illustrates a MAC unit that can be used in a systolic array used to multiply matrix A and matrix B, such as the systolic array described in connection with FIG. 4A and / or FIG. 5. [Figure 7] FIG. 1 is a flow diagram of an example process for computing the result of a multiply-accumulate operation. [Figure 8] FIG. 2 is a block diagram of an example electronic device according to an aspect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Detailed Description FIG. 1A illustrates an example systolic array 100 used to multiply matrix A 110 and matrix B 120 to generate output matrix C 130. In particular, systolic array 100 may be a 2D array of multiply-accumulate (MAC) units with a weight-preserving technique used for dense matrix multiplication of matrix A 110 and matrix B 120 to generate matrix C 130. In some examples, systolic array 100 may be 4×4 in size. Matrix C 130 may be derived by computing the dot product of each row of matrix A 110 and each column of matrix B 120. Thus, matrix C 130 may be the output of systolic array 100. Each entry of systolic array 100 may represent a MAC unit that performs the dot product calculation to generate output matrix C 130. The scalar values ​​in matrix B 120 may move horizontally through systolic array 100 to be used by several MAC units. Intermediate values ​​between MAC units in systolic array 100 may not be known.

[0013] Figure 1B shows an example MAC unit 150 that may be used in a systolic array used to multiply matrix A and matrix B, such as the systolic array described in connection with Figure 1A. MAC unit 150 may include flip / flops 152, 154, 156, and 160, a multiplier 158, and an adder 162. Figure 1B also shows additional flip / flops 170, 172, and 174, which may be in other MAC units of the systolic array.

[0014] Flip / flops 152 and 154 can preload and latch scalar values ​​of matrix A. In particular, flip / flop 152 can be used to preload a value of scalar value a of matrix A and pass this value to flip / flop 154. Flip / flop 154 ​​can be used to load and latch scalar value a and reuse this value several times in a calculation until it is reloaded. Flip / flop 156 can be used to load scalar value b of matrix B. The scalar value b loaded into flip / flop 156 can be used sequentially in each of the MAC units in a row of the systolic array. Multiplier 158 can multiply two scalar numerical values ​​a and b loaded and / or latched in flip / flop 154 ​​and flip / flop 156, respectively. In particular, multiplier 158 may receive as inputs scalar values ​​a and b from flip / flops 154 and 156, respectively, and may multiply these scalar values. Multiplier 158 may output the result of the multiplication to adder 162. Flip / flop 160 may load and / or latch a partial sum, which may be, for example, output by a preceding MAC unit in the systolic array. Adder 162 may receive as inputs the output of multiplier 158 and the partial sum loaded and / or latched in flip / flop 160, and may output the sum of these inputs as a partial sum output, which may be stored by a flip / flop, such as flip / flop 174, of a downstream MAC unit in the systolic array. Additional flip / flops 170, 172, and 174 may be in other MAC units in the systolic array. The intermediate result may be the partial sum output by adder 162 in the MAC unit. This partial sum may not be associated with a MAC unit at the bottom of the systolic array and may not be used as the final result. Instead, the final result may be output by an adder in the MAC unit at the bottom of the systolic array.

[0015] Based on the above observations regarding the operation of conventional MAC units in a systolic array, extended multipliers and adders for the MAC units in the systolic array can be designed. In particular, design optimizations can be performed on the MAC units to take advantage of how data is input and used and / or reused in the systolic array. In some examples, the extensions to the multiplier and adder designs may be applicable for general purposes. In some examples, the extensions to the multiplier and adder designs may not be applicable for general purposes.

[0016] One of the obstacles to using high-radix multipliers, such as Booth3 or Booth4 multipliers, in the MAC units in the systolic arrays may be hard multiples. This is because, as mentioned above, such hard multiples may have to be calculated before other steps in the multiplication of scalar value a and scalar value b are performed. Based on the first observation above, hard multiples may be calculated when preloading each scalar value a of matrix A. The calculated hard multiples may be used several times until a new scalar value a is preloaded by the MAC unit. The calculated hard multiples may be out of the critical path of the multiplication and, in some instances, may be implemented in a multi-cycle operation. Thus, high-radix Booth multipliers may be used in the MAC units in the systolic arrays. The high-radix Booth multipliers do not have to perform hard multiple calculations every clock cycle and do not have to perform hard multiple calculations for use in the critical path of the multiplication. The high-radix Booth multipliers may be faster than conventional multipliers because they generate fewer partial products.

[0017] Based on the second observation above, the scalar value b of matrix B can be Booth encoded first. Once the scalar value is Booth encoded, it can be streamed to the MAC units of the systolic array. By Booth encoding the scalar value b before being streamed to the systolic array, the Booth encoding function can be offloaded from the critical path of multiplications performed by the multipliers in each MAC unit.

[0018] Each MAC unit in the systolic array can use an adder that adds the product from the multiplier in each MAC unit to the partial sum from the MAC unit / systolic array cell above. The MAC unit can then pass the result to the MAC unit / systolic array cell below. Based on the third observation above, the adders in each MAC unit can be simplified. In particular, a partially redundant form can be used for each adder.

[0019] FIG. 2 shows a partially redundant format of a number 200 that may be several bits long. In the partially redundant format, the number 200 may be represented using several smaller segments. For example, as shown in FIG. 2, if the number 200 is a 24-bit integer value, it may be represented as six 4-bit segments, such as segments 202, 204, 206, 208, 210, and 212, and each segment except the last may include a respective 1-bit carry to the next segment, such as carries 214, 216, 218, 220, and 222. The last segment does not include or use a carry. Thus, the 24-bit integer value may be represented in a total of 6×4+5 bits=29 bits. Such a redundant format may be used for partial sums within each MAC unit. This may be for partial sums that are accumulated vertically, and in each MAC unit, products from multipliers may be added, such as 16-bit numbers if 8-bit×8-bit multiplication is being performed.

[0020] Each adder in a conventional MAC unit, such as adder 162 described in connection with Figure 1B, can be replaced with a parallel split adder. Based on the partially redundant format of a number, such as the number 200, an appropriate number of split adders can operate in parallel within each MAC unit.

[0021] For example, for number 200 in Figure 2, six 4-bit adders can be operated in parallel, one for each segment associated with number 200. The carry from the MAC unit / systolic array cell above (if there is one) can be used in the addition operation performed by each MAC unit in the systolic array. The carry output by each MAC unit can be passed on to the MAC unit / systolic array cell below (if there is one). At the bottom of the systolic array, the partially redundant form can be converted to a non-redundant form.

[0022] Figure 3 shows an example operation 300 performed using parallel split adders in each MAC unit in a systolic array. In Figure 3, parallel split adders are used to add the products from the multipliers in each MAC unit to the partial sums from the MAC unit / systolic array cell above. In the example operation 300, a 16-bit number 310, which is the product from the multipliers in each MAC unit and contains two 8-bit segments, is added to a partially redundant 24-bit partial sum 320 that contains three 8-bit segments. The number 310 and the partial sum 320 are added using three 8-bit parallel split adders.

[0023] A 16-bit number 310 can be represented using two segments p1 and p0. The number p1:p0 can be a 16-bit product. At each systolic array MAC unit, this number can be the product output from its multiplier. Additionally, a 24-bit partial sum 320 in partially redundant form can be represented using segments s2, s1, and s0, and carries c2 and c1. The numbers s2:s1:s0 and c2:c1 can be represented in partially redundant form using three 8-bit segments along with two carry bits. The partial sums can be received from the MAC unit / systolic array cells above each MAC unit in the systolic array.

[0024] Two numbers, such as a 16-bit product value 310 and a partial sum 320, can be added using multiple parallel split adders. For example, a 16-bit product value 310 and a partial sum 320 can be added using three 8-bit parallel split adders 340, 350, and 360. Although not shown, a carry from the MAC unit / systolic array cell above each MAC unit in the systolic array can be received and used as an input to the first of the parallel split adders 340. A carry that may be output by the last parallel split adder does not have to propagate to the next segment, but rather may be passed to the MAC unit / systolic array cell below each MAC unit. The output of the parallel split adders 340, 350, and 360, along with carries 376 and 378, can be segments 370, 372, and 374, respectively. Segments 370, 372, and 374 together with carries 376, 378 may form a final result 380 for operation 300 performed using a parallel split adder.

[0025] Operation 300 shows a 16-bit number 310, which is the product from the multiplier, being added to a 24-bit partial sum 320, although numbers or partial sums of any length can be added in a similar manner. Additionally, while Figure 3 shows 8-bit segments and a particular number of carries, any substantially equal sized segments and any number of carries can be used in a similar manner to Figure 3. Additionally, while Figure 3 shows three parallel split adders, more or less parallel split adders can be used in a similar manner to Figure 3.

[0026] A carry save adder (CSA) may be used by the multiplier in each MAC unit. A CSA may be a digital adder that may be used in a multiplier to calculate the sum of three or more input numbers as part of a multiplication operation. The CSA may output two numbers that may be summed to generate a final result of the original numbers to be summed. The CSA may be associated with a tree that may have several levels of addition that the CSA must perform to output two numbers that are used to generate the final result of the original sum. A carry propagate adder (CPA) may be used in the multiplier and adder of each MAC unit to properly calculate the carry bits when the segments are added together.

[0027] FIG. 4A illustrates an example systolic array 400 used to multiply matrix A 410 and matrix B 420 to generate output matrix C 430. In particular, systolic array 400 may be a 2D array of multiply-accumulate (MAC) units with weight-preserving techniques used for dense matrix multiplication of matrix A 410 and matrix B 420 to generate matrix C 430. In some examples, systolic array 400 may be 4×4 in size. FIG. 4A also illustrates Booth encoder 440 and bottom CPA 445. Booth encoder 440 may be used to Booth encode scalar values ​​b of matrix B 420. Once these scalar values ​​are Booth encoded, they may be streamed to the MAC units in systolic array 400. By Booth encoding scalar values ​​b before being streamed to systolic array 400, the Booth encoding function may be offloaded from the critical path of the multipliers in each MAC unit. The output b values ​​of Booth encoder 440 and the scalar values ​​of matrix A 410 can be used as inputs to fused multipliers and adders in each MAC unit in systolic array 400. The fused multipliers and adders can be used, at least in part, in computing the dot products of each row of matrix A 410 with each column of matrix B 420. CPA 445 can operate similarly to multiple parallel split adders implemented as CPAs as described above. The outputs of the fused multipliers and adders in the bottom MAC units of systolic array 400 can be output to bottom CPA 445 and summed to generate values ​​of matrix C 430.

[0028] FIG. 4B illustrates a MAC unit 450 that can be used in a systolic array used to multiply matrix A and matrix B, such as the systolic array described in connection with FIG. 4A. MAC unit 450 can be used in a systolic array, such as systolic array 400 described in connection with FIG. 4A. MAC unit 450 can include a fused version of a conventional multiplier and adder found in a conventional MAC unit, such as MAC unit 150 of FIG. 1B. MAC unit 450 can include flip / flops 452, 454, 458, and 462, a hard multiplier calculator 456, a multiplexer 460, a CSA tree 470, a 3-input 2-output CSA 472, and a parallel split adder 480.

[0029] Flip / flops 452 and 454 can be used to preload and latch scalar values ​​of matrix A. In particular, flip / flop 452 can be used to preload a value of scalar value a of matrix A and output this value to flip / flop 454. Flip / flop 454 can be used to load and latch scalar value a from flip / flop 452 and reuse this value several times in the calculation until it is reloaded. Thus, loading and latching scalar value a may take only one clock cycle or only a few clock cycles. Flip / flop 454 can output the received value to hard multiple calculator 456. Hard multiple calculator 456 can receive scalar value a preloaded and latched by flip / flops 452 and 454. Hard multiple calculator 456 can multiply the received numeric value by one or more integer multiples and output the result of the multiplication to flip / flop 458 to precalculate the hard multiple. In particular, the hard multiple calculator 456 can output hard multiples of the scalar value a. Each hard multiple of a can be any multiple that is not a power of two. For example, the hard multiple calculator 456 can output multiples ±3a, ±5a, and / or ±7a to the flip / flop 458. The pre-computation of the hard multiples can occur in one or a few clock cycles and may not need to occur every clock cycle. Thus, the computation of the hard multiples can be off the critical path of the multiplication. The flip / flop 458 can receive the scalar value a from the flip / flop 454 and / or from the hard multiple calculator 456. Additionally, the flip / flop 458 can receive the value output by the hard multiple calculator 456. The flip / flop 458 can load and / or latch the received values ​​and output these values ​​to the multiplexer 460.

[0030] Flip / flop 462 may be used to load the Booth-encoded scalar value b of matrix B. For example, the scalar value b of matrix B may be Booth3 encoded and loaded, such as by Booth3 encoding, such as with a Booth encoding using a radix of 8, and loading the result into flip / flop 462. The Booth-encoded scalar value b loaded into flip / flop 462 may be used sequentially by each of the MAC units in a row of the systolic array. Flip / flop 462 may output the loaded Booth-encoded value of b to multiplexer 460.

[0031] Multiplexer 460 may receive as input the value of a, the Booth encoded value of b, and the received hard multiple of a, and may output multiple partial products to CSA tree 470. For example, multiplexer 460 may receive the value of a from flip / flop 458, the hard multiple value of a from hard multiple calculator 456, and the Booth encoded value of b from flip / flop 462, and may output three partial products to CSA tree 470. Multiplexer 460 may be used, for example, to implement a Booth triple multiplication of the inputs.

[0032] CSA tree 470 may be a modified version of a conventional CSA that may include only the first level of a multi-input, two-output CSA. This may mean that CSA tree 470 accepts multiple inputs and outputs two numbers. For example, CSA tree 470 may include the first level of a three-input, two-output CSA, which may accept three input numbers and output two numbers. In some examples, there may be multiple levels of CSAs that may be referred to as CSA trees, such as three-input, two-output and / or four-input, two-output CSAs, depending on the number of bits that may be multiplied by the multipliers. The numbers output by CSA tree 470 may each be in a partially redundant form. These numbers may be added to a partially redundant form of the partial sums from the MAC units / systolic array cells above. Flip / flops 474, which may or may not be part of MAC unit 450, may load and / or latch the partial sums from the MAC units / systolic array cells above MAC unit 450. Flip / flop 474 can output this value to a multiple-input, two-output CSA, such as 3-input, two-output CSA 472 .

[0033] In the MAC unit 450, the partially redundant form of the numbers output by the CSA tree, which may represent the product of a conventional multiplier, may be added to the partially redundant form of the partial sum from the MAC unit / systolic array cell above. This addition may be performed using a multi-input, two-output CSA, such as the three-input, two-output CSA 472. Furthermore, in the MAC unit 450, which includes a fused version of a conventional MAC unit multiplier and adder, the CPA used in the conventional multiplier may not be used, i.e., may be skipped. This addition operation performed by the three-input, two-output CSA 472 may be performed in a manner similar to the operation 300 described in connection with FIG. 3. For example, two 16-bit numbers in partially redundant form output by the CSA tree 470 may be added to a 24-bit number and five carry bits in partially redundant form of the partial sum from the flip / flop 474. This result may be reduced using the three-input, two-output CSA 472, which can accept three numbers as inputs and output two numbers. In particular, two numbers from CSA tree 470, such as two 16-bit numbers that may represent the output of a multiplier, can be input to 3-input 2-output CSA 472 along with a partially summed number, such as a 24-bit number. The output of 3-input 2-output CSA 472 can be two numbers. For example, if two 16-bit numbers and a 24-bit number are added, 3-input 2-output CSA 472 can generate two 24-bit numbers as output. In some examples, CSA tree 470 and 3-input 2-output CSA 472 may be combined into a single 4-input 2-output CSA, which may replace CSA tree 470 and 3-input 2-output CSA 472. The resulting 4-input 2-output CSA can accept as inputs the same three inputs that CSA tree 470 normally receives, plus the partially summed numbers that normally enter 3-input 2-output CSA 472. The resulting 4-input, 2-output CSA can output the two numbers normally output by the 3-input, 2-output CSA 472 .

[0034] The numbers output by the 3-input 2-output CSA 472 may be summed, for example, by several parallel split adders 480. For a MAC unit at the bottom of a systolic array, the parallel split adders may be implemented as the bottom CPA. For example, if Booth3 encoding and multiplication are performed, six 4-bit parallel split adders may be used to perform this summation. This summation may produce a number output by the parallel split adders 480 that represents the output of the MAC unit 450. For example, the number output by the MAC unit 450 may include 24 bits along with five carry bits in a partially redundant format.

[0035] In the MAC unit 450, multipliers and adders traditionally found in MAC units can be fused to create an extended MAC unit design. The extended MAC unit design can be more efficient and optimized to perform matrix multiplication, can require less hardware, and can be more energy efficient when compared to traditional MAC unit designs. The extended MAC unit design can include these and other advantages when used for matrix multiplication in systolic arrays such as those used in accelerators for DNNs.

[0036] The above examples relating to the MAC unit 450 can be described as using Booth3 encoding and multiplication. However, if the numbers to be multiplied and / or accumulated are of higher precision, such as a multiplication of a 24-bit number with another 24-bit number, a higher radix Booth design for the MAC unit can be used. In some examples, when such a high radix Booth design is used, the hard multiplication calculation can be accomplished by preloading and latching the next scalar value a of the matrix A, although this may involve more complexity. Alternatively or additionally, the hard multiplication calculation can be performed as a multi-cycle operation. A higher radix Booth design for the MAC unit can include increasing the height of the CSA tree 470, possibly adjusting the parallel split adder 480, and also adjusting the partially redundant format of the numbers input and output by the components of the MAC unit. For example, the parallel split adder 480 can include eight 6-bit adders used to perform 48-bit additions.

[0037] The complexity of the multiplier in the MAC unit can be denoted as O(n2) for an n-bit by n-bit multiplication. The complexity of the multiplication operation and the MAC up to the multiplier can be affected by the numerical values ​​of the multiplicand and the multiplier involved in the multiplication. In particular, the precision of the multiplicand, such as the scalar value a of matrix A, can affect the hard multiplication calculation, the width of the tree contraction of the carry-save adder, and the final carry propagate adder. The precision of what is multiplied by the multiplicand, i.e., the multiplier, such as the scalar value b of matrix B, can affect the height of the tree contraction of the carry-save adder. As mentioned above, in some examples, the hard multiplication can be pre-computed. In such examples, the width of the contraction tree can be mostly in the tree domain, rather than in the critical path. Additionally, in such examples, the final carry propagate adder can be mixed with a parallel split adder. Thus, in such examples, the precision of the scalar value a of matrix A may not affect the computation latency as significantly as the precision of the scalar value b of matrix B. Thus, in some examples, asymmetric precision may be used for the values / numbers input to the systolic array, and higher precision may be used for matrix A rather than matrix B. In these examples, the matrix containing the higher precision numbers may be defined as matrix A. For example, matrix A may use 16-bit or 32-bit integers, while matrix B may contain 8-bit integers.

[0038] While the above examples have assumed the use of integer arithmetic, floating point arithmetic can be performed using a systolic array with respective MAC units, such as systolic array 400 and MAC unit 450 described in relation to Figures 4A and 4B, respectively. Techniques such as those proposed above can be applied to floating point arithmetic for mantissa multiplication in a manner similar to that described above for integer arithmetic. Techniques such as those proposed above can also be applied to block floating point arithmetic and related numeric formats in a manner similar to that described above for integer arithmetic. Partially redundant formats of numbers, such as partially redundant formats of numbers 200 described in relation to Figure 2, and parallel split adders, such as parallel split adder 480 described in relation to Figure 4B, can also be used to perform floating point arithmetic by the MAC units. However, additional changes to the design of the MAC units may be required to perform such floating point arithmetic.

[0039] FIG. 5 illustrates an example systolic array 500 used to multiply matrix A 510 and matrix B 520 to generate output matrix C 530. In particular, the systolic array 500 may be a 2D array of multiply-accumulate (MAC) units with weight-preserving techniques used for dense matrix multiplication of matrix A 510 and matrix B 520 to generate matrix C 530. In some examples, the systolic array 500 may be 4×4 in size. FIG. 5 also illustrates a hard multiple calculator 540. The hard multiple calculator 540 may receive a scalar value a of matrix A 510. The hard multiple calculator 540 may multiply the received value by one or more integer multiples to pre-compute one or more hard multiples and output the results of the multiplication to the MAC units in the systolic array 500. In particular, the hard multiple calculator 540 may output hard multiples, each of which may be any multiple of the value a that is not a power of two. For example, the hard multiple calculator 540 may output multiples ±3a, ±5a, and / or ±7a. The pre-computation of the hard multiples may occur in one or a few clock cycles and may not need to be recalculated over several cycles. Thus, the pre-computation of the hard multiples may not need to occur every clock cycle, which may increase computational efficiency. Thus, the computation of the hard multiples may be off the critical path of the multiplication. The scalar values ​​of matrix A 510, the hard multiples output by the hard multiple calculator 540, and the scalar value b of matrix B 520 may be used as inputs to fused multipliers and adders in each MAC unit in the systolic array 500. The fused multipliers and adders may be used at least in part in computing the dot products of each row of matrix A 510 and each column of matrix B 520 to generate the rows and columns of matrix C 530.

[0040] The area overhead of high radix Booth multipliers used in the MAC units can be reduced by sharing hard multiple calculators. One design technique can be to place a hard multiple calculator at the top of a systolic array, such as hard multiple calculator 540 shown as being at the top of systolic array 500. This hard multiple calculator can be used when matrix A 510 is loaded. Thus, each MAC unit in the systolic array may not include a hard multiple calculator. With this technique, additional wiring may be used to pass the calculated hard multiples to the MAC units in the systolic array. By placing the hard multiple calculator at the top of the systolic array, the functionality of each of the MAC units in the systolic array may not change. For example, any software that can be used to perform calculations in each MAC unit does not need to know whether or not hard multiples are calculated before matrix A is pushed into the systolic array. As will be described in more detail, FIG. 6A below demonstrates a technique in which a hard multiple calculator can be used at the top of a systolic array, along with additional wiring that can be used to pass the calculated hard multiples.

[0041] Alternatively, the reloading of the scalar value a of matrix A 510 may be at a rate faster than the normal clock rate. For example, in a Booth3 multiplier, a and 3a may be pushed first, and then the flip / flops may be reloaded with these values ​​at twice the normal clock rate. For example, the scalar value of a may be reloaded on the rising edge of the clock, and 3a may be loaded on the falling edge of the clock. In this example, the hard multiple calculator logic that can be used to calculate hard multiples such as 3a may also be designed for this faster clock rate. As will be described in more detail, FIG. 6B below demonstrates a technique that may reload the scalar value of a at a clock rate faster than the normal clock rate.

[0042] Another design technique may be to share the hard multiple calculator with some of the adjacent MAC units in the systolic array. For example, two vertically adjacent MAC units or 2×2 MAC units in the systolic array may share one hard multiple calculator. Using this technique, the hard multiples may be distributed using additional wiring, which may be local wiring. Using this technique, the scalar value of a and the hard multiples of a may alternatively be reloaded at a higher clock rate.

[0043] Figure 6A shows a MAC unit 600 that may be used in a systolic array used to multiply matrix A and matrix B. For example, MAC unit 600 may be in a systolic array described in connection with Figures 4A and / or 5. MAC unit 600 may include flip / flops 610, 612, 614, 616, 618, and 622, a multiplier 620, and an adder 630. Figure 6A also shows additional flip / flops 640, 642, 644, and 648, which may be in other MAC units in the systolic array.

[0044] Flip / flops 610 and 612 can preload and latch scalar values ​​of matrix A. In particular, flip / flop 610 can be used to preload scalar value a of matrix A and pass this value to flip / flop 612. Flip / flop 612 can be used to load and latch scalar value a and reuse this value several times in the calculation until it is reloaded. Flip / flops 614 and 616 can preload and latch hard multiples of scalar values ​​of matrix A. In particular, flip / flop 614 can be used to preload precomputed hard multiples of scalar value a of matrix A and pass these values ​​to flip / flop 616. Flip / flop 616 can be used to load and latch precomputed hard multiples of scalar value a and reuse these values ​​several times in the calculation until it is reloaded. Flip / flop 618 can be used to load scalar value b of matrix B. The scalar value b loaded into flip / flop 618 is available sequentially to each of the MAC units in a row of the systolic array.

[0045] Multiplier 620 can multiply the values ​​loaded and / or latched into flip / flop 612 and / or flip / flop 616, as well as into flip / flop 618. In particular, multiplier 620 can receive as inputs scalar value a, and / or hard multiples of scalar value a, and scalar value b from flip / flops 612, 616, and / or 618, and can multiply one or more of these values ​​to generate partial products. Multiplier 620 can output the result of the multiplication to adder 630 included in MAC unit 600. Flip / flop 622 can load and / or latch partial sums that may have been output by a preceding MAC unit in the systolic array. Adder 630 can receive as inputs the output of multiplier 620 and the partial sums loaded and / or latched into flip / flop 622, and can output the sum of these inputs as a partial sum output. This output may be stored by a flip / flop, such as flip / flop 644, of a downstream MAC unit in the systolic array. Additional flip / flops 640, 642, 644, and 648 may be in other MAC units in the systolic array. The intermediate result, which may be a partial sum output by adder 630 in a MAC unit that is not in the bottom row of the systolic array, may not be used as the final result. Instead, the final result may be output by an adder in a MAC unit in the bottom row of the systolic array.

[0046] Figure 6B shows a MAC unit 650 that may be used in a systolic array used to multiply matrix A and matrix B. For example, MAC unit 650 may be in the systolic array described in connection with Figures 4A and / or 5. MAC unit 650 may include a demultiplexer 662, flip / flops 660, 664, 666, 668, and 672, a multiplier 670, and an adder 680. Figure 6A also shows additional flip / flops 690, 692, and 694, which may be in other MAC units in the systolic array.

[0047] Flip / flop 660 can preload and / or latch scalar values ​​of matrix A and hard multiples of these scalar values. In particular, flip / flop 660 can be used to preload scalar value a of matrix A as well as preload pre-computed hard multiples of value a and pass these values ​​to demultiplexer 662. Flip / flop 660 can operate at double data rate and can therefore be considered to be a double data rate flip / flop. Demultiplexer 662 can be used to load and latch scalar value a and pre-computed hard multiples of a and output these values ​​to flip / flops 664 and 666, respectively. Flip / flop 664 can be used to load and latch scalar value a and reuse this value several times in the calculation until this value is reloaded. Flip / flop 666 can be used to load and latch pre-computed hard multiples of scalar value a and reuse these values ​​several times in a calculation until they are reloaded. Flip / flop 668 can be used to load scalar value b of matrix B. The scalar value b loaded into flip / flop 668 can be used sequentially in each of the MAC units in a row of the systolic array.

[0048] Multiplier 670 may multiply the values ​​loaded and / or latched into flip / flop 664 and / or flip / flop 666, as well as into flip / flop 668. In particular, multiplier 670 may receive as inputs scalar value a, and / or hard multiples of scalar value a, and scalar value b from flip / flops 664, 666, and / or 668, and may multiply one or more of these values ​​to generate partial products. Multiplier 670 may output the result of the multiplication to adder 680 included in MAC unit 650. Flip / flop 672 may load and / or latch partial sums that may have been output by a preceding MAC unit in the systolic array, for example. Adder 680 may receive as inputs the output of multiplier 670 and the partial sums loaded and / or latched into flip / flop 672, and may output the sum of these inputs as a partial sum output. This output may be stored by a flip / flop, such as flip / flop 692, of a downstream MAC unit in the systolic array. Additional flip / flops 690, 692, and 694 may be in other MAC units in the systolic array. The intermediate results, which may be partial sums output by adders 680 in MAC units that are not in the bottom row of the systolic array, may not be used as the final result. Instead, the final result may be output by an adder in a MAC unit in the bottom row of the systolic array.

[0049] By streaming Booth encoded scalar value b instead of the uncoded scalar value of b in matrix B, the number of wires for each MAC unit in the systolic array may be increased. For example, an 8-bit multiplication using Booth3 encoding may use three sets of Booth encodings, each using five wires. In this example, wires may be used for a, 2a, 3a, 4a, and sign, where a, 2a, 3a, 4a are one-hot encoded, and all four of these can be zero, except for 0. In this example, 15 wires may be used, compared to eight wires for uncoded 8-bit data. One way to reduce the number of wires may be to use a different encoding. For example, a 4-bit two's complement signed representation may be used for Booth3 encoding. In this example, numbers between -8 and +7 may be used to cover all possible cases between -4 and +4. In this example, each MAC unit, such as MAC unit 450 described in connection with FIG. 4B, may use a decoder to drive each multiplexer, such as multiplexer 460 described in connection with FIG. 4B. Another approach to inputting the scalar value b of matrix B may be to use a higher data rate, similar to that described above with reference to FIG. 6B. Using this approach, the scalar value a of matrix A and the associated hard multiple values ​​are preloaded at double the data rate. For example, using this approach, eight wires can be used to transfer and / or load the Booth-encoded scalar value b of matrix B at twice the normal clock rate, i.e., twice the data rate.

[0050] In some examples, output-holding systolic arrays can be used. Output-holding systolic arrays do not need to hold / store the multiplier operands in each MAC unit. In addition, pre-computed hard multiples can be passed vertically downwards in output-holding systolic arrays, similar to those described above. In these examples, Booth-encoded scalar values ​​b of matrix B can be streamed. Additionally, these examples can use partially redundant forms of numbers along with parallel split adders.

[0051] 7 is a flow diagram of an example process 700 for computing a result of at least one multiply-accumulate operation. Process 700 may be performed by various elements of a MAC unit, such as the MAC units described in connection with FIGS. 4B, 6A, and 6B.

[0052] In block 710, a first flip / flop, such as flip / flop 458 described in connection with FIG. 4B, may be used to latch the first number and a multiplier value based on the first number. For example, a scalar value a of matrix A and a hard multiplier value may be latched. The latched numbers may be reused several times in calculations within the systolic array until they are reloaded and / or re-latched. These latched numbers may be output to a multiplexer.

[0053] At block 720, a second number may be loaded using a second flip / flop, such as flip / flop 462 described in connection with Figure 4B. For example, a Booth encoded scalar value b of matrix B may be loaded. The loaded number may be streamed into the systolic array of the MAC unit.

[0054] At block 730, a multiplexer, such as multiplexer 460 described in connection with FIGURE 4B, may be used to generate multiple partial products based on the first number, the multiplexer value, and the second number. For example, the multiplexer may take as input the value of a, the Booth encoded value of b, and the received hard multiple of a, and output multiple partial products based on these numbers.

[0055] At block 740, the partial products and partial sums may be received by at least one carry-save adder, such as CSA tree 470 and / or 3-input, 2-output CSA 472 described in connection with Figure 4B. The partial products may be received from a multiplexer, such as multiplexer 460 described in connection with Figure 4B. The partial sums may be loaded by and received from flip / flops, such as flip / flop 474 described in connection with Figure 4B.

[0056] In block 750, at least one carry-save adder, such as the three-input, two-output CSA 472 described in connection with FIG. 4B, may be used to generate at least two partially summed numbers based on the multiple partial products and partial sums. For example, the two partially summed 16-bit numbers in partially redundant form output by the CSA tree 470 described in connection with FIG. 4B may be added to the partially redundant 24-bit number of the partial sum from the flip / flop 474 described in connection with FIG. 4B and five carry bits. This result may be reduced using the three-input, two-output CSA 472, which may accept three numbers as inputs and output two numbers. In particular, two numbers from the CSA tree 470, such as two 16-bit numbers, may be input to the three-input, two-output CSA 472 along with the partial sum number, such as a 24-bit number. The output of the at least one carry-save adder may be the two partially summed numbers. For example, if two 16-bit numbers and a 24-bit number are being added, the 3-input 2-output CSA 472 can generate as output two 24-bit numbers that can be output to a parallel split adder.

[0057] At block 760, the at least two partially summed values ​​may be received using multiple parallel split adders, such as parallel split adder 480 described in connection with FIG. 4B. Continuing with the previous example, the 3-input 2-output CSA 472 may generate as output two 24-bit values, which may be the at least two partially summed values ​​received by the parallel split adder 480.

[0058] In block 770, an addition operation may be performed on at least two partially summed numbers to calculate a result. The numbers output by the carry-save adders may be added by a parallel split adder, such as several parallel split adders 480. For example, if Booth3 encoding and multiplication are performed, six 4-bit parallel split adders may be used to perform the addition. The addition may produce a number output by the parallel split adders. This number may represent the output of a MAC unit capable of performing process 700. Continuing with the previous example, the number output by a MAC unit, such as MAC unit 450 described in connection with FIG. 4B, may include 24 bits along with five carry bits in a partially redundant format.

[0059] Although the operations of process 700 are described in a particular order, it should be understood that the order may be changed, operations may be performed in parallel, and further, it should be understood that operations may be added or omitted.

[0060] 8 illustrates a block diagram of an example electronic device 800. The electronic device 800 may include one or more processors 810, such as one or more xPUs, a system memory 820, a bus 830, a networking interface 840, and other components (not shown), such as storage, output device interfaces, input device interfaces, etc. The bus 830 may be used to communicate between the processor 810, the system memory 820, the networking interface 840, and the other components. Any or all of the components of the electronic device 800 may be used with the subject matter of this disclosure.

[0061] Depending on the desired configuration, the processor 810 may be of any type, including but not limited to a tensor processing unit (TPU), a microprocessor, a microcontroller, a digital signal processor (DSP), or any combination thereof. The processor 810 may include a systolic array, such as the systolic array described in connection with FIG. 4A and / or FIG. 5. The processor 810 may include a level 1 cache 811 and a level 2 cache 812, etc. 1 of 1 t or multiple The processor 810 may include a processor core 813, one or more MAC units 850, and registers 814. The processor core 813 may include one or more arithmetic logic units (ALUs), one or more floating point units (FPUs), one or more DSP cores, or any combination thereof. In some examples, the one or more MAC units 850 may be implemented within the processor core 813. A memory controller 815 may also be used with the processor 810, or in some implementations, the memory controller 815 may be an internal part of the processor 810.

[0062] Depending on the desired configuration, the physical memory 820 can be of any type, including, but not limited to, volatile memory such as RAM, non-volatile memory such as ROM, flash memory, etc., or any combination thereof. The physical memory 820 can include an operating system 821, one or more applications 822, and program data 824, which can include service data 825. The non-transitory computer readable medium program data 824 can include storing instructions that, when executed by one or more processing devices, implement a process that calculates a result of a multiply-accumulate operation 823. In some examples, the one or more applications 822 can be arranged to operate on the operating system 821 with the program data 824 and the service data 825.

[0063] The electronic device 800 may have additional features or functionality and additional interfaces that facilitate communication between the basic configuration 801 and any necessary devices and interfaces.

[0064] The physical memory 820 may be an example of a computer storage medium, including but not limited to RAM, ROM, EEPROM, flash memory or other memory technology, or any other medium that can be used to store desired information and that can be accessed by the electronic device 800. Any such computer storage medium may be part of the device 800.

[0065] The network interface 840 can couple the electronic device 800 to a network (not shown) and / or to another electronic device (not shown). Thus, the electronic device 800 can be part of a network of electronic devices, such as one of a local area network ("LAN"), a wide area network ("WAN"), an intranet, or a network such as the Internet. In some examples, the electronic device 800 can include a network connection interface that forms a network connection to a network and a local communication connection interface that forms a tethering connection with another device. The connection can be wired or wireless. The electronic device 800 can bridge the network connection and the tethering connection to connect other devices to the network via the network interface 840.

[0066] One or more MAC units 850 may be used to perform multiply-and-accumulate operations, such as those that need to be performed for matrix multiplication. One or more MAC units 850 may be part of a systolic array. For example, the MAC unit 850 and the systolic array in which it operates may be used in an accelerator that may be used for DNN implementations. One or more MAC units 850 may be any one of the MAC units described above. For example, the MAC unit 850 may be similar to or include the MAC unit 450 described in connection with FIG. 4B, the MAC unit 600 described in connection with FIG. 6A, and / or the MAC unit 650 described in connection with FIG. 6B.

[0067] The MAC unit 850 may include a fused multiplier as described herein. and It can be considered an extended MAC unit, such as one that includes an adder or other extension function. Such an extended MAC unit can be more efficient, practical, and optimized for performing matrix multiplication, can require less hardware, and can be more energy efficient, when compared to a conventional MAC unit. The extended MAC unit can include these and other advantages when used for matrix multiplication in systolic arrays, such as those used in accelerators for DNNs.

[0068] The electronic device 800 can be implemented as part of a small form factor portable (or mobile) electronic device, such as a speaker, a headphone, an earphone, a mobile phone, a smartphone, a smart watch, a personal digital assistant (PDA), a personal media player device, a tablet computer (tablet), a wireless web watch device, a personal headset device, a wearable device, an application-specific device, or a hybrid device including any of the above functionality. The electronic device 800 can also be implemented as a personal computer, including both laptop and non-laptop computer configurations. The electronic device 800 can also be implemented as a server, accelerator, or larger system.

[0069] Aspects of the present disclosure can be implemented as a computer-implemented process, a system, or as an article of manufacture, such as a memory device or a non-transitory computer-readable storage medium. The computer-readable storage medium can be readable by an electronic device and can include instructions for causing the electronic device or other device to execute the processes and techniques described in this disclosure. The computer-readable storage medium can be implemented by a volatile computer memory, a non-volatile computer memory, a solid-state memory, a flash drive, and / or other memory, or other non-transitory and / or transitory medium. Aspects of the present disclosure can be implemented in different forms of software, firmware, and / or hardware. Furthermore, the teachings of the present disclosure can be implemented by, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other components.

[0070] Aspects of the present disclosure may be executed on a single device or on multiple devices. For example, program modules including one or more components described herein may be located on different devices, each executing one or more aspects of the present disclosure. As used in this disclosure, the terms "a" or "an" can include one or more items unless otherwise specified. Additionally, the phrase "based on" is intended to mean "based at least in part on" unless otherwise specified.

[0071] The above-described aspects of the disclosure are intended to be illustrative. They have been selected to illustrate the principles and applications of the disclosure, and are not intended to be exhaustive or to limit the disclosure. Many modifications and variations of the disclosed aspects may be apparent to those skilled in the art.

[0072] Unless otherwise noted, the above alternatives are not mutually exclusive and can be implemented in various combinations to achieve unique advantages. These and other variations and combinations of the above-described features can be utilized without departing from the subject matter defined by the claims, and therefore the description of the above-described examples should be construed as illustrative, but not limiting, of the subject matter defined by the claims. In addition, the provision of examples described herein, as well as phrases such as "such as," "including," and the like, should not be construed as limiting the subject matter of the claims to any particular example, but rather, the example is intended to illustrate only one of many possible examples. Additionally, the same reference numbers in different drawings may identify the same or similar elements.

[0073] Numerous examples are described herein and are presented for illustrative purposes only. The described examples are not, and are not intended to be, limiting in any sense. Those skilled in the art will appreciate that the disclosed subject matter can be implemented with various modifications and alterations, such as structural, logical, software, and electrical changes. It should be understood that the described features are not limited to use in one or more specific examples or drawings referenced in describing them, unless expressly specified otherwise.

Claims

1. a multiply-accumulate (MAC) unit for multiplying two numbers to produce a result, a first flip / flop configured to latch a first value and output the first value and a multiplication value based on the first value; a second flip / flop configured to load a second value and output said second value; a multiplexer in communication with the first flip / flop and the second flip / flop, configured to receive the first number and the multiplier value from the first flip / flop and the second number from the second flip / flop, and to output a plurality of partial products based on the first number, the multiplier value, and the second number; at least one carry-save adder in communication with the multiplexer configured to receive the plurality of partial products and the partial sums and to output at least two partially summed numbers based on the plurality of partial products and the partial sums; a plurality of parallel split adders in communication with the at least one carry-save adder, the split adders configured to receive the at least two partially summed numbers, perform an addition operation on the at least two partially summed numbers, and output a result; A MAC unit comprising:

2. The MAC unit of claim 1 , wherein the second number is encoded using Booth encoding.

3. 3. The MAC unit of claim 1, further comprising at least one hard multiplier calculator in communication with the first flip / flop, the hard multiplier calculator configured to receive a preloaded value and output the multiplier value to the first flip / flop.

4. 3. The MAC unit of claim 1, wherein the at least one carry-save adder includes only multiple-input, two-output carry-save adders.

5. 3. The MAC unit of claim 1, wherein the at least one carry-save adder includes a carry-save adder and a multiple-input two-output carry-save adder.

6. 6. The MAC unit of claim 5, further comprising a third flip / flop in communication with the multiple-input, two-output carry-save adder configured to load the partial sum and output the partial sum to the multiple-input, two-output carry-save adder.

7. 7. The MAC unit of claim 6, wherein the third flip / flop is configured to load the partial sum from a partial sum output from another MAC unit.

8. 3. A MAC unit as claimed in claim 1 or 2, wherein the plurality of parallel split adders are configured to operate in parallel on segments of a value in partially redundant form.

9. 3. A MAC unit as claimed in claim 1 or 2, wherein the first flip / flop is configured to latch the first numerical value at a rate faster than a clock rate.

10. The MAC unit of claim 1 or 2, wherein the MAC unit is an enhanced MAC unit that uses a fused version of a multiplier and an adder.

11. The MAC unit of claim 1 or 2, wherein the MAC unit is in a systolic array.

12. 1. A method for computing a result of at least one multiply-accumulate operation, comprising the steps of: latching a first value and a multiplication value based on the first value using a first flip / flop; loading a second value using a second flip / flop; and generating a plurality of partial products based on the first number, the multiple value, and the second number using a multiplexer; receiving the plurality of partial products and partial sums using at least one carry save adder; generating at least two partially summed numbers based on the plurality of partial products and the partial sums using the at least one carry save adder; receiving the at least two partially summed values ​​using a plurality of parallel split adders; and performing an addition operation on the at least two partially summed numbers to calculate a result.

13. The method of claim 12 , wherein the second number is encoded using Booth encoding.

14. The method of claim 12 or 13, wherein the multiple value is calculated using at least one hard multiple calculator.

15. loading the partial sums; 14. The method of claim 12 or 13, further comprising: outputting the partial sum to the at least one carry-save adder.

Images