Arithmetic unit

Abstract

To reduce the number of times of access to an RF per product-sum operation.SOLUTION: For an L * N matrix A, an N * M matrix B, an L * M matrix C, and Cin, where L and M are both integers greater than or equal to 2 and N is an integer greater than or equal to 1, accumulating an outer product Ok of a k-th column of A and a k-th row of B in an array of L * M accumulators for an integer k greater than or equal to 0 and less than N; An array of l * m product-sum operators that perform an L * M * N matrix product operation C = A * B or an L * M * N matrix product-sum operation C = A * B + Cin, where either l or m is l = L or m = M, and the other is an integer of 2 ≤ l <L or 2 ≤ m <M, and the l * m product-sum operators perform accumulation of an outer product Ok in a plurality of steps.SELECTED DRAWING: Figure 17

Description

[Technical Field]

[0001] The present invention relates to a computing unit. [Background technology]

[0002] In particular, in applications such as AI (Artificial Intelligence), it is becoming increasingly important to increase the speed and power consumption of matrix multiply-accumulate (MMA) operations using low-precision elements such as FP16 (half-precision floating-point numbers).

[0003] MMA calculators are classified into inner product and outer product types.

[0004] The MMA processor stores the target matrix in a layer of the memory hierarchy directly connected to the processor, such as a register file (RF), and performs the operation while accessing that layer. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication No. 2022-506418 [Non-patent literature]

[0006] [Non-Patent Document 1] Y. Wang, et al.: Dual-side Sparse Tensor Core, Int'l Symp. on Computer Architecture (ISCA), pp. 1083-1095 (2021). Summary of the Invention [Problem to be solved by the invention]

[0007] Especially in applications such as AI, when dealing with data of low precision such as FP16, the power consumed by the multiply-accumulate (FMA) operation of each matrix element is low, so the power due to RF access relatively increases.

[0008] On one side, it aims to reduce the number of RF accesses per multiply-accumulate operation.

Means for Solving the Problem

[0009] On one side, when L and M are both integers of 2 or more and N is an integer of 1 or more, for the L×N matrix A, N×M matrix B, L×M matrix C, and Cin, for an integer k from 0 to less than N, by accumulating the outer product Ok of the k-th column of A and the k-th row of B into an array of L×M accumulators, an array of l×m multiply-accumulators that perform the L×M×N matrix product operation C = A*B, or the L×M×N matrix product-sum operation C = A*B + Cin is provided, where either l or m is l = L or m = M, and the other is an integer where 2 ≤ l < L or 2 ≤ m < M, and the l×m multiply-accumulators perform the accumulation of the outer product Ok in multiple steps.

Effect of the Invention

[0010] On one side, the number of RF accesses per multiply-accumulate operation can be reduced.

Brief Explanation of Drawings

[0011] [Figure 1] It is a block diagram schematically showing a configuration example of an arithmetic processing unit in a related example. [Figure 2] It is a diagram explaining the three-dimensional representation of a matrix product in a related example. [Figure 3] It is a diagram explaining the three-dimensional representation of an L×M×N matrix product in a related example. [Figure 4] It is a table explaining the correspondence relationship between the number of FMAs and the number of steps in each dimension in a related example. [Figure 5]FIG. 10 is a diagram illustrating a zero-dimensional MMA calculator in a related example. [Figure 6] FIG. 10 is a diagram illustrating a one-dimensional MMA calculator in a related example. [Figure 7] FIG. 10 is a diagram illustrating a two-dimensional MMA calculator in a related example. [Figure 8] FIG. 10 is a block diagram schematically illustrating an example of the configuration of a two-dimensional MMA calculator in a related example. [Figure 9] FIG. 10 is a diagram illustrating a three-dimensional MMA calculator in a related example. [Figure 10] FIG. 10 is a diagram illustrating the number of RF accesses of a one-dimensional MMA calculator in a related example. [Figure 11] 10 is a table illustrating the correspondence between the number of FMAs for each dimension and the number of steps in the related examples and the embodiment. [Figure 12] FIG. 2 is a diagram illustrating a 1.5-dimensional MMA calculator according to an embodiment. [Figure 13] FIG. 10 is a diagram illustrating a 1.5-dimensional MMA calculator in a modified example. [Figure 14] 1(a) is a diagram illustrating a one-dimensional intermediate product type MMA calculator, and FIG. 1(b) is a diagram illustrating a one-dimensional outer product type MMA calculator. [Figure 15] 10 is a table showing the number of RF accesses when peak performance is constant in each method. [Figure 16] FIG. 2 is a block diagram schematically illustrating an example of the configuration of a 2×8 1.5-dimensional MMA calculator according to an embodiment. [Figure 17] 17 is a block diagram schematically illustrating an example of the configuration of a processing element (PE) illustrated in FIG. 16. FIG. [Figure 18] FIG. 14 is a block diagram schematically illustrating a configuration example of a PE corresponding to FIG. 13 and having a thickness in the k-axis direction. DETAILED DESCRIPTION OF THE INVENTION

[0012] [A] Related Examples FIG. 1 is a block diagram schematically illustrating an example of the configuration of a processing unit 60 in a related example.

[0013] 1, the arithmetic processing device 60 includes an MMA arithmetic unit 6, a register file (RF) 7, a cache 8, and a main memory device 9. The RF 7, the cache 8, and the main memory device 9 form a memory hierarchy.

[0014] The main memory device 9 stores various data including matrices that are the subject of MMA calculations.

[0015] The cache 8 stores a copy of part of the data stored in the main memory 9. The cache may be a local memory. The cache or local memory may have two or more levels.

[0016] The RF 7 stores a copy of a portion of the data stored in the cache 8. The RF may be a buffer memory.

[0017] The MMA calculator 6 is directly connected to the RF 7 and performs matrix multiplication and matrix multiply-accumulate (MMA) operations on matrices stored in the RF 7. In the example shown in Fig. 1, the RF 7 stores matrices A, B, Cin, and C, and the MMA calculator 6 performs the MMA operation C = A * B + Cin.

[0018] Figure 2 is a diagram for explaining a three-dimensional representation of matrix multiplication in a related example. For simplicity, Figure 2 assumes the case of matrix multiplication operation C = A * B, but it can be easily extended to the case of MMA operation C = A * B + Cin.

[0019] In Figure 2, matrix A is defined on the ik plane, matrix B is defined on the kj plane, and matrix C is defined on the ij plane. Matrix A is a i,k and matrix B is represented by b k,j and matrix C is expressed as c i,j It should be noted that matrix A is expressed by rotating it by 90 degrees. The same applies to Fig. 3, Fig. 5 to Fig. 7, Fig. 9, Fig. 10, and Fig. 12 to Fig. 14.

[0020] In Figure 2, since it is a 4x4x4 three-dimensional representation, i=0 to 3, j=0 to 3, and k=0 to 3.

[0021] FMA calculation is performed at each intersection of the central cube. i,k and b k,j Multiply by and perform accumulation (Accumulate) above and below (k direction). The matrix multiplication is completed when the 4x4x4 intersections of the cube are covered, typically from above. As shown in Figure 2, in the matrix multiplication, there is no dependency in the left-right direction, but there is dependency in the up-down direction, and intermediate accumulations are sent up and down.

[0022] FIG. 3 is a diagram illustrating a three-dimensional representation of an L×M×N matrix product in a related example.

[0023] Reference symbol A1 in Fig. 3 represents an LxMxN (L and M are both integers of 2 or greater, and N is an integer of 1 or greater) matrix product C = A * B. Matrix A is an LxN matrix, matrix B is an NxM matrix, and matrix C is an LxM matrix.

[0024] Reference symbol A2 in Figure 3 shows a three-dimensional representation of an LxMxN matrix multiplication. The three-dimensional representation is basically the same as that in Figure 2, but takes into account the calculation order of the cross product type. Each plane shown by reference symbol A21 represents the cross product Ok of column k of matrix A and row k of matrix B, i.e., the multiplication of all combinations of each element, and the intersections represent the products of corresponding elements. As shown by reference symbol A22, matrix C is obtained by summing up the products of the same i and j on each plane in the vertical direction.

[0025] FIG. 4 is a table illustrating the correspondence between the number of FMA operators constituting an MMA operator and the number of steps in the related example.

[0026] To cover 4x4x4 FMA operations with 1 to 4x4x4 FMA operators, in 0 dimensions the number of steps is 4x4x4 for one FMA operator, and in 1 dimension the number of steps is 4x4 for four FMA operators, as shown in Figure 4. In 2 dimensions the number of steps is 4x4 for 4x4 FMA operators, and in 3 dimensions the number of steps is 1 for 4x4x4 FMA operators.

[0027] FIG. 5 is a diagram for explaining the 0-dimensional MMA calculator 6 in a related example.

[0028] As shown by symbol B1 in FIG. 5, in the 0th dimension (point), one FMA calculator performs normal sequential processing of 4×4×4 steps.

[0029] The calculation order is generally assumed to be the inner product type ijk (an example shown by the solid arrow of symbol B2), the middle product type ikj, and the outer product type kij. For example, the calculation order of the inner product type ijk means the order from the outside when expressed in a for loop, as shown below, and conversely, it means processing from the inner variables in the order of priority: k, j, i. for i for j for k FMA operation; The inner product type ijk is advantageous in terms of the number of RF accesses.

[0030] FIG. 6 is a diagram for explaining the one-dimensional MMA calculator 6 in the related example.

[0031] As shown by symbol C1 in Figure 6, in one dimension (line), 4 × 4 step processing is performed by four FMA operators. Symbol C2 indicates the broadcast direction.

[0032] There are various possible calculation orders within the MMA calculator, but in general, the intermediate product type ikj (shown by the solid arrow with symbol C3) is advantageous.

[0033] Although a cross-product type kij can be considered, it is not practical because it requires a large number of accumulators to hold intermediate cumulative sums of the matrix C.

[0034] FIG. 7 is a diagram for explaining the two-dimensional MMA calculator 6 in a related example.

[0035] As shown by symbol D1 in Fig. 7, in two dimensions (plane), four steps of processing are performed by 4 x 4 FMA computing units. Symbol D2 indicates the broadcast direction.

[0036] The calculation order inside the MMA calculator is advantageous for the cross product type kij (an example shown by the solid arrow with symbol D3).

[0037] 8 is a block diagram schematically showing an example of the configuration of a two-dimensional outer product type MMA calculator 6 in a related example. In FIG. 8, an example of a 4×4 outer product type is shown.

[0038] In C=A*B, 4x4 products of all combinations of elements in the kth column of matrix A and the kth row of matrix B are calculated in one step. Then, these products are accumulated for four steps from k=0 to 3 to obtain the matrix product.

[0039] In the example shown in FIG. 8, the processing elements (PEs) 61 are arranged in a 4×4 tile pattern to calculate the product of square matrices with sides of 4.

[0040] The four elements of each of the k columns of A shown as symbol E1 and the k rows of B shown as symbol E2 are broadcast in the row or column direction of the tile shown as symbol E3, and 4 x 4 multiplication and accumulation operations for all combinations of input elements are performed in one step.

[0041] To perform the FMA operation, the PE 61 includes a multiplier 611, a multiplexer (mux) 612, an adder 613, and an accumulator (acc) 614. The multiplier 611 and the adder 613 may be configured as an FMA operation unit.

[0042] The multiplier 611 multiplies a and b. The mux 612 selects and outputs the output of either cin or acc 614 within the PE 61. The adder 613 adds the output of the multiplier 611 and the output of mux 612. The acc 614 holds the output of the adder 613 and outputs it as the calculation result c.

[0043] Values ​​are input to the tile indicated by symbol E3, shifting the column A and row B at each step (in other words, cycle). The input column A and row B are broadcast to each PE 61 in the row or column direction, and the sum of products of all combinations is calculated.

[0044] After four steps, input of all matrix data is completed, and the calculation result C is held in acc 614 of each PE 61.

[0045] FIG. 9 is a diagram for explaining the three-dimensional MMA calculator 6 in a related example.

[0046] As indicated by symbol F1 in Fig. 9, in three dimensions (cube), one step of processing is performed by 4 x 4 x 4 FMA computing units. Note that symbol F2 indicates the broadcast direction.

[0047] FIG. 10 is a diagram for explaining the number of RF accesses of the one-dimensional MMA calculator 6 in the related example.

[0048] When the matrix sum of products C=A*B+Cin of N×N square matrices A, B, Cin, and C is calculated, the number of RF accesses per FMA operation is calculated.

[0049] First, find the minimum number of accesses. The total number of FMA operations is N 3 On the other hand, the minimum number of RF accesses per matrix is ​​N 2 There are four matrices, A, B, Cin, and C, so the total is 4N 2 Therefore, the minimum number of RF accesses per FMA operation is 4N 2 / N 3= 4 / N. In other words, the larger N (the size of the matrix to be calculated) is, the fewer RF accesses per FMA operation will be. However, in practice, depending on the MMA processor system, it may not be possible to achieve 4 / N.

[0050] In the example shown in FIG. 10, four one-dimensional (linear) FMA operators perform 4×4 step intermediate product type ikj processing.

[0051] The elements of matrix B have low reusability. In other words, when the calculation is performed up to the bottom a03 of row a00 of matrix A and the calculation moves to the next row a10 of matrix A, b00 to b03 of matrix B must be reread. Each element of matrix B is accessed N times, and the number of times matrix B is accessed is N 2 ×N=N 3 Therefore, the number of RF accesses is the minimum of 4N 2 From 3N 2 +N 3 (A,Cin,C:N 2 ,B:N 3 ) increases.

[0052] The reusability of matrix elements that have been accessed once differs depending on the method used (0-dimensional to 3-dimensional).

[0053] The number of RF accesses is 2N in 0 dimensions. 2 +2N 3 (Cin,C:N 2 ,A,B:N 3 ) and in one dimension it becomes 3N 2 +N 3 (A,Cin,C:N 2 ,B:N 3 ) For B (or A), the same element needs to be accessed N times.

[0054] In 2D and 3D, the number of RF accesses is 4N 2 (A,B,Cin,C:N 2 ) Since every element is accessed only once, the number of RF accesses is minimized.

[0055] [B] Embodiment An embodiment will be described below with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude various modifications or application of techniques not explicitly stated in the embodiment. In other words, this embodiment can be implemented with various modifications within the scope of its purpose. Furthermore, each figure does not intend to include only the components shown in the figure, but may also include other functions, etc.

[0056] In the following drawings, the same reference numerals denote similar parts, and therefore the description thereof will be omitted.

[0057] FIG. 11 is a table illustrating the correspondence between the number of FMA operators and the number of steps for each dimension in the related example and the embodiment.

[0058] The table shown in FIG. 11 shows the correspondence between the number of FMA operators and the number of steps in 1.5 dimensions in the embodiment, in addition to the correspondence between the number of FMA operators and the number of steps in 0-dimension, 1-dimension, 2-dimension, and 3-dimension in the related example shown in FIG. 4.

[0059] In 1.5 dimensions, the number of FMA operators is 2x4, while the number of steps is 2x4.

[0060] FIG. 12 is a diagram illustrating the 1.5-dimensional MMA calculator 1 according to the embodiment.

[0061] As shown by symbol H1 in Figure 12, in the 1.5-dimensional MMA calculator 1, a 2x4 region on the ij plane corresponds to one step. The dotted arrow shown by symbol H2 indicates the broadcast direction. The calculation order follows the direction of the solid arrow shown by symbol H3.

[0062] FIG. 13 is a diagram illustrating the 1.5-dimensional MMA calculator 1 in the modified example.

[0063] One step may have a thickness in the k-axis direction. As shown by the reference sign I1 in FIG. 13, in the 1.5D MMA calculator 1 in the modification, when A is an L×N matrix, B is an N×M matrix, and C is an L×M matrix, an l×m×n (m = M, 2 ≤ l < L, 1 ≤ n ≤ N) region constitutes one step. Note that the dotted arrow indicated by the reference sign I2 shows the direction of broadcasting. Also, the calculation order is in the direction of the solid arrow indicated by the reference sign I3. l and m in the i,j directions may be swapped.

[0064] FIG. 14(a) is a diagram for explaining a 1D intermediate product type MMA calculator, and FIG. 14(b) is a diagram for explaining a 1D outer product type MMA calculator.

[0065] For the 1D type of calculation order (which order to process i, j, k), a general intermediate product type (ikj order (processing in the order of j→k→i from the back)) is assumed, but it is also conceivable to use the outer product type (kij order) for the 1D type. By using the outer product type, the number of RF accesses / FMA count can be made even smaller, but since the number of Accs in one PE becomes extremely large at N, it is difficult to implement.

[0066] In the 1D intermediate product type (ikj order) shown in FIG. 14(a), as shown by the reference sign J11, one dimension (line) constitutes one step. Note that the solid arrow indicated by the reference sign J12 shows the direction of broadcasting. As shown by the solid arrow of the reference sign J13, the calculation order is in the direction parallel to the k-axis.

[0067] On the other hand, in the 1D outer product type (kij order) shown in FIG. 14(b), as shown by the reference sign J12, one dimension (line) constitutes one step as in the intermediate product type. Note that the solid arrow indicated by the reference sign J22 shows the direction of broadcasting. As shown by the solid arrow of the reference sign J23, the calculation order is in the direction parallel to the i-axis.

[0068] FIG. 15 is a table showing the number of RF accesses in the case where the peak performance is constant for each method.

[0069] 15, the number of RF accesses / number of FMA operations for each method is calculated when the MMA operator is configured with 64 FMA operators (with the same peak operation performance). The size of the target matrix (one side: N) varies depending on the method. With the 1.5-dimensional MMA (rectangular outer product type) operator in the embodiment, the number of RF accesses per FMA operation can be reduced to half or less compared to Related Examples 1 to 3.

[0070] FIG. 16 is a block diagram schematically illustrating an example of the configuration of a 2×8 1.5-dimensional MMA calculator 1 according to an embodiment.

[0071] The 1.5-dimensional MMA calculator 1 is an example of a matrix multiply-accumulate calculator, and may be provided in the calculation processing device 60, similar to the MMA calculator 6 in the related example shown in FIG.

[0072] A two-dimensional MMA (cross product type) calculator divides the elements of a column of matrix A into groups of multiple elements each, and performs multiplication of only one group in one step. A two-dimensional MMA (cross product type) calculator calculates one cross product step over multiple steps.

[0073] In the example shown in FIG. 16, for simplicity, a case is described in which matrix A is input in multiple steps, but matrices A and B may be interchanged.

[0074] When the number of FMA operators is the same as that of the two-dimensional MMA operators, this embodiment makes it possible to increase the target matrix size N. As a result, the ratio of the required data access amount per calculation amount can be reduced.

[0075] 16, the product of two 8×8 square matrices is calculated. In FIG. 16, PEs 11 are arranged in a 2×8 tile pattern. One group of elements in the columns of matrix A (see groups 5 to 8 of symbol K1) and all elements in the rows of matrix B (see groups 5 to 8 of symbol K2) are input to the 1.5-dimensional MMA calculator 1 from one side of the tile and broadcast, and the product-sum operation for all combinations of input elements is calculated in one step.

[0076] Matrix A is input to the 1.5-dimensional MMA calculator 1 while switching between two elements every four steps, and matrix B may be input to the 1.5-dimensional MMA calculator 1 once every four steps, with all elements of one row being input. As a result, the number of elements input to the 1.5-dimensional MMA calculator 1 per step is the same for A and B.

[0077] The operation of 2x8 matrix multiplication is basically the same as that of a 2-dimensional MMA calculator (cross product type). The 8 elements of the columns of matrix A are divided into groups of 2 elements x 4, and one group is processed in one step (in other words, one step of the cross product type is processed over four steps).

[0078] At each step, the input data of matrix A is input while being switched, and four acc114 (described later with reference to FIG. 17) hold the intermediate cumulative sums while being switched for each step.

[0079] For matrix B, once a row is entered, the same row is maintained for four steps.

[0080] For simplicity, FIG. 16 illustrates a case where matrix A is input in multiple steps, but matrix A and matrix B may be interchanged.

[0081] FIG. 17 is a block diagram schematically illustrating an example of the configuration of the PE 11 shown in FIG.

[0082] The PE 11 is an example of a processing element, and includes a multiplier 111, an adder 112, four muxes 113, four accs (accumulators) 114, and a mux 115. The multiplier 111 and the adder 112 may be FMA calculators.

[0083] PE11 is basically the same as the related example in that it has a multiplier and an adder and executes FMA operations, but it executes one step of the cross-product operation by interleaving it in steps equal to the number of divisions. For this reason, PE11 is equipped with acc114, which holds the accumulated results of each area, for the number of divisions.

[0084] The multiplier 111 multiplies the element A of one input matrix and the element B of the other input matrix. The adder 112 adds the output of the multiplier 111 and the output of the mux 115. The four muxes 113 respectively select and output either Cin that has accumulated the output C within the PE11 or the output of the acc114 and the output of the adder 112. For example, if the four muxes 113 are numbered mux1131 to 1134 from left to right, in a certain cycle, to correspond to the group of input A, mux1131 selects the output of the adder 112, and the remaining muxes 1132 to 1134 respectively select their own outputs of the acc114. In the next cycle, the second mux 1132 from the left selects the output of the adder 112, and the remaining muxes 1131, 1133 to 1134 respectively select their own outputs of the acc114. By circulating and selecting the output of the adder 112 every cycle like this, the rectangular area is switched. The four acc114 respectively hold the outputs of the four muxes 113. The mux 115 selects one of the outputs of the four acc114 as the calculation result C.

[0085] In other words, the 1.5 - dimensional MMA calculator 1 performs the L×M×N matrix - product operation C = A*B or the L×M×N matrix - product - sum operation C = A*B + Cin for the L×N matrix A, the N×M matrix B, the L×M matrix C, and Cin by accumulating the outer product Ok of the k - th column of A and the k - th row of B for 0≦k<N in an array of L×M accumulators, and includes an array of l×m product - sum calculators 11 where either l or m is l = L or m = M and the other is 2≦l<L or 2≦m<M, and the l×m product - sum calculators 11 perform the accumulation of the outer product Ok in multiple steps. Also, when the l×m product - sum calculators perform the accumulation of the outer product Ok in multiple steps, they execute the multiple steps in a pipeline.

[0086] So far, for the sake of simplicity, the examples have mainly been explained using the matrix product of square matrices. However, the two matrices may be matrices of L×N and N×M respectively (L, M, and N are integers greater than or equal to 3). A plurality of PE11s may be provided with l×m (l is an integer greater than or equal to 2 and less than L, m = M). Each of the plurality of PE11s may be provided with L / l or more accumulators acc114. (l and m may be interchanged)

[0087] FIG. 18 is a block diagram schematically showing a configuration example of PE11a corresponding to FIG. 13 having a thickness in the k-axis direction.

[0088] The 1.5D MMA calculator 1a shown in FIG. 18 may include n (n is an integer greater than or equal to 1 and less than or equal to N) PE11as (in other words, FMA calculators) for each of the plurality of PE11as. The accumulator acc114 is shared by n FMA calculators and processes n outer products in parallel. FIG. 18 shows an example of l×m×n = 2×8×2, and two outer products will be processed in parallel. Also, in FIG. 18, the adder 112 has a three-input configuration shared by two multipliers 111, but it may be configured as a tree of ordinary two-input adders.

[0089] 〔C〕Effect According to the matrix product-sum calculator in the above-described embodiment, for example, the following operational effects can be achieved.

[0090] When L and M are both integers greater than or equal to 2 and N is an integer greater than or equal to 1, the 1.5D MMA calculator 1 accumulates the outer product Ok of the k-th column of matrix A and the k-th row of matrix B into an array of L×M accumulators for integers k from 0 to less than N, thereby performing an L×M×N matrix product operation C = A*B, or an L×M×N matrix product-sum operation C = A*B + Cin. It includes an array of l×m product-sum calculators, where either l or m is l = L or m = M, and the other is an integer such that 2≤l<L or 2≤m<M. The l×m product-sum calculators perform the accumulation of the outer product Ok in multiple steps.

[0091] This makes it possible to reduce the number of accesses to the RF per multiply-and-accumulate operation.

[0092] In the 1.5-dimensional MMA processor 1, each of the l × m multiply-accumulate processors has the function of performing n multiplications (n ​​is an integer between 2 and N, N is an integer greater than or equal to 2) and accumulating the results of these multiplications, and the l × m multiply-accumulate processors process the accumulation of n outer products in parallel. This makes it possible to appropriately set the number of FMA processors included in PE 11, even when one step of processing has thickness in the k-axis direction, as shown in Figure 13, and to reduce the number of RF accesses per FMA operation.

[0093] The l×m multiply-add units execute the multiple steps in a pipeline manner when accumulating the cross product Ok in multiple steps, thereby enabling the accumulation of the cross product Ok to be calculated appropriately.

[0094] [D] Other The disclosed technology is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present embodiment. The configurations and processes of the present embodiment can be selected or combined as needed.

[0095] In the above-described embodiment, a multiply-and-accumulate operation is performed between square matrices, but this is not limiting. In the above-described embodiment, a multiply-and-accumulate operation may be performed between a square matrix and a matrix other than a square matrix, or between matrices other than a square matrix. [Explanation of symbols]

[0096] 1: 1.5-dimensional MMA calculator 11,61:PE 111,611: Multiplier 112,613: Adder 113,115,612:mux 114,614:acc 60: Processing unit 6: MMA calculator 7: Buffer / RF 8: Local memory / cache 9: Main memory

Claims

1. an array of l×m multiply-add operators that performs an L×M×N matrix multiplication operation C=A*B or an L×M×N matrix multiply-add operation C=A*B+Cin by accumulating an outer product Ok of column k of A and row k of B in an array of L×M accumulators, where L and M are both integers of 2 or greater, and N is an integer of 1 or greater, for an L×N matrix A, an N×M matrix B, an L×M matrix C, and Cin, for an integer k greater than or equal to 0 and less than N; One of l and m is l=L or m=M, and the remaining one is an integer satisfying 2≦l<L or 2≦m<M; The l×m multiply-add calculators accumulate the outer product Ok in multiple steps. Arithmetic unit.

2. Each of the l×m multiply-accumulate units has a function of performing n multiplications (n ​​is an integer between 2 and N, and N is an integer greater than or equal to 2) and accumulating the results of these multiplications; The l×m multiply-accumulate units process the accumulation of n outer products in parallel. The computing unit according to claim 1 .

3. The l×m multiply-add calculators execute the multiple steps in a pipeline when accumulating the outer product Ok in multiple steps.

3. The computing unit according to claim 1 or 2.

Images