Systolic array matrix multiplier and operation method of systolic array matrix multiplier

Abstract

To execute various sizes of matrix multiplication while suppressing increase in circuit scale.SOLUTION: A systolic array matrix multiplier includes a plurality of processing elements arranged in a matrix and executes matrix multiplication. Each of the plurality of processing elements includes a first holding unit which sequentially holds respective elements of a first matrix received from a first input terminal provided in one end side in a first direction, a first path which outputs an output of the first holding unit to a first output terminal provided in the other end side in the first direction, a second holding unit which sequentially holds respective elements of the first matrix received from a second input terminal provided in the other end side in the first direction, a second path which outputs an output of the second holding unit to a second output terminal provided in one end side in the first direction, a product-sum operator connected to the first path, a first selection unit which connects the first path or the first output terminal to the second path, and a second selection unit which connects the second path or an output of the first holding unit to the first path.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a systolic array matrix multiplier and a method of operating a systolic array matrix multiplier. [Background technology]

[0002] Matrix operations are often used in scientific and technical calculations, machine learning, and the like. It is known that there is a limit to the performance improvement of large-scale matrix operations using a general-purpose CPU. Therefore, a systolic array accelerator has been proposed that arranges multiple processing elements vertically and horizontally to perform large-scale matrix multiplication at high speed (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent Application Publication No. 2018-0267936 Summary of the Invention [Problem to be solved by the invention]

[0004] The scale of the systolic array increases as the size of the matrix to be operated on increases. For example, when an operation on a matrix smaller than the maximum size that can be performed by the systolic array is performed by the systolic array, some processing elements are used only to transfer the elements of the matrix.

[0005] In this case, the multiply-accumulate unit included in the processing element executes an unnecessary operation (0×0+C) so that the multiply-accumulate result C transferred from upstream is not changed. Since there are fewer processing elements that actually execute operations, the processing efficiency of operations decreases.

[0006] By dividing the processing elements in a systolic array, it becomes possible to perform operations on matrices of various sizes. However, this requires memory for each group of divided processing elements, which increases the circuit size.

[0007] Furthermore, when inputting data used in an operation into a systolic array and outputting the operation results from the systolic array in parallel, there is a risk that the operation performance will be reduced due to bandwidth limitations of the bus connected to the systolic array.

[0008] In one aspect, the present invention aims to provide a systolic array type matrix multiplier capable of suppressing a decrease in arithmetic performance due to a bus bandwidth limitation. [Means for solving the problem]

[0009] According to one aspect, a systolic array-type matrix multiplier includes a plurality of processing elements arranged in a matrix and performs matrix multiplication, wherein each of the plurality of processing elements has: a first holding unit that sequentially holds each element of a first matrix received from a first input terminal provided at one end in a first direction; a first path that outputs an output of the first holding unit to a first output terminal provided at the other end in the first direction; a second holding unit that sequentially holds each element of the first matrix received from a second input terminal provided at the other end in the first direction; a second path that outputs an output of the second holding unit to a second output terminal provided at one end in the first direction; a multiply-accumulate unit connected to the first path; a first selection unit that connects the first path or the first output terminal to the second path; and a second selection unit that connects the second path or the output of the first holding unit to the first path. [Effects of the Invention]

[0010] It is possible to provide a systolic array type matrix multiplier that can suppress a decrease in arithmetic performance due to bus bandwidth limitations. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 2 is a block diagram illustrating an example of a systolic array type matrix multiplier according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a processing element of FIG. 1. [Figure 3] FIG. 2 is a diagram illustrating an example of matrix multiplication performed by the systolic array of FIG. [Figure 4] 2 is a block diagram showing an example of an information processing device in which the systolic array of FIG. 1 is installed. [Figure 5] FIG. 10 is a block diagram showing an example of another systolic array type matrix multiplier. [Figure 6] FIG. 6 is a diagram illustrating an outline of the operation of the systolic array type matrix multiplier of FIG. 5. [Figure 7] FIG. 10 is a block diagram showing an example of yet another systolic array type matrix multiplier. [Figure 8] FIG. 2 is a diagram illustrating an example of the operation of the systolic array type matrix multiplier of FIG. [Figure 9] FIG. 9 is a diagram showing a continuation of FIG. 8. [Figure 10] FIG. 10 is a diagram showing a continuation of FIG. 9. [Figure 11] FIG. 11 is a diagram showing a continuation of FIG. [Figure 12] FIG. 12 is a diagram showing a continuation of FIG. [Figure 13] FIG. 13 is a diagram showing a continuation of FIG. 12. [Figure 14] FIG. 14 is a diagram showing a continuation of FIG. [Figure 15] FIG. 15 is a diagram showing a continuation of FIG. [Figure 16] FIG. 16 is a diagram showing a continuation of FIG. [Figure 17] FIG. 17 is a diagram showing the continuation of FIG. [Figure 18] FIG. 18 is a diagram showing a continuation of FIG. 17. [Figure 19] 2 is a diagram showing an example of values ​​output by each flip-flop of the upper right PE+ in the systolic array of FIG. 1. FIG. [Figure 20]FIG. 2 is a diagram illustrating another example of the operation of the systolic array type matrix multiplier of FIG. [Figure 21] FIG. 21 is a diagram showing a continuation of FIG. 20. [Figure 22] FIG. 22 is a diagram showing a continuation of FIG. 21. [Figure 23] FIG. 23 is a diagram showing a continuation of FIG. 22. [Figure 24] This is a diagram showing a continuation of Figure 23. [Figure 25] This is a diagram showing the continuation of Figure 24. [Figure 26] This is a diagram showing the continuation of Figure 25. [Figure 27] FIG. 2 is a diagram illustrating an example of the operation when the systolic array of FIG. 1 is divided into four, 2 rows and 2 columns, and four matrix multiplications are performed in parallel. [Figure 28] This is a diagram showing the continuation of Figure 27. [Figure 29] This is a figure showing the continuation of Figure 28. [Figure 30] This is a diagram showing the continuation of Figure 29. [Figure 31] This is a figure showing the continuation of Figure 30. [Figure 32] This is a figure showing the continuation of Figure 31. [Figure 33] This is a figure showing the continuation of Figure 32. [Figure 34] FIG. 8 is a diagram illustrating features of the systolic array type matrix multipliers of FIGS. 2, 5 and 7. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments will be described with reference to the drawings.

[0013] Fig. 1 shows an example of a systolic array-type matrix multiplier according to an embodiment. The systolic array-type matrix multiplier 100 shown in Fig. 1 includes a plurality of processing elements PE+ arranged in a matrix and a bus BUS connected to the surrounding processing elements PE+. Hereinafter, the systolic array-type matrix multiplier 100 will also be referred to as a systolic array 100, and the processing elements PE+ will also be referred to as PE+.

[0014] Systolic array 100 multiplies matrix A with m rows and k columns by matrix B with k rows and n columns, and adds the m rows and n columns multiplication result to matrix C with m rows and n columns by having multiple PEs+ perform calculations using a bucket brigade of data (for example, m, k, and n are integers equal to or greater than 2). Note that, for simplicity of explanation, it is assumed below that m=n=k. Matrix A is an example of a first matrix, and matrix C is an example of a second matrix.

[0015] Also, while FIG. 1 shows an example in which systolic array 100 has 16 PE+s arranged in 4 rows and 4 columns with k=4 and n=4, systolic array 100 may have more PE+s, such as 8 rows and 8 columns or 16 rows and 16 columns. The number of rows and columns may also be different. Although FIG. 1 shows fold paths around some of the PE+s, in reality, the fold paths are included inside the PE+s. An example of the internal configuration of a PE+ is shown in FIG. 2.

[0016] In each PE+, the right input terminal A1R and the left input terminal A2L can input each element of matrix A, and the left output terminal A1L and the right output terminal A2R can output each element of matrix A. In each PE+, the upper input terminal BU can input each element of matrix B, and the lower output terminal BD can output each element of matrix B. In each PE+, the upper input terminal C2U and the lower input terminal C1D can input each element of matrix C, and the lower output terminal C2D and the upper output terminal C1U can output each element of matrix C. Hereinafter, the elements included in each of matrices A, B, and C are also referred to as data.

[0017] The input terminal A1R is an example of a first input terminal, and the output terminal A1L is an example of a first output terminal. The input terminal A2L is an example of a second input terminal, and the output terminal A2R is an example of a second output terminal. The input terminal C2U is an example of a third input terminal, and the output terminal C2D is an example of a third output terminal. The input terminal C1D is an example of a fourth input terminal, and the output terminal C1U is an example of a fourth output terminal.

[0018] For example, in the systolic array 100 shown in Figure 1, data of matrix A and data of matrix C are passed in a bucket brigade from the upper right PE+ to the lower left PE+. Data of matrix B is transferred to each PE+ in advance. For example, each PE+ executes the calculation "A x B + C" using the multiply-accumulate unit FMA shown in Figure 2, and transfers the calculation result as data of matrix C to the PE+ below.

[0019] Figure 2 shows an example of a processing element PE+ in Figure 1. PE+ has flip-flops FF1, FF2, FF3, FF4, and FF5, multiplexers MUX1, MUX2, MUX3, MUX4, MUX5, and MUX6, and a multiply-accumulate unit FMA. The multiply-accumulate unit FMA multiplies data of matrix A and data of matrix B using a multiplier, and adds the multiplication result to data of matrix C using an adder.

[0020] Each of the flip-flops FF1, FF2, FF3, FF4, and FF5 can hold one element of a matrix. For example, the multiplexers MUX1-MUX6 of each PE+ can be controlled independently of each other by a sequencer disposed outside the systolic array 100. Hereinafter, the flip-flops FF1, FF2, FF3, FF4, and FF5 will also be referred to as FF1, FF2, FF3, FF4, and FF5, respectively. The multiplexers MUX1, MUX2, MUX3, MUX4, MUX5, and MUX6 will also be referred to as MUX1, MUX2, MUX3, MUX4, MUX5, and MUX6, respectively. The multiply-accumulate unit FMA will also be referred to as FMA.

[0021] FF1 is an example of a first holding unit, FF2 is an example of a second holding unit, FF3 is an example of a third holding unit, and FF4 is an example of a fourth holding unit. Multiplexer MUX1 is an example of a first selection unit, and multiplexer MUX2 is an example of a second selection unit. Multiplexer MUX3 is an example of a third selection unit, and multiplexer MUX4 is an example of a fourth selection unit. Multiplexer MUX5 is an example of a fifth selection unit.

[0022] MUX6, FF1, and MUX2 are arranged in series between the right-side input terminal A1R and the left-side output terminal A1L. MUX6 connects either the right-side input terminal A1R or the output of FF2 to the input of FF1. MUX2 connects either the output of FF1 or the output of FF2 to the multiplication input of FMA, the left-side output terminal A1L, and the input of MUX1. The path connecting the output of MUX2 to the input of FMA, the output terminal A1L, and the input of MUX1 is an example of a first path.

[0023] MUX1 and FF2 are arranged in series between the left input terminal A2L and the right output terminal A2R. MUX1 connects either the left input terminal A2L or the output of MUX2 to the input of FF2. The output of FF2 is connected to the right output terminal A2R, the input of MUX6, and the input of MUX2. The path connecting the output of FF2 to the output terminal A2R, the input of MUX6, and the input of MUX2 is an example of a second path.

[0024] FF5 is arranged between the upper input terminal BU and the lower output terminal BD. The output of FF5 is connected to one input of the multiplier of the FMA and the lower output terminal BD. Note that before the matrix multiplication starts, the data of matrix B is transferred sequentially, for example, from the upper PE+ to the lower PE+, and held in FF5 of each PE+.

[0025] MUX4, FMA, MUX5, and FF3 are arranged in series between the upper input terminal C2U and the lower output terminal C2D. MUX4 connects either the upper input terminal C2U or the output of FF4 to one input of the adder of FMA. The path connecting the output of MUX4 to the input of FMA and the input of MUX5 is an example of a third path. The path connecting the output of FF4 to the output terminal C1U and the input of MUX4 is an example of a fourth path.

[0026] MUX3 and FF4 are arranged in series between the lower input terminal C1D and the upper output terminal C1U. MUX3 connects either the lower input terminal C1D or the output of MUX5 to the input of FF4.

[0027] FMA multiplies the data of matrix A output from MUX2 by the data of matrix B output from FF5 in a multiplier and outputs the multiplication result to the other input of the adder. The addition result by the adder is output to MUX5. MUX5 connects either the output of FMA or the output of MUX4 to the input of FF3 and the input of MUX3.

[0028] With the above configuration, PE+ can not only output the data of matrix A held in FF2 to output terminal A2R, but also supply it to MUX2, or have it held in FF1 via MUX6. PE+ can not only output the data of matrix A output from MUX2 to output terminal A1L, but also have it held in FF2 via MUX1. That is, systolic array 100 can fold back the data of matrix A transferred from left to right to the left at any PE+, and can fold back the data of matrix A transferred from right to left to the right at any PE+.

[0029] Furthermore, the PE+ can not only output the result of the product-sum operation by the FMA as data of matrix C to output terminal C2D, but also store it in FF4 via MUX3. Alternatively, the PE+ can not only output the data of matrix C stored in FF4 to output terminal C1U, but also output it to the FMA and MUX5 via MUX4. That is, the systolic array 100 can fold back the data of matrix C transferred from the top to the bottom to the top at any PE+, and can fold back the data of matrix C transferred from the bottom to the top to the bottom at any PE+.

[0030] Fig. 3 shows an example of matrix multiplication performed by the systolic array 100 of Fig. 1. In the example shown in Fig. 3, matrix A with m rows and k columns is multiplied by matrix B with k rows and n columns to obtain a matrix product with m rows and n columns, and matrix C with m rows and n columns is added to the obtained matrix product to obtain a new matrix C.

[0031] Fig. 4 shows an example of an information processing device 200 equipped with the systolic array 100 of Fig. 1. The information processing device 200 is used for learning or inference such as image processing using a neural network, or for both learning and inference.

[0032] For example, the information processing device 200 is a server, and includes a CPU 210, a systolic array 100, a memory 220, an auxiliary storage device 230, a communication interface 240, and an input / output interface 250, all of which are interconnected by a bus such as a memory bus. The information processing device 200 may include multiple systolic arrays 100, or may include elements other than those shown in the figure. The systolic array 100 may also be mounted on the information processing device 200 as an accelerator.

[0033] The CPU 210 controls the entire information processing device 200, and may generate a data string (matrix elements) for causing the systolic array 100 to perform matrix multiplication, and transfer the data string to the systolic array 100 via the bus BUS. The CPU 210 may receive the result of the matrix multiplication performed by the systolic array 100 via the bus BUS.

[0034] The systolic array 100 holds data received from the CPU 210 etc. via the bus BUS in FFs, and performs matrix multiplication using the data held in the FFs. During the execution of the matrix multiplication, data of matrix C, which is the calculation result, is output from the systolic array 100 to the bus BUS, but there is no need to input data of matrices A and C to the systolic array 100 from the bus BUS.

[0035] That is, while the systolic array 100 is executing the matrix multiplication, no data is input from the memory 220 to the systolic array 100. This makes it possible to reduce the bandwidth of the bus BUS between the systolic array 100 and the memory 220 compared to when the data of matrices A and C are input to the systolic array 100 during the execution of the matrix multiplication. As a result, it is possible to suppress a decrease in the operation performance due to the bandwidth limitation of the bus BUS.

[0036] The memory 220 may store target data for matrix multiplication (matrices A, B, C), an execution result of the matrix multiplication (matrix C), various programs, etc. The auxiliary storage device 230 may store an OS (Operating System) executed by the CPU 210 and various programs such as an information processing program that operates the information processing device 200.

[0037] For example, the program stored in the auxiliary storage device 230 may be transferred to the memory 220 and executed by the CPU 210. Furthermore, the data and various variables used in the neural network calculations stored in the auxiliary storage device 230 may be transferred from the auxiliary storage device 230 to the memory 220 before training of the neural network is performed or before inference using the neural network is performed.

[0038] The communication interface 240 may have a function for communicating with other information processing devices, for example, via a network. This enables neural network calculations to be executed in parallel using multiple information processing devices. The input / output interface 250 may have a function for inputting and outputting data, programs, etc. to and from a recording medium 300 connected to the information processing device 200. A program recorded on the recording medium 300 may be transferred to the auxiliary storage device 230 via the input / output interface 250, then deployed on the memory 220 and executed by the CPU 210.

[0039] Fig. 5 shows an example of another systolic array-type matrix multiplier. The systolic array-type matrix multiplier 102 shown in Fig. 5 has 16 processing elements PE arranged in 4 rows and 4 columns, and is connected to a memory MEM1 for matrix A, a memory MEM2 for matrix B, and a memory MEM3 for matrix C. Hereinafter, the systolic array-type matrix multiplier 102 will also be referred to as a systolic array 102, and the processing elements PE will also be referred to as PEs. Note that the systolic array 102 may have k rows and n columns.

[0040] For example, memories MEM1, MEM2, and MEM3 may be scratchpad memories. Memory MEM1 stores data for matrix A with m rows and k columns, memory MEM2 stores data for matrix B with k rows and n columns, and memory MEM3 stores data for matrix C with m rows and n columns.

[0041] Each PE has FF6, FF7, FF8 and FMA. Each of FF6, FF7, and FF8 can hold one element of a matrix. Unlike the PE+ shown in Figure 2, the PE does not have a path for transferring data of matrix A from left to right, or a path for transferring data of matrix C from bottom to top.

[0042] FF6 holds the data of matrix A received at input AR and outputs the held data to FMA and output terminal AL. FF7 holds the calculation result by FMA and outputs the held calculation result to output terminal CD. Input terminal CU receives the data of matrix C. FF8 holds the data of matrix B received at input terminal BU and outputs the held data to FMA and output terminal BD.

[0043] For the four PEs in each row, data of matrix A is read from memory MEM1 arranged on the right side of systolic array 102 and is transferred sequentially from the right PE to the left PE, and is held in FF6. For the four PEs in each column, data of matrix B is read from memory MEM2 arranged on the upper side of systolic array 102 and is transferred sequentially from the upper PE to the lower PE, and is held in FF8. Furthermore, for the four PEs in each column, data of matrix C is read from memory MEM3 arranged on the lower side of systolic array 102 and is transferred sequentially from the upper PE to the lower PE, and is input to each FMA. The result of the product-sum operation by the FMA is held in FF7.

[0044] Fig. 6 shows an overview of the operation of the systolic array type matrix multiplier 102 in Fig. 5. Before performing a matrix multiplication, 16 pieces of data a, 16 pieces of data b, and 16 pieces of data c contained in matrices A, B, and C of the matrix multiplication to be performed are stored in advance in memories MEM1, MEM2, and MEM3, respectively.

[0045] First, the data of matrix B stored in memory MEM2 is transferred in a bucket brigade fashion between the FF8s of the PEs arranged vertically (Figure 5) every clock, and the FF8s of each PE hold one of the data B0,0 to B3,3 of matrix B.

[0046] Next, the data of matrix A stored in memory MEM1 is transferred in a bucket brigade for each row of PEs, with a one-clock delay, and the data of matrix C stored in memory MEM3 is transferred in a bucket brigade for each column of PEs, with a one-clock delay. For example, in clock cycles t=0 to 3, the data of matrix A is transferred sequentially to the top row of PEs in FIG. 6, and the data of matrix C is transferred sequentially to the rightmost column in FIG. 6. Similarly, in clock cycles t=1 to 4, the data of matrix A is transferred sequentially to the second row of PEs from the top in FIG. 6, and the data of matrix C is transferred sequentially to the second column from the right in FIG. 6.

[0047] Each PE multiplies one element of matrix A by one element of matrix B, adds the multiplication result to an element of matrix C, which is a partial sum transferred from the PE immediately above, and transfers the addition result to the PE immediately below. For example, if there are k PEs in the column direction, the matrix C input from the top of the systolic array 102 is i,j When the data is output from the bottom PE, C i,j =C i,j +Σ(K=0,1,…,k-1)a i,K ×b K,j and stored in the memory MEM3 for matrix C.

[0048] When four matrix multiplications of two rows and two columns ((m, n, k)=(2, 2, 2)) are performed in the systolic array 102, the four PEs of two rows and two columns out of the 16 PEs are used to perform the matrix multiplications four times. The FMA of the PE not used for the matrix multiplications executes an unnecessary operation (0×0+0).

[0049] For example, when a 2x2 matrix multiplication is performed using a systolic array including four PEs (2x2), the operation result can be output in four clock cycles.In contrast, a 2x2 matrix multiplication using the systolic array 102 takes six clock cycles, the same as the processing time for a 4x4 matrix multiplication.

[0050] Fig. 7 shows another example of a systolic array-type matrix multiplier. The systolic array-type matrix multiplier 104 shown in Fig. 7 is capable of performing both a 4x4 matrix multiplication by 16 PEs and a 2x2 matrix multiplication by four PEs, so that a memory MEM is assigned to each 2x2 PE. The symbols A, B, and C attached to the memory MEM indicate that the memory MEM stores matrix A, matrix B, and matrix C, respectively. Hereinafter, the systolic array-type matrix multiplier 104 will also be referred to as a systolic array 104.

[0051] 5, the systolic array 104 requires the addition of memory MEM, which is shown shaded, resulting in an increase in the memory mounting area and the wiring between the PEs and the bus BUS. Note that the systolic array 104 can perform 4x4 matrix multiplication and 2x2 matrix multiplication in the shortest processing time.

[0052] Furthermore, for example, if a systolic array including 64 PEs (8 rows and 8 columns) is used to perform one 8x8 matrix multiplication, four 4x4 matrix multiplications, and sixteen 2x2 matrix multiplications, the implementation area of ​​the memory MEM and the wiring between the PEs and the bus BUS will increase further. As the number of PEs increases, the implementation area of ​​the memory MEM and the wiring between the PEs and the bus BUS also increase, making the systolic array 104 impractical.

[0053] 8 to 18 show an example of the operation of the systolic array-type matrix multiplier 100 of FIG. 1. For example, FIGS. 8 to 18 show an example of a case where a 4-row, 4-column matrix multiplication C=A×B+C is executed once. In FIGS. 8 to 18, only the FF and FMA are shown, and the data paths of matrices A, B, and C input to the FMA and the output path of the FMA are omitted. The symbol t indicates a clock cycle. Furthermore, data c of matrix C is updated sequentially because the multiplication results of data a and b are sequentially added.

[0054] The 16 elements of matrix A are represented by a(3,0)~a(0,0), a(3,1)~a(0,1), a(3,2)~a(0,2), and a(3,3)~a(0,3) from the top row. The 16 elements of matrix B are represented by b(0,0)~b(3,0), b(0,1)~b(3,1), b(0,2)~b(3,2), and b(0,3)~b(3,3) from the right column. The 16 elements of matrix C are represented by c(0,0)~c(3,0), c(0,1)~c(3,1), c(0,2)~c(3,2), and c(0,3)~c(3,3) from the right column.

[0055] Figure 8 (clock cycle t=0) shows the initial state in which the data of matrices A, B, and C are held in FF2, FF5, and FF4 of each PE+. The data shown within the frame of each FF indicates that it is held in the FF and output from the FF. From Figure 8 onwards, no data is input or output between systolic array 100 and bus BUS, except for the output of the calculation results in FMA to bus BUS.

[0056] The following explanation will also refer to Figure 2. The PE+ in the rightmost column, which folds the elements of matrix A to the left, selects input terminal A2L with multiplexer MUX1 and selects the output of FF2 with multiplexer MUX2. The PE+ in the leftmost column, which folds the elements of matrix A to the right, selects the output of FF1 with multiplexer MUX2 and selects the output of multiplexer MUX2 with multiplexer MUX1.

[0057] The PE+ in the top row, which folds the elements of matrix C downward, selects input terminal C1D with multiplexer MUX3, selects the output of FF4 with multiplexer MUX4, and selects the output of FMA with multiplexer MUX5. The PE+ in the bottom row, which folds the elements of matrix C upward, selects input terminal C2U with multiplexer MUX4, selects the output of FMA with multiplexer MUX5, and selects the output of multiplexer MUX5 with multiplexer MUX3.

[0058] The central PE+, which is not adjacent to the right, left, top, or bottom ends, selects input terminal A1R with multiplexer MUX6, selects the output of FF1 with multiplexer MUX2, and selects input terminal A2L with multiplexer MUX1.The central PE+ also selects input terminal C2U with multiplexer MUX4, selects the output of FMA with multiplexer MUX5, and selects input terminal C1D with multiplexer MUX3.

[0059] The dashed arrows in Figures 9 to 18 indicate that data is transferred from the transfer source to the transfer destination. The shaded FFs in Figures 9 to 18 indicate that new data has been stored. The result of adding the multiplication result of FMA and data c is shown as data c.

[0060] In Figure 9 (clock cycle t=1), data a(0,0) from the PE+ in the top row is transferred to the PE+ one block to the left, and data a(1,0), a(2,0), and a(3,0) are transferred to the PE+ one block to the right (Figures 9(a), (b), (c), and (d)). Also, in the PE+ in the top right, data c(0,0) output from FF4 is added to the multiplication result, and the addition result is stored in FF3 as data c(0,0) (Figure 9(e)). Data c(1,0), c(2,0), and data c(3,0) from the PE+ in the rightmost column are transferred to the PE+ one block above (Figures 9(f), (g), and (h)).

[0061] In Figure 10 (clock cycle t=2), data a(0,0) and a(1,0) from the PE+ in the top row are transferred to the PE+ one position to the left, and data a(2,0) and a(3,0) are transferred to the PE+ one position to the right (Figures 10(a), (b), (c), (d)). Data a(1,0) from the PE+ in the second row from the top is transferred to the PE+ one position to the left, and data a(1,1), a(2,1), and a(3,1) are transferred to the PE+ one position to the right (Figures 10(e), (f), (g), (h)).

[0062] In addition, the data c(0,0) of the upper right PE+ is added to the multiplication result of the PE+ one level below, and the addition result is stored in FF3 as c(0,0) (Fig. 10(i)). The data c(1,0) of the upper right PE+ is added to the multiplication result, and the addition result is stored in FF3 as c(1,0) (Fig. 10(j)). The data c(2,0) and (3,0) of the PE+ in the rightmost column are transferred to the PE+ one level above (Fig. 10(k) and (l)).

[0063] In the top PE+ in the second column from the right, data c(0,1) is added to the multiplication result, and the addition result is stored in FF3 as data c(0,1) (Fig. 10(m)). Data c(1,1), c(2,1), and c(3,1) in the PE+ in the second column from the right are transferred to the PE+ one level above (Fig. 10(n), (o), (p)).

[0064] 11 (clock cycle t=3) to 18 (clock cycle t=10), as in FIGS. 9 and 10, data a in each row is transferred to the PE+ one step to the right, and then folded back and transferred to the PE+ one step to the left. For example, in FIG. 12, data a(0,0) held in FF1 of the upper left PE+ is transferred to FF2 (FIG. 12(a)). This allows the elements of matrix A, which are transferred sequentially from right to left in FIG. 12, to be folded back within systolic array 100 and transferred sequentially to the PE+ on the right.

[0065] Data c of each column is transferred to the PE+ one level above, then folded back at the topmost PE+ and added in the FMA, and the addition result is stored in FF3 as data c. The addition result stored in FF3 is transferred to the PE+ one level below in the next clock cycle and added in the FMA. In Figure 12 (clock cycle t=4) to Figure 18 (clock cycle t=10), the matrix multiplication results held in FF3 in the PE+ in the bottom row are output to the bus BUS starting from the PE+ on the right.

[0066] The multiplexer MUX3 of the PE+ in the bottom row (FIG. 2) selects the output of the multiplexer MUX5. As a result, for example, in the PE+ in the bottom right of FIG. 12, the matrix multiplication result c(0,0) can be stored in FF4 as well, and the elements of the matrix C, which is the matrix multiplication result, can be folded back within the systolic array 100 and transferred sequentially to the PE+ above (FIG. 12(b)).

[0067] 18, data a and c held in each PE+ return to the state at clock cycle t=0 in FIG. 8. Data b is held in the same state as at clock cycle t=0. Therefore, the next matrix multiplication can be started without re-inputting data a, b, and c from the bus BUS to each PE+, and an increase in the bandwidth of the bus BUS can be suppressed.

[0068] 8 to 18, by folding back data a at the rightmost PE+, data a can be supplied to each PE+ without using the bus BUS during execution of the matrix multiplication. Also, by folding back data c at the topmost PE+, data c can be supplied to each PE+ without using the bus BUS during execution of the matrix multiplication. This makes it possible to reduce the bandwidth of the bus BUS compared to, for example, when data a and c are input to systolic array 100 via the bus BUS while the matrix multiplication is being executed.

[0069] 19 shows an example of values ​​output by FF1 to FF5 of the upper right PE+ in the systolic array 100 of FIG. 1. FF1 of the upper right PE+ is used when inputting data a into the PE+ via the bus BUS, and is not used during the execution of the matrix multiplication. FF2 sequentially holds data a(0,0), a(1,0), a(2,0), and a(3,0) to be used in the multiply-and-accumulate operation from clock cycle t=0 to t=3, and outputs them to the FMA. FF2 also holds data a(0,0) folded back from the right PE+ at clock cycle t=7, and continues to hold it until clock cycle t=10.

[0070] FF3 sequentially holds data c(0,0), c(1,0), c(2,0), and c(3,0), which are the results of the multiply-and-accumulate operation, from clock cycles t=1 to t=4, and outputs them to the PE+ one level below. FF4 sequentially holds data c(0,0), c(1,0), c(2,0), and c(3,0), which are used in the multiply-and-accumulate operation, from clock cycles t=0 to t=3, and outputs them to FMA. FF4 also holds data c(0,0), which is the result of the multiply-and-accumulate operation, from clock cycle t=7, and continues to hold it until clock cycle t=10. FF5 continues to hold data b(0,0) without transferring it.

[0071] 20 to 26 show another example of the operation of the systolic array type matrix multiplier 100 of Fig. 1. Fig. 20 to 26 show an example of loading data of matrices A' and C' into the systolic array 100 when, after performing a 4-by-4 matrix multiplication C=A×B+C, another 4-by-4 matrix multiplication C'=A'×B+C' is performed using the values ​​of matrix B in common. Fig. 20 to 26 show the operation from time t=4 onwards, which follows clock cycle t=3 in Fig. 11. The operation from clock cycles t=0 to t=3 is the same as that in Figs. 8 to 11.

[0072] The PE+ in the top row receives data a'(0,0), a'(1,0), a'(2,0), and a'(3,0) sequentially from clock cycle t=4 to t=7, and these are held in FF2 of each PE+. The PE+ in the second row receives data a'(0,1), a'(1,1), a'(2,1), and a'(3,1) sequentially from clock cycle t=5 to t=8, and these are held in FF2 of each PE+.

[0073] The PE+ in the third row from the top receives data a'(0,2), a'(1,2), a'(2,2), and a'(3,2) sequentially from clock cycle t=6 to t=9, and these are held in FF2 of each PE+. The PE+ in the bottom row receives data a'(0,3), a'(1,3), a'(2,3), and a'(3,3) sequentially from clock cycle t=7 to t=10, and these are held in FF2 of each PE+.

[0074] The PE+ in the rightmost column receives data c'(0,0), c'(1,0), c'(2,0), and c'(3,0) sequentially from clock cycle t=4 to t=7, and these are stored in the FF4 of each PE+. The PE+ in the second column from the right receives data c'(0,1), c'(1,1), c'(2,1), and c'(3,1) sequentially from clock cycle t=5 to t=8, and these are stored in the FF4 of each PE+.

[0075] The PE+ in the third column from the right receives data c'(0,2), c'(1,2), c'(2,2), and c'(3,2) sequentially from clock cycle t=6 to t=9, and these are stored in the FF4 of each PE+. The PE+ in the bottom row receives data c'(0,3), c'(1,3), c'(2,3), and c'(3,3) sequentially from clock cycle t=7 to t=10, and these are stored in the FF4 of each PE+.

[0076] 20 to 26, the systolic array 100 can input data a' and data c' to be used in the next matrix multiplication to each PE+ via the bus BUS while the matrix multiplication is being performed, thereby improving the usability of the systolic array 100.

[0077] 27 to 33 show an example of the operation when the systolic array 100 in FIG. 1 is divided into four 2-row, 2-column matrix multiplications and four 2-row, 2-column matrix multiplications are performed in parallel. Detailed explanations of operations similar to those in FIGS. 8 to 18 will be omitted. Hereinafter, a systolic array including four PE+s arranged in 2 rows and 2 columns will be referred to as a subarray.

[0078] 27 to 33 show an example in which each subarray executes a 2x2 matrix multiplication C = A × B + C once and transfers the multiplication result to the bus BUS. The multiplication results of the upper two subarrays are transferred to the bus BUS via the lower two subarrays. The initial state (clock cycle t = 0 in Fig. 27) of data a, b, c of matrices A, B, C held by the four PE+ in each subarray is the same as the data a, b, c of matrices A, B, C held by the four PE+ in the upper right corner of Fig. 8.

[0079] Each subarray performs matrix multiplication using data a(0,0), a(1,0), a(0,1), a(1,1) of matrix A, data b(0,0), b(1,0), b(0,1), b(1,1) of matrix B, and data c(0,0), c(1,0), c(0,1), c(1,1) of matrix C. Note that the data values ​​of matrix A used by the four subarrays in the matrix multiplication may be different from each other. The data values ​​of matrix B used by the four subarrays in the matrix multiplication may be different from each other. The data of matrix C used by the four subarrays in the matrix multiplication may be different from each other.

[0080] The operation of the two sub-arrays on the left side is the same as that of the two sub-arrays on the right side. Therefore, the operation of the two sub-arrays on the right side will be explained below. Also, for ease of explanation, the eight PE+s included in the two sub-arrays on the right and left sides are assigned identification numbers (0) to (7).

[0081] The upper and lower subarrays operate in the same way, except for some differences in the operation of the PE+ in the lower row of each subarray. For example, the upper subarray outputs the results of the sum-of-products operation in PE+(1) and PE+(3) to the lower subarray, then loops them back and stores them. In contrast, the lower subarray outputs the results of the sum-of-products operation in PE+(7) and PE+(5) to the bus BUS, then loops them back and stores them.

[0082] In Figure 28 (clock cycle t=1), data a(1,0) held in FF2 of PE+(2) is transferred to FF2 of PE+(0) (Figure 28(a)). Data a(0,0) held in FF2 of PE+(0) is transferred to FF1 of PE+(2) (Figure 28(b)). Data c(1,0) held in FF4 of PE+(1) is transferred to FF4 of PE+(0) (Figure 28(c)). Data c(0,0) held in FF4 of PE+(0) is added to the multiplication result of PE+(0), and the addition result is stored in FF2 of PE+(0) as data c(0,0) (Figure 28(d)). The operations of PE+(4), PE+(5), and PE+(6) are similar to those of PE+(0), PE+(1), and PE+(2).

[0083] In Figure 29 (clock cycle t=2), data a(1,0) held in FF2 of PE+(0) is transferred to FF1 of PE+(2) (Figure 29(a)). Data a(0,0) held in FF1 of PE+(2) is transferred to FF2 of PE+(2) (Figure 29(b)). Data a(1,1) held in FF2 of PE+(3) is transferred to FF2 of PE+(1) (Figure 29(c)). Data a(0,1) held in FF2 of PE+(1) is transferred to FF1 of PE+(3) (Figure 29(d)).

[0084] The data c(1,0) held in FF4 of PE+(0) is added to the multiplication result of PE+(0), and the addition result is stored in FF3 of PE+(0) as data c(1,0) (Figure 29(e)). The data c(0,0) held in FF2 of PE+(0) is added to the multiplication result of PE+(1), and the addition result is stored in FF3 and FF4 of PE+(1) as data c(0,0) (Figure 29(f)).

[0085] The data c(1,1) held in FF4 of PE+(3) is transferred to FF4 of PE+(2) (Fig. 29(g)). The data c(0,1) held in FF4 of PE+(2) is added to the multiplication result of PE+(2), and the addition result is stored in FF2 of PE+(2) as data c(0,1) (Fig. 29(h)). The operation of PE+(4) to PE+(7) is similar to the operation of PE+(0) to PE+(3) except that the data c(0,0) that is the result of the multiplication and accumulation operation in PE+(5) is not only stored in FF3 and FF4 but is also output to the bus BUS (Fig. 29(i)).

[0086] In Figure 30 (clock cycle t=3), data a(1,0) held in FF1 of PE+(2) is transferred to FF2 of PE+(2) (Figure 30(a)). Data a(0,0) held in FF2 of PE+(2) is transferred to FF2 of PE+(0) (Figure 30(b)). As a result, FF2 of PE+(0) and PE+(2) return to their initial state, holding data a(0,0) and a(1,0), respectively.

[0087] Data a(1,1) held in FF2 of PE+(1) is transferred to FF1 of PE+(3) (Fig. 30(c)). Data a(0,1) held in FF1 of PE+(3) is transferred to FF2 of PE+(3) (Fig. 30(d)).

[0088] The data c(1,0) held in FF3 of PE+(0) is added to the multiplication result of PE+(1), and the addition result is stored in FF3 and FF4 of PE+(1) as data c(1,0) (Figure 30(e)). The data c(0,0) held in FF4 of PE+(1) is transferred to FF4 of PE+(0) (Figure 30(f)). As a result, FF4 of PE+(0) and PE+(1) return to their initial state, holding data c(0,0) and a(1,0), respectively. However, data c(0,0) and a(1,0) are not the original data but the results of the multiplication and accumulation.

[0089] The data c(0,0) held in FF3 of PE+(1) is stored in FF3 of PE+(4) without being input to the FMA of PE+(4) (FIG. 31(g)). In other words, the matrix multiplication result in the upper subarray is bypassed through the FMA toward the bus BUS without being calculated in the lower subarray. Bypassing the matrix multiplication result can be achieved by providing a bypass path in each PE+ that directly connects the output of MUX4 to the input of MUX5. Note that by stopping the clock supplied to the bypassed FMA, the power consumption of the systolic array 100 can be reduced.

[0090] The data c(1,1) held in FF4 of PE+(2) is added to the multiplication result of PE+(2), and the addition result is stored in FF3 of PE+(2) as data c(1,1) (Figure 30(h)). The data c(0,1) held in FF3 of PE+(2) is added to the multiplication result of PE+(3), and the addition result is stored in FF3 and FF4 of PE+(3) as data c(0,1) (Figure 30(i)).

[0091] The operation of PE+(4) to PE+(7) is the same as that of PE+(0) to PE+(3) except that the data c(1,0) and c(0,1), which are the results of the multiplication and addition operations in PE+(5) and PE+(7), are not only stored in FF3 and FF4, respectively, but are also output to the bus BUS (Figure 30(j) and (k)).

[0092] In Figure 31 (clock cycle t=4), data a(1,1) held in FF1 of PE+(3) is transferred to FF2 of PE+(3) (Figure 31(a)). Data a(0,1) held in FF2 of PE+(3) is transferred to FF2 of PE+(1) (Figure 31(b)). As a result, FF2 of PE+(1) and PE+(2) return to their initial state, holding data a(0,1) and a(1,1), respectively.

[0093] The data c(1,1) held in FF3 of PE+(2) is added to the multiplication result of PE+(2), and the addition result is stored in FF3 and FF4 of PE+(3) as data c(1,1) (Figure 31(c)). The data c(0,1) held in FF4 of PE+(3) is transferred to FF4 of PE+(2) (Figure 31(d)). As a result, FF4 of PE+(2) and PE+(3) return to their initial state, holding data c(0,1) and a(1,1), respectively. However, data c(0,1) and a(1,1) are not the original data but the results of the multiplication and accumulation.

[0094] The data c(1,0) held in FF3 of PE+(1) is stored in FF3 of PE+(4) without being input to the FMA of PE+(4) (Fig. 31(e)). The data c(0,0) held in FF2 of PE+(4) is stored in FF3 of PE+(5) without being input to the FMA of PE+(5) and is output to the bus BUS (Fig. 31(f)).

[0095] The data c(0,1) held in FF3 of PE+(3) is stored in FF3 of PE+(6) without being input to the FMA of PE+(6) (Fig. 31(g)). The transfers in Fig. 31(e), (f), and (g) are bypass operations that transfer the matrix multiplication results in the upper subarray to the bus BUS without performing calculations in the FMA of the lower subarray.

[0096] The operation of PE+(4) to PE+(7) is the same as that of PE+(0) to PE+(3), except that the result of the multiplication and accumulation operation in PE+(7), data c(1,1), is output to the bus BUS (Figure 31(h)), and the matrix multiplication result from the upper subarray bypasses the FMA.

[0097] From Figure 32 (clock cycle t=5) to Figure 33 (clock cycle t=6), matrix multiplication by FMA is not performed, and the multiplication result in the upper sub-array is transferred to the bus BUS. Note that the upper sub-array stops operating because it has returned to its initial state.

[0098] In Figure 32, data c(1,0) held in FF3 of PE+(4) is stored in FF3 of PE+(5) without being input to the FMA of PE+(5) and is output to the bus BUS (Figure 32(a)). Data c(1,1) held in FF3 of PE+(3) is stored in FF3 of PE+(6) without being input to the FMA of PE+(6) (Figure 32(b)). Data c(0,1) held in FF3 of PE+(6) is stored in FF3 of PE+(6) without being input to the FMA of PE+(7) and is output to the bus BUS (Figure 32(c)).

[0099] In Figure 33, data c(1,1) held in FF3 of PE+(6) is stored in FF3 of PE+(6) without being input to the FMA of PE+(7), and is output to the bus BUS (Figure 33(a)).

[0100] FIG. 34 shows the characteristics of the systolic array-type matrix multipliers shown in FIGS. 2, 5, and 7. In the table, a circle indicates a good result, a double circle indicates the best result, a triangle indicates a poor result, and a cross indicates a problem. FIG. 34 shows the evaluation results for a 4×4 matrix multiplication and a p×p matrix multiplication (p is an integer equal to or greater than 4). The evaluation results were determined based on the processing time for a 4×4 matrix multiplication or a p×p matrix multiplication, the processing time for a 2×2 matrix multiplication or a (p / 2)×(p / 2) matrix multiplication, and the circuit area of ​​the systolic arrays 100, 102, and 104.

[0101] The evaluation of 4x4 matrix multiplication was calculated by adding up the processing times of (a) to (c) below. (a) The number of clock cycles (=4) until the first matrix multiplication result c(0,0) is output from each of the systolic arrays 100, 102, and 104 (b) Furthermore, the number of clock cycles (=3) until the final matrix multiplication result c(3,0) is output from each of the systolic arrays 100, 102, and 104 from the right column. (c) Furthermore, the number of clock cycles (=3) until the final matrix multiplication result c(3,3) is output from each of the systolic arrays 100, 102, and 104

[0102] The evaluation of the systolic array 100 for executing four matrix multiplications of 2 rows and 2 columns was carried out by adding up the processing times of the following (d) to (g). (d) The number of clock cycles (=2) until the first matrix multiplication result c(0,0) of the lower subarray is output from the systolic array 100 (e) Furthermore, the number of clock cycles (=1) until the final matrix multiplication result c(1,0) of the lower subarray is output from the systolic array 100 (f) Furthermore, the number of clock cycles (=2) until the final matrix multiplication result c(1,0) of the right column of the upper subarray is output from the systolic array 100 (g) Furthermore, the number of clock cycles (=1) until the final matrix multiplication result c(1,1) of the upper subarray is output from the systolic array 100

[0103] In the evaluation of the systolic array 102 that executes four matrix multiplications of 2 rows and 2 columns, the processing time was calculated assuming that matrix multiplications of 4 rows and 4 columns were repeated four times.

[0104] In the evaluation of the systolic array 104 that executes four matrix multiplications with two rows and two columns, the processing times of the following (h) to (j) were totaled, where the subarray includes four PE+s with two rows and two columns. (h) Number of clock cycles (=2) until the first matrix multiplication result c(0,0) of the four subarrays is output from the systolic array 104 (i) Furthermore, the number of clock cycles (=1) until the final matrix multiplication result c(1,0) in the right column of the four subarrays is output from the systolic array 104 (j) Furthermore, the number of clock cycles (=1) until the final matrix multiplication result c(1,1) of the left column of the four subarrays is output from the systolic array 104

[0105] There was no significant difference in the processing time for 4x4 matrix multiplication among the systolic arrays 100, 102, and 104. For the processing time for 2x2 matrix multiplication, the systolic array 104 was the best, the systolic array 100 was good, and the systolic array 102 had problems.

[0106] In terms of circuit area, the systolic arrays 100 and 102 were good, while the systolic array 104 had problems. As a result, the evaluations of the systolic arrays 100, 102, and 104 were highest in this order. In the evaluation of matrix multiplication with p rows and p columns, as in the evaluation of matrix multiplication with 4 rows and 4 columns, the evaluations of the systolic arrays 100, 102, and 104 were highest in this order.

[0107] As described above, in this embodiment, data of matrix A transferred from left to right can be folded back to the left in any PE+ and stored, and data of matrix A transferred from right to left can be folded back to the right in any PE+ and stored. Furthermore, systolic array 100 can fold back data of matrix C transferred from top to bottom in any PE+ and store it, and data of matrix C transferred from bottom to top in any PE+ and store it.

[0108] This allows the data a and c to be supplied to each PE+ without using the bus BUS while the systolic array 100 is performing the matrix multiplication. Since no data is transferred from the memory 220 to the systolic array 100 while the systolic array 100 is performing the matrix multiplication, it is possible to reduce the bandwidth of the bus BUS between the systolic array 100 and the memory 220. As a result, it is possible to suppress a decrease in operation performance due to bandwidth limitations of the bus BUS.

[0109] When the systolic array 100 completes the matrix multiplication, the data of matrix A and the data of matrix C are returned to their initial positions. Therefore, the next matrix multiplication can be started without inputting the data a, b, and c to each PE+ from the bus BUS, thereby preventing an increase in the bandwidth of the bus BUS.

[0110] 20 to 26, when the data of matrix A and matrix C to be used in the next matrix multiplication is changed from the data of matrix A and matrix C used in the currently executing matrix multiplication, the data can be transferred to each PE+ during the execution of the matrix multiplication. This allows the next matrix multiplication to be started after the execution of the current matrix multiplication is completed without transferring data from memory 220 via bus BUS. Since the operation of systolic array 100 can be changed depending on whether the data of matrix A and matrix C used in the previous matrix multiplication will be used in the next matrix multiplication, the usability of systolic array 100 can be improved.

[0111] 27 to 33, systolic array 100 can be divided into a plurality of subarrays, and matrix multiplication can be performed in parallel in the plurality of subarrays. In this case, matrix multiplication can be performed in parallel without providing a memory MEM for each subarray as shown in Fig. 7. Therefore, matrix multiplication for systolic arrays of a plurality of sizes can be performed by a single systolic array 100 without increasing the circuit scale of systolic array 100 and information processing device 200 in which systolic array 100 is installed.

[0112] 30 to 33, the matrix multiplication result of the upper subarray can be output to the bus BUS, bypassing the FMA of the lower subarray, so that the correct matrix multiplication result can be output. The operation of bypassing the FMA of the lower subarray can be achieved by providing MUX5 in PE+ in FIG. 2 and providing a path that directly connects MUX4 and MUX5. For example, by stopping the clock supplied to the bypassed FMA, the power consumption of systolic array 100 can be reduced.

[0113] 1 illustrates systolic array 100 including four PE+s in a row and four PE+s for ease of explanation, but systolic array 100 may be larger, including 128 PE+s in a row and 128 PE+s in a column, or 256 PE+s in a row and 256 PE+s in a column and 256 PE+s in a column. In this case, multiple matrix multiplications of any of 2x2, 4x4, 8x8, ..., 32x32, 64x64, etc. can be executed in parallel without adding memory.

[0114] The following additional notes are provided regarding the above-described embodiments. (Appendix 1) A systolic array type matrix multiplier including a plurality of processing elements arranged in a matrix and performing matrix multiplication, Each of the plurality of processing elements a first holding unit that sequentially holds each element of a first matrix received from a first input terminal provided on one end side in a first direction; a first path that outputs the output of the first holding unit to a first output terminal that is provided on the other end side in the first direction; a second holding unit that sequentially holds each element of the first matrix received from a second input terminal provided on the other end side in the first direction; a second path that outputs the output of the second holding unit to a second output terminal that is provided on one end side in the first direction; a multiply-accumulate unit connected to the first path; a first selection unit that connects the first path or the second input terminal to the second path; a second selection unit that connects the output of the second path or the first holding unit to the first path; Systolic array matrix multiplier. (Appendix 2) Among the plurality of processing elements, a processing element arranged at one end in the first direction connects the second path to the first path by the second selection unit; a processing element disposed at the other end in the first direction among the plurality of processing elements connects the first path to the second path by the first selection unit; Of the plurality of processing elements, a processing element disposed at a location other than one end in the first direction and the other end in the first direction connects the output of the first holding unit to the first path by the second selection unit and connects the second input terminal to the input of the second holding unit by the first selection unit. 10. The systolic array matrix multiplier of claim 1. (Appendix 3) When the plurality of processing elements are partitioned into a plurality of subarrays each including i×j processing elements in i rows and j columns (i and j are integers of 2 or greater), and matrix multiplication is performed in each of the plurality of subarrays, In each of the plurality of subarrays, Among the plurality of processing elements, a processing element arranged at one end in the first direction connects the second path to the first path by the second selection unit; a processing element disposed at the other end in the first direction among the plurality of processing elements connects the first path to the second path by the first selection unit; Of the plurality of processing elements, a processing element disposed at a location other than one end in the first direction and the other end in the first direction connects the output of the first holding unit to the first path by the second selection unit and connects the second input terminal to the input of the second holding unit by the first selection unit. 10. The systolic array matrix multiplier of claim 1. (Appendix 4) Each of the plurality of processing elements a third path that receives elements of a second matrix from a third input terminal provided at one end side in a second direction intersecting with the first direction and outputs the elements to the multiply-accumulate calculator; a third holding unit that sequentially holds the calculation results output from the multiply-add calculator and outputs the results to a third output terminal that is provided on the other end side in the second direction; a fourth holding unit that sequentially holds each element of the second matrix received from a fourth input terminal provided on the other end side in the second direction; a fourth path that outputs the output of the fourth holding unit to a fourth output terminal that is provided on one end side in the second direction; a third selection unit that connects the third path or the fourth input terminal to the fourth path; a fourth selection unit that connects the fourth path or the third input terminal to the third path. 10. The systolic array matrix multiplier of claim 1. (Appendix 5) a processing element disposed at one end in the second direction among the plurality of processing elements connects the fourth path to the third path by the fourth selection unit; a processing element disposed at the other end in the second direction among the plurality of processing elements connects the third path to the fourth path by the third selection unit; Of the plurality of processing elements, processing elements arranged at locations other than one end in the second direction and the other end in the second direction connect the third input terminal to the third path by the fourth selection unit and connect the fourth input terminal to the fourth path by the third selection unit. 5. The systolic array matrix multiplier according to claim 4. (Appendix 6) When the plurality of processing elements are partitioned into a plurality of subarrays each including i×j processing elements in i rows and j columns (i and j are integers of 2 or greater), and matrix multiplication is performed in each of the plurality of subarrays, In each of the plurality of subarrays, a processing element disposed at one end in the second direction among the plurality of processing elements connects the fourth path to the third path by the fourth selection unit; a processing element disposed at the other end in the second direction among the plurality of processing elements connects the third path to the fourth path by the third selection unit; Of the plurality of processing elements, processing elements arranged at locations other than one end in the second direction and the other end in the second direction connect the third input terminal to the third path by the fourth selection unit and connect the fourth input terminal to the fourth path by the third selection unit. 5. The systolic array matrix multiplier according to claim 4. (Appendix 7) each of the plurality of processing elements includes a fifth selection unit that connects an output of the multiply-accumulate unit or the third path to the third holding unit; the third holding unit holds an operation result output from the multiply-accumulate unit or an operation result output from the multiply-accumulate unit of the processing element arranged on one end side in the second direction, Among the plurality of subarrays, the processing element included in a subarray that has another subarray disposed at one end in the second direction is When the matrix multiplication result is transferred from the other sub-array, the third input terminal is connected to the third holding unit by the fourth selection unit and the fifth selection unit. 7. The systolic array matrix multiplier according to claim 6. (Appendix 8) the plurality of processing elements are arranged in a matrix, each of the plurality of processing elements comprising: a multiply-accumulate circuit; a first holding unit that sequentially holds each element of a first matrix received from a first input terminal provided at one end in a first direction; a first path that outputs an output of the first holding unit to a first output terminal provided at the other end in the first direction; a second holding unit that sequentially holds each element of the first matrix received from a second input terminal provided at the other end in the first direction; a second path that outputs an output of the second holding unit to a second output terminal provided at one end in the first direction; and a multiply-accumulate operator connected to the first path; a first selection unit that connects the first path or the first output terminal to the second path, and a second selection unit that connects the second path or an output of the first holding unit to the first path, said method comprising: Among the plurality of processing elements, a processing element arranged at one end in the first direction connects the second path to the first path by the second selection unit; a processing element disposed at the other end in the first direction among the plurality of processing elements connects the first path to the second path by the first selection unit; Of the plurality of processing elements, a processing element disposed at a location other than one end in the first direction and the other end in the first direction connects the output of the first holding unit to the first path by the second selection unit and connects the second input terminal to the input of the second holding unit by the first selection unit. How a systolic array matrix multiplier works. (Appendix 9) When the plurality of processing elements are partitioned into a plurality of subarrays each including i×j processing elements in i rows and j columns (i and j are integers of 2 or greater), and matrix multiplication is performed in each of the plurality of subarrays, In each of the plurality of subarrays, Among the plurality of processing elements, a processing element arranged at one end in the first direction connects the second path to the first path by the second selection unit; a processing element disposed at the other end in the first direction among the plurality of processing elements connects the first path to the second path by the first selection unit; Of the plurality of processing elements, a processing element disposed at a location other than one end in the first direction and the other end in the first direction connects the output of the first holding unit to the first path by the second selection unit and connects the second input terminal to the input of the second holding unit by the first selection unit. 9. A method of operating the systolic array matrix multiplier according to claim 8. (Appendix 10) Each of the plurality of processing elements has: a third path that outputs each element of a second matrix received from a third input terminal provided at one end side in a second direction intersecting with the first direction to the multiply-accumulate calculator; a third holding unit that sequentially holds the operation results output from the multiply-accumulate calculator and outputs the results to a third output terminal provided at the other end side in the second direction; a fourth holding unit that sequentially holds each element of the second matrix received from a fourth input terminal provided at the other end side in the second direction; a fourth path that outputs an output of the fourth holding unit to a fourth output terminal provided at one end side in the second direction; a third selection unit that connects the third path or the fourth input terminal to the fourth path; and a fourth selection unit that connects the fourth path or the third input terminal to the third path. a processing element disposed at one end in the second direction among the plurality of processing elements connects the fourth path to the third path by the fourth selection unit; a processing element disposed at the other end in the second direction among the plurality of processing elements connects the third path to the fourth path by the third selection unit; Of the plurality of processing elements, processing elements arranged at locations other than one end in the second direction and the other end in the second direction connect the third input terminal to the third path by the fourth selection unit and connect the fourth input terminal to the fourth path by the third selection unit. 9. A method of operating the systolic array matrix multiplier according to claim 8. (Appendix 11) When the plurality of processing elements are partitioned into a plurality of subarrays each including i×j processing elements in i rows and j columns (i and j are integers of 2 or greater), and matrix multiplication is performed in each of the plurality of subarrays, In each of the plurality of subarrays, a processing element disposed at one end in the second direction among the plurality of processing elements connects the fourth path to the third path by the fourth selection unit; a processing element disposed at the other end in the second direction among the plurality of processing elements connects the third path to the fourth path by the third selection unit; Of the plurality of processing elements, processing elements arranged at locations other than one end in the second direction and the other end in the second direction connect the third input terminal to the third path by the fourth selection unit and connect the fourth input terminal to the fourth path by the third selection unit. 11. A method of operating the systolic array matrix multiplier of claim 10. (Appendix 12) each of the plurality of processing elements includes a fifth selection unit that connects an output of the multiply-accumulate unit or the third path to the third holding unit; the third holding unit holds an operation result output from the multiply-accumulate unit or an operation result output from the multiply-accumulate unit of the processing element arranged on one end side in the second direction, Among the plurality of subarrays, the processing element included in a subarray that has another subarray disposed at one end in the second direction is When the matrix multiplication result is transferred from the other sub-array, the third input terminal is connected to the third holding unit by the fourth selection unit and the fifth selection unit. 12. A method of operating the systolic array matrix multiplier of claim 11.

[0115] The features and advantages of the embodiments will be apparent from the above detailed description. It is intended that the claims encompass the features and advantages of the above-described embodiments without departing from the spirit and scope of the claims. Furthermore, any improvements and modifications will be readily apparent to those skilled in the art. Therefore, it is not intended that the scope of the inventive embodiments be limited to the above-described embodiments, and appropriate improvements and equivalents within the scope of the disclosed embodiments may be utilized. [Explanation of symbols]

[0116] 100, 102, 104 Systolic array matrix multiplier 200 Information processing device 210 CPU 220 memory 230 Auxiliary storage 240 Communication Interface 250 Input / Output Interface 300 Recording Media A1R, A2L, AR input terminals BU, C1D, C2U, CU input terminals A1L, A2R, AL output terminals BD, C1U, C2D, CD output terminals BUS FF1, FF2, FF3, FF4, FF5 flip-flops FF6, FF7, FF8 flip-flops FMA Multiply-accumulate unit MEM, MEM1, MEM2, MEM3 memory MUX1, MUX2, MUX3 multiplexers MUX4, MUX5, MUX6 multiplexers Processing Element PE, PE+

Claims

1. A systolic array type matrix multiplier including a plurality of processing elements arranged in a matrix and performing matrix multiplication, Each of the plurality of processing elements a first holding unit that sequentially holds each element of a first matrix received from a first input terminal provided on one end side in a first direction; a first path that outputs the output of the first holding unit to a first output terminal that is provided on the other end side in the first direction; a second holding unit that sequentially holds each element of the first matrix received from a second input terminal provided on the other end side in the first direction; a second path that outputs the output of the second holding unit to a second output terminal that is provided on one end side in the first direction; a multiply-accumulate calculator connected to the first path; a first selection unit that connects the first path or the second input terminal to the second path; a second selection unit that connects the output of the second path or the output of the first holding unit to the first path; Systolic array matrix multiplier.

2. a processing element disposed at one end in the first direction among the plurality of processing elements connects the second path to the first path by the second selection unit; a processing element disposed at the other end in the first direction among the plurality of processing elements connects the first path to the second path by the first selection unit; Of the plurality of processing elements, a processing element disposed at a location other than one end in the first direction and the other end in the first direction connects the output of the first holding unit to the first path by the second selection unit and connects the second input terminal to the input of the second holding unit by the first selection unit.

2. The systolic array type matrix multiplier according to claim 1.

3. When the plurality of processing elements are partitioned into a plurality of subarrays each including i × j processing elements in i rows and j columns (i and j are integers of 2 or greater), and matrix multiplication is performed in each of the plurality of subarrays, In each of the plurality of subarrays, a processing element disposed at one end in the first direction among the plurality of processing elements connects the second path to the first path by the second selection unit; a processing element disposed at the other end in the first direction among the plurality of processing elements connects the first path to the second path by the first selection unit; Of the plurality of processing elements, a processing element disposed at a location other than one end in the first direction and the other end in the first direction connects the output of the first holding unit to the first path by the second selection unit and connects the second input terminal to the input of the second holding unit by the first selection unit.

2. The systolic array type matrix multiplier according to claim 1.

4. Each of the plurality of processing elements a third path that receives elements of a second matrix from a third input terminal provided at one end side in a second direction intersecting with the first direction and outputs the elements to the multiply-accumulate calculator; a third holding unit that sequentially holds the calculation results output from the multiply-add calculator and outputs the results to a third output terminal provided on the other end side in the second direction; a fourth holding unit that sequentially holds each element of the second matrix received from a fourth input terminal provided on the other end side in the second direction; a fourth path that outputs the output of the fourth holding unit to a fourth output terminal that is provided on one end side in the second direction; a third selection unit that connects the third path or the fourth input terminal to the fourth path; a fourth selection unit that connects the fourth path or the third input terminal to the third path; 2. The systolic array type matrix multiplier according to claim 1.

5. a processing element disposed at one end in the second direction among the plurality of processing elements connects the fourth path to the third path by the fourth selection unit; a processing element disposed at the other end in the second direction among the plurality of processing elements connects the third path to the fourth path by the third selection unit; Of the plurality of processing elements, processing elements arranged at locations other than one end in the second direction and the other end in the second direction connect the third input terminal to the third path by the fourth selection unit and connect the fourth input terminal to the fourth path by the third selection unit.

5. The systolic array type matrix multiplier according to claim 4.

6. When the plurality of processing elements are partitioned into a plurality of subarrays each including i × j processing elements in i rows and j columns (i and j are integers of 2 or greater), and matrix multiplication is performed in each of the plurality of subarrays, In each of the plurality of subarrays, a processing element disposed at one end in the second direction among the plurality of processing elements connects the fourth path to the third path by the fourth selection unit; a processing element disposed at the other end in the second direction among the plurality of processing elements connects the third path to the fourth path by the third selection unit; Of the plurality of processing elements, processing elements arranged at locations other than one end in the second direction and the other end in the second direction connect the third input terminal to the third path by the fourth selection unit and connect the fourth input terminal to the fourth path by the third selection unit.

5. The systolic array type matrix multiplier according to claim 4.

7. each of the plurality of processing elements includes a fifth selection unit that connects an output of the multiply-accumulate unit or the third path to the third holding unit; the third holding unit holds an operation result output from the multiply-accumulate unit or an operation result output from the multiply-accumulate unit of the processing element arranged on one end side in the second direction, The processing element included in a subarray among the plurality of subarrays, which has another subarray disposed at one end in the second direction, When the matrix multiplication result is transferred from the other sub-array, the third input terminal is connected to the third holding unit by the fourth selection unit and the fifth selection unit.

7. The systolic array type matrix multiplier according to claim 6.

8. a first holding unit configured to sequentially hold each element of a first matrix received from a first input terminal provided at one end in a first direction, a first path configured to output an output of the first holding unit to a first output terminal provided at the other end in the first direction, a second holding unit configured to sequentially hold each element of the first matrix received from a second input terminal provided at the other end in the first direction, a second path configured to output an output of the second holding unit to a second output terminal provided at one end in the first direction, a multiply-accumulate unit connected to the first path, a first selection unit configured to connect the first path or the first output terminal to the second path, and a second selection unit configured to connect the second path or the output of the first holding unit to the first path, a processing element disposed at one end in the first direction among the plurality of processing elements connects the second path to the first path by the second selection unit; a processing element disposed at the other end in the first direction among the plurality of processing elements connects the first path to the second path by the first selection unit; Of the plurality of processing elements, a processing element disposed at a location other than one end in the first direction and the other end in the first direction connects the output of the first holding unit to the first path by the second selection unit and connects the second input terminal to the input of the second holding unit by the first selection unit. How a systolic array matrix multiplier works.

Images