Matrix product operator

Abstract

To calculate a complex matrix product at a low cost.SOLUTION: In calculation of a matrix product using a first complex matrix, a second complex matrix, and a third complex matrix, separating a complex number included in the second complex matrix into a real part and an imaginary part, and loading the real part and the imaginary part of the complex number into the plurality of arithmetic unit elements 8; A first multiplication of a first complex-number matrix and a real part of a second complex-number matrix and addition of a third complex-number matrix to a result of the first multiplication are performed, an imaginary part and the real part are replaced with each other in each complex number of the first complex-number matrix, a second multiplication of the first complex-number matrix whose imaginary part and real part are replaced with each other and the imaginary part of the second complex-number matrix and addition of the third complex-number matrix to a result of the second multiplication are performed.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a matrix multiplication unit. [Background technology]

[0002] Matrix multiplication is widely used in HPC (High Performance Computing), and libraries that compile a series of matrix multiplication calculations, such as BLAS (Basic Linear Algebra Subprograms), are publicly available. In addition, some applications require matrix multiplication of complex matrices.

[0003] For example, matrix multiplication is frequently used in DNNs (Deep Neural Networks). For this reason, systems equipped with a matrix multiplication calculator (accelerator) that accelerates matrix multiplication calculations are known as hardware configurations (see, for example, Patent Document 1). In recent years, layers using complex numbers have been used in DNNs, and complex matrix multiplication is also required in AI (Artificial Intelligence).

[0004] Since a complex matrix product can be calculated by replacing it with a real matrix product, by replacing the complex matrix product with a real matrix product, the complex matrix product can also be calculated by a real matrix product calculator. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication No. 2020-509501 [Patent Document 2] US Patent Application Publication No. 2011 / 0040822 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technique of replacing complex matrix multiplication with real matrix multiplication requires a large number of rearrangements and additional memory, which increases costs.

[0007] In one aspect, the present invention aims to enable complex matrix multiplication to be calculated at low cost. [Means for solving the problem]

[0008] For this reason, this matrix multiplication calculator has a plurality of calculator elements, and in calculating a matrix product using a first complex matrix, a second complex matrix, and a third complex matrix, separates the complex numbers included in the second complex matrix into real parts and imaginary parts, loads the real parts and imaginary parts of the complex numbers into the plurality of calculator elements, performs a first multiplication of the first complex matrix and the real parts of the second complex matrix, and adds the third complex matrix to the result of the first multiplication, swaps the imaginary parts and real parts for each complex number in the first complex matrix, performs a second multiplication of the first complex matrix with the swapped imaginary parts and the imaginary parts of the second complex matrix, and adds the third complex matrix to the result of the second multiplication. [Effects of the Invention]

[0009] According to one embodiment, complex matrix products can be computed at low cost. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is a diagram illustrating a configuration of an accelerator according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a configuration of an accelerator according to an embodiment. [Figure 3] FIG. 10 is a diagram for explaining the size of a matrix. [Figure 4] FIG. 10 is a diagram illustrating an example of an i-swap configuration in an accelerator according to an embodiment. [Figure 5]FIG. 10 is a diagram illustrating an operation of i-swap in an accelerator according to an embodiment. [Figure 6] FIG. 10 illustrates pseudocode representing the operation of i-swap in an accelerator according to an embodiment. [Figure 7] 10 is a flowchart illustrating a process of complex matrix multiplication in an accelerator according to an embodiment. [Figure 8] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 9] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 10] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 11] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 12] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 13] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 14] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 15] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 16] FIG. 2 is a diagram illustrating complex matrices A, B, and C processed in an accelerator according to an embodiment. [Figure 17] FIG. 10 is a diagram showing a matrix product where the read complex matrix size is (128, 128, 128). [Figure 18] FIG. 10 is a diagram for explaining a method for calculating a matrix product while moving submatrices. [Figure 19] FIG. 10 is a diagram for explaining a method for calculating a matrix product while moving submatrices. [Figure 20] FIG. 2 is a diagram illustrating processing of matrices A and B in an accelerator according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the matrix multiplication calculator will be described with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modifications and techniques not explicitly stated in the embodiments. In other words, the present embodiment can be implemented with various modifications within the scope of its purpose. Furthermore, each figure does not intend to include only the components shown in the figure, but may also include other functions, etc.

[0012] (A) Configuration 1 and 2 are diagrams illustrating the configuration of an accelerator 1 according to an embodiment.

[0013] The accelerator 1 is a hardware accelerator having a function of performing matrix multiplication calculations, and is installed in, for example, an HPC.

[0014] In Fig. 1, symbol A indicates a schematic configuration example of an accelerator 1. Symbol B indicates a schematic configuration example of an ACC block 2 provided in the accelerator 1. Symbol C indicates a schematic configuration example of an ACC core 4 provided in the ACC block 2. Fig. 2 also shows a schematic configuration example of an LMP (Large Matrix-multiplication Processor) 7 provided in the ACC core 4.

[0015] As indicated by the symbol A in FIG. 1, the accelerator 1 has multiple memories 3 and multiple ACC blocks 2. The memories 3 may be provided corresponding to the ACC blocks 2. Furthermore, as indicated by the symbol B, the ACC blocks 2 have multiple ACC cores 4. As indicated by the symbol C, each ACC core 4 has a core 5 with an arithmetic unit, a scratchpad memory 6, and an LMP 7.

[0016] The accelerator 1 is an example of a matrix multiplication calculator having a scratchpad memory 6 (memory) and a plurality of PEs (Processing Elements) 8 (calculator elements).

[0017] The core with arithmetic unit 5 controls, for example, the execution of matrix multiplication calculations by the LMP7 of the ACC core 4 in which it is mounted. The core with arithmetic unit 5 causes the LMP7 of the ACC core 4 in which it is mounted to perform the matrix multiplication calculations.

[0018] In the example shown below, the matrix product αA×B+βC is calculated, where α and β are scalars, and A, B, and C are matrices.

[0019] The accelerator 1 can perform a complex matrix multiplication operation, and the matrix multiplication may be a complex matrix multiplication. That is, matrix A, matrix B, and matrix C may each be a complex matrix. Hereinafter, matrix A, which is a complex matrix, may be referred to as complex matrix A. Similarly, matrix B, which is a complex matrix, may be referred to as complex matrix B, and matrix C, which is a complex matrix, may be referred to as complex matrix C.

[0020] Complex matrix A is an example of a first complex matrix, complex matrix B is an example of a second complex matrix, and complex matrix C is an example of a third complex matrix.

[0021] The core 5 with arithmetic unit can read and write data from the memory 3 and the scratchpad memory 6 of the ACC core 4 in which it is mounted. The core 5 with arithmetic unit can also know when the LMP 7 of the ACC core 4 in which it is mounted has completed its calculation.

[0022] In calculating the complex matrix product αA×B+βC, the arithmetic unit core 5 reads the complex matrix B from the memory 3, separates the real and imaginary parts of each complex number contained in the complex matrix B, and writes them to the scratchpad memory 6. The arithmetic unit core 5 then loads the data of the real and imaginary parts of the complex matrix B written to the scratchpad memory 6 into the registers of the PE8. The registers of the PE8 can store two pieces of data, and the arithmetic unit core 5 stores the real and imaginary parts of the complex number in the registers of each PE8, respectively. As a result, the real and imaginary parts of one element (complex number) constituting the complex matrix B are stored in one PE8.

[0023] Furthermore, when performing a complex matrix multiplication, the processor-equipped core 5 causes the multiple PEs 8 to perform a calculation of A×B real part+C using the real part of complex matrix B, complex matrix C, and complex matrix A. In this calculation, the A×B real part is an example of a first multiplication.

[0024] Furthermore, the arithmetic unit core 5 swaps the real and imaginary parts of the elements of complex matrix A using i-swap 10, and inverts the sign of the input imaginary part before outputting it.The arithmetic unit core 5 then causes the multiple PEs 8 to perform a calculation of A×B imaginary part+C using the imaginary part of complex matrix B, complex matrix C, and complex matrix A. In this calculation, the A×B imaginary part is an example of a second multiplication.

[0025] The scratch pad memory 6 is a memory that allows high-speed data access, and may be placed, for example, near a processor (not shown) of the HPC.

[0026] The scratchpad memory 6 stores (expands) data of matrices A, B, and C to be input to the LMP 7, which will be described later. Also, as will be described later, the complex matrix B is rearranged in this scratchpad memory 6.

[0027] LMP7 has a two-dimensional systolic array configuration and performs matrix multiplication (αAB+βC).

[0028] FIG. 3 is a diagram for explaining the size of a matrix.

[0029] 3 shows the matrix product αA×B+βC using matrices A, B, and C. For example, if the number of rows in matrix A is m, the number of columns in matrix B is n, and the number of rows in matrix B is k, then the size of this matrix product can be expressed as (m, n, k).

[0030] LMP7 is configured as a matrix multiplication unit that accelerates relatively large matrix multiplication calculations, such as (m,n,k)=(64,64,64). For reference, the size (matrix product size) of conventional matrix multiplication units used in DNNs is at most (16,16,16) in the case of FP16 (16-bit floating-point numbers), and calculations are performed in relatively small matrix multiplication units. Therefore, LMP7, which has a relatively large matrix multiplication size such as (m,n,k)=(64,64,64), can be said to be a large matrix multiplication unit that accelerates large matrix multiplications. LMP7 can also be called a matrix multiplication engine.

[0031] The LMP7 performs matrix multiplication using data stored in the scratchpad memory 6 of the ACC core 4 in which it is installed. The LMP7 receives instructions from the core 5 with arithmetic unit and executes the matrix multiplication.

[0032] As shown in FIG. 2, the LMP 7 includes timing adjustment blocks 9a and 9b, a plurality of i-swaps 10, and a plurality of PEs 8.

[0033] PE8 is a computing element that performs the multiplication and addition (a*b+c) calculation. In the LMP7, multiple PE8s are arranged in a two-dimensional lattice, lining up in both row and column directions. In Figure 2, the left-right arrangement of multiple PE6s on the paper corresponds to a row, and the top-bottom arrangement on the paper corresponds to a column. In the LMP7, the multiple PE8s arranged in a two-dimensional lattice can be called a PE group. In this accelerator 1, the PE group is specialized for real matrix multiplication, and is configured to have as many PE8s as possible to achieve maximum performance.

[0034] Each PE8 also has a register capable of storing two elements (data). The data of matrix B is stored in this register. More specifically, for the multiple complex numbers that make up complex number matrix B, the real part and imaginary part obtained by separating one complex number are stored in the register of one PE8.

[0035] For example, matrix A is divided into a plurality of submatrices and input from the left end of the two-dimensional lattice, and matrix C is divided into a plurality of submatrices and input from the top end of the two-dimensional lattice to a plurality of PEs 8 (PE group) arranged in a two-dimensional lattice.

[0036] The timing adjustment blocks 9a and 9b are hardware that adjusts the timing of inputting matrix elements to the LMP7.

[0037] The timing adjustment block 9a adjusts the input timing of the matrix C to the plurality of PEs 8 from the scratchpad memory 6. The timing adjustment block 9a adjusts the timing so that the elements of the matrix C are input at the same timing to each of the PEs 8 constituting the first row (the top row on the paper in the example shown in FIG. 2) of the PE group.

[0038] The timing adjustment block 9b adjusts the input timing of the matrix A to the group of PEs. The timing adjustment block 9b adjusts the timing so that the elements of the matrix A are input at the same timing to each of the PEs 8 constituting the first column (the leftmost column on the paper in the example shown in FIG. 2) of the group of PEs.

[0039] The i-swap 10 realizes the swapping of the imaginary and real parts of the complex numbers in the complex matrix A, and may be, for example, a two-element input / output hardware. The i-swap 10 is disposed between the scratchpad memory 6 and the timing adjustment block 9b. The i-swap 10 may be controlled by the core 5 with arithmetic unit.

[0040] The imaginary and real parts of the complex numbers of the complex matrix A and a flag (control signal) are input to i-swap10. For example, the real part is input to input 0 of i-swap10, and the imaginary part is input to input 1. When the real and imaginary parts are swapped (for example, flag = 1), i-swap10 outputs the imaginary part from output 0 and the real part from output 1. On the other hand, when the real and imaginary parts are not swapped (for example, flag = 0), i-swap10 outputs the real part from output 0 and the imaginary part from output 1.

[0041] The processing by i-swap10 can be said to be equivalent to swapping the imaginary and real parts of the complex numbers in complex matrix A by multiplying the imaginary and real parts of the complex numbers by i, the imaginary unit.

[0042] i-swap10 is an example of a swapping unit that swaps the imaginary and real parts of each complex number in complex matrix A (first complex matrix) by multiplying the real and imaginary parts of complex matrix A by the imaginary unit i, respectively.

[0043] FIG. 4 is a diagram illustrating the configuration of the i-swap 10 in the accelerator 1 according to an embodiment.

[0044] The i-swap 10 illustrated in FIG. 4 includes two selectors 11 and 12 and a sign inversion block 13.

[0045] Each of the two selectors 11 and 12 has three input terminals and one output terminal. In accordance with a control signal (flag) input to one of the three input terminals, each of the selectors 11 and 12 outputs one of the signals input to the remaining two input terminals from the output terminal.

[0046] In selectors 11 and 12, which are schematically shown using a trapezoid in Fig. 4, of the two input terminals arranged on the bottom surface of the trapezoid, the input terminal on the upper side of the paper in Fig. 4 may be referred to as the first input terminal, and the input terminal on the lower side of the paper may be referred to as the second input terminal. Also, the input terminal arranged on the slope of the trapezoid may be referred to as the third input terminal.

[0047] Input 0, input 1, and input 2 are input to the two selectors 11 and 12, respectively. The real parts of the complex numbers in complex matrix A are input to input 0. The imaginary parts of the complex numbers in complex matrix A are input to input 1. A flag (control signal) indicating whether or not to swap the real and imaginary parts is input to input 2.

[0048] In selector 11, input 0 is connected to the first input terminal and input 1 is connected to the second input terminal. On the other hand, in selector 12, input 0 is connected to the second input terminal and input 1 is connected to the first input terminal. In both selectors 11 and 12, input 2 is connected to the third input terminal.

[0049] Furthermore, in both selectors 11 and 12, when 0 is input as flag (input 2), the input of the first input terminal (input terminal at the top of the paper) is selected as the output, and when 1 is input as flag (input 2), the input of the second input terminal (input terminal at the bottom of the paper) is selected as the output.

[0050] The output of the selector 11 is input to the sign inversion block 13. When the real part and the imaginary part are swapped by a flag (for example, flag = 1), the sign inversion block 13 outputs the input value with the sign inverted. In this way, inverting the sign of the input value is equivalent to multiplying the input value of the sign inversion block 13 by -1. This makes it possible to eliminate the influence of the - sign caused by squaring i on the PE group.

[0051] On the other hand, when the real part and the imaginary part are not swapped by the flag (for example, flag = 0), the sign inversion block 13 outputs the input value as is. In this way, outputting the input value as is is equivalent to multiplying the input value of the sign inversion block 13 by +1.

[0052] The output of the sign inversion block 13 is output 0. Also, the output of the selector 12 is output 1. When viewed from the PE group, output 0 appears to output the real part of a complex number, and output 1 appears to output the imaginary part of a complex number.

[0053] 5 is a diagram for explaining the operation of i-swap 10 in accelerator 1 according to one embodiment. In FIG. 5, symbol A indicates the operation of i-swap 10 when the real part and the imaginary part are swapped (flag=1), and symbol B indicates the operation of i-swap 10 when the real part and the imaginary part are not swapped (flag=0). Also, in FIG. 5, the flow of values ​​(real parts) input from input 0 is indicated by dashed lines, and the flow of values ​​(imaginary parts) input from input 1 is indicated by dashed lines.

[0054] As indicated by symbol A, when the real part and the imaginary part are interchanged (flag=1), flag=1 is input to each of selectors 11 and 12 and sign inversion block 13.

[0055] Selector 11 selects the input to the second input terminal (the imaginary part of input 1), and outputs the imaginary part of input 1. The imaginary part of input 1 output from selector 11 is input to sign inversion block 13. Sign inversion block 13 inverts the sign of the value of the imaginary part of input 1 (multiplies the imaginary part of input 1 by -1). Sign inversion block 13 outputs the value (imaginary part) obtained by inverting the sign of the value of the imaginary part of input 1 (output 0).

[0056] Furthermore, the selector 12 selects the input to the second input terminal (the real part of input 0), and outputs the real part of this input 0 (output 1).

[0057] Furthermore, as indicated by symbol B, when the real part and the imaginary part are not interchanged (flag=0), flag=0 is input to each of the selectors 11 and 12 and the sign inversion block 13.

[0058] Selector 11 selects the input (real part of input 0) at its first input terminal and outputs the real part of input 0. The real part of input 0 output from selector 11 is input to sign inversion block 13. Sign inversion block 13 outputs the value of the real part of input 0 as is (output 0).

[0059] In addition, the selector 12 selects the input to the first input terminal (the imaginary part of input 1), and outputs the imaginary part of this input 1 (output 1).

[0060] FIG. 6 is a diagram illustrating pseudocode representing the operation of i-swap10 in the accelerator 1 according to an embodiment.

[0061] In the pseudocode illustrated in FIG. 6, for example, it is specified that when the selector 11 does not swap the imaginary and real parts, it sets the value of the real part of input 0 (input_val.real) to TMP.real, and when it swaps the imaginary and real parts, it sets the value of the imaginary part of input 1 (input_val.imag) to TMP.real (see symbol P1).

[0062] It is also specified that if the selector 12 does not swap the imaginary and real parts, it sets the value of the imaginary part of input 1 (input_val.imag) to TMP.imag, and if it swaps the imaginary and real parts, it sets the value of the real part of input 0 (input_val.real) to TMP.imag (see symbol P2).

[0063] Furthermore, it is specified that when the sign inversion block 13 swaps the imaginary part and the real part, it multiplies the value of TMP.real by −1 (see symbol P3).

[0064] Returning to the explanation using Fig. 2, in Fig. 2, arrows connecting multiple PEs 8 indicate the flow of data, and for example, data is passed to the next stage PE 8 every clock.

[0065] For example, in each row, the data of matrix A coming from the first PE 8 is sequentially sent to each of the subsequent cascaded PEs 8. Similarly, in each column, the data of matrix C coming from the first PE 8 is sequentially sent to each of the subsequent cascaded PEs 8.

[0066] Each PE 8 performs a multiply-and-accumulate operation using data a of matrix A, data b of matrix B, and data c of matrix C. The operation result may be accumulated with the previous result.

[0067] In addition, the output (matrix C) from each PE8 constituting the last row (the bottom row on the paper in Figure 2) may be input to the scratchpad memory 6 and overwrite the matrix C previously stored in the scratchpad memory 6.

[0068] (B) Operation The process of complex matrix multiplication in the accelerator 1 according to the embodiment configured as described above will be described according to the flowchart (steps S1 to S7) shown in Fig. 7 with reference to Figs. 8 to 16. Figs. 8 to 16 are diagrams illustrating examples of complex matrices A, B, and C processed in the accelerator 1, respectively.

[0069] Figure 8 shows complex number matrices A, B, and C input to the accelerator 1. Each of the complex number matrices A, B, and C has multiple complex numbers. Each complex number has a real part and an imaginary part. These complex number matrices A, B, and C are stored in memory 3.

[0070] In step S1, the processor-equipped core 5 reads the complex matrix C from the memory 3 and writes it into the scratchpad memory 6 (see FIG. 9).

[0071] In step S2, the arithmetic unit core 5 reads the complex matrix A from the memory 3 and writes it into the scratchpad memory 6 (see FIG. 10).

[0072] In step S3, the arithmetic unit core 5 reads the complex number matrix B from the memory 3, separates the real part and the imaginary part, and writes them into the scratchpad memory 6 (see FIG. 11).

[0073] The order of steps S1 to S3 is not limited to this, and the order may be changed, or at least some of the processes may be performed in parallel, and the steps may be changed and performed as appropriate.

[0074] In step S4, the arithmetic unit core 5 loads the data of the real and imaginary parts of the complex matrix B into the register of the PE8.

[0075] Figures 12 and 13 show an example of loading the real and imaginary parts of a complex matrix B on the scratchpad memory 6 into 16 PEs 8 (PE group) arranged in a 4x4 matrix, where Figure 12 shows the state of each register in the PE group before loading, and Figure 13 shows the state of each register in the PE group after loading.

[0076] 13, the real part and imaginary part of the complex numbers included in the complex number matrix B are loaded into the register of each PE8. As a result, each PE8 stores the real part and imaginary part of a corresponding complex number.

[0077] In step S5, the arithmetic unit core 5 multiplies the complex matrix A by the real part of the complex matrix B (first multiplication) and adds the complex matrix C (A×B real part+C).

[0078] At this time, as shown in FIG. 14, the complex matrix A on the scratchpad memory 6 is read into i-swap 10, and is stored as is in the timing adjustment block 9b without being multiplied by i in i-swap 10, i.e., without swapping the real and imaginary parts (see symbol P1).

[0079] The complex matrix C on the scratchpad memory 6 is also stored in the timing adjustment block 9a (see symbol P2).

[0080] Then, in the PE group, the real part of the complex matrix B, the complex matrix C of the timing adjustment block 9a, and the complex matrix A of the timing adjustment block 9b are used to calculate A×B real part + C. The calculation result is written back to the complex matrix C of the scratchpad memory 6 (see symbol P3).

[0081] In step S6, the arithmetic unit core 5 multiplies the complex matrix A by the imaginary part of the complex matrix B (second multiplication) and adds the matrix C (A×B imaginary part+C).

[0082] 15, the complex matrix A on the scratchpad memory 6 is read into i-swap 10, where it is multiplied by i and the real and imaginary parts are swapped (see symbol P11). The value of matrix A, whose imaginary and real parts have been swapped by i-swap 10, is stored in timing adjustment block 9b (see symbol P12).

[0083] The complex matrix C on the scratchpad memory 6 is also stored in the timing adjustment block 9a (see symbol P13).

[0084] Then, in the PE group, the imaginary part of the complex matrix B, the complex matrix C of the timing adjustment block 9a, and the complex matrix A of the timing adjustment block 9b are used to calculate A×B imaginary part + C. The calculation result is written back to the complex matrix C of the scratchpad memory 6 (see symbol P14).

[0085] In step S7, the arithmetic unit core 5 copies the data of the matrix C that has been overwritten and updated on the scratchpad memory 6 to the memory 3 as a complex number matrix C (see FIG. 16), and then ends the process.

[0086] In actual cases, there are cases where a complex matrix multiplication calculation is performed with a size exceeding the size of LMP7 (LMP size).

[0087] Figure 17 shows the matrix multiplication when the input complex matrix size is (128,128,128).

[0088] FIG. 17 shows an example in which the LMP size is (64, 64, 64), the size of the scratch pad memory 6 is 756 KB, and the size of the read complex number matrix is ​​(128, 128, 128).

[0089] In such a case, for example, complex matrix C can be fixed and complex matrices A and B can be moved. Moving complex matrices A and B means calculating the matrix product while shifting the submatrices of the complex matrices A and B to be operated on one by one in the inner product direction.

[0090] 18 and 19 are diagrams for explaining a method for calculating a matrix product while moving submatrices (complex matrices). Fig. 18 shows an example in which matrix A and matrix B are each divided into 2x2 submatrices to calculate matrix C, and Fig. 19 shows an example in which matrix A and matrix B are each divided into 3x3 submatrices to calculate matrix C.

[0091] Matrix C can be obtained by calculating the partial matrix products of the submatrices of matrix A and matrix B while shifting the submatrix to be calculated, and then summing these partial matrix products.

[0092] (C) Effects In the accelerator 1 configured as described above, the arithmetic unit core 5 reads the complex matrix B from the memory 3, separates the real part and the imaginary part, and writes them to the scratchpad memory 6 when calculating the matrix product αA×B+βC using complex matrices A, B, and C. The arithmetic unit core 5 then loads the data of the real part and the imaginary part of the complex matrix B written to the scratchpad memory 6 into the registers of the PEs 8. At this time, the arithmetic unit core 5 stores the real part and the imaginary part in the registers of each PE 8, respectively.

[0093] At this time, only an area of ​​the same size as the elements of the read complex matrix is ​​used in the scratchpad memory 6. This makes it possible to fix the size of the scratchpad memory 6 in use and also to reduce the size used, compared to conventional methods (e.g., the M1 algorithm) that perform DGEMM (double-precision matrix multiplication) by multiplying each element of the complex matrix A by i and adding the result to the adjacent column.

[0094] Furthermore, in the scratchpad memory 6, rearrangement of matrix elements only needs to be performed on the complex matrix B, thereby reducing the number of elements to be rearranged. Furthermore, reducing the number of rearrangements reduces the waiting time of the LMP7, allowing the matrix multiplication size per operation to be increased. For example, a large LMP7 (with a large number of PEs) capable of processing large matrix multiplications such as (64,64,64) can be installed, allowing processing of large matrices (complex matrices) that can demonstrate the performance of this large LMP7. As a result, complex matrix multiplication can be calculated at low cost.

[0095] Also, by reducing the number of matrix element rearrangements, the latency of the LMP7 is reduced, allowing for a larger LMP7 to be installed. A larger LMP7 generally has a more efficient hardware implementation than a smaller LMP, making it easier to achieve actual performance.

[0096] Matrix multiplication generally involves multi-stage blocking, and matrix B can be reused in the registers of PE8 or scratchpad memory 6, so the cost of rearrangement appears relatively small. For example, with the ARM sve instruction (supported by a64fx, etc.), values ​​from memory such as HBM (High Bandwidth Memory) can be read while separating the real and imaginary parts using instructions such as ld2. The read value can be written directly to scratchpad memory 6, so there is no particular overhead for matrix B.

[0097] Furthermore, each complex number included in the complex number matrix B is separated into a real part and an imaginary part. Then, the arithmetic unit core 5 loads the data of the real part and the imaginary part of each complex number into the register of the PE 8. As a result, each PE 8 stores the real part and the imaginary part of a corresponding complex number.

[0098] The arithmetic unit core 5 also causes the LMP7 to multiply the real parts of matrix A and matrix B, and add matrix C (A × B real parts + C). The arithmetic unit core 5 also causes the LMP7 to multiply the imaginary parts of matrix B, with the imaginary parts of matrix A swapped, and add matrix C (A × B imaginary parts + C). By separating matrix B into real and imaginary parts, the number of elements that need to be rearranged when performing complex matrix multiplication can be reduced.

[0099] In addition, by using i-swap10 to swap the imaginary and real parts of complex matrix A, it is possible to efficiently calculate complex matrix multiplication and reduce the cost of rearranging the elements of complex matrix A. Furthermore, by using i-swap10, it is possible to eliminate the need to rearrange the elements of matrix A.

[0100] Furthermore, since the real and imaginary parts of matrix A can be handled as they are for calculation, the number of elements to be rearranged increases slightly, but the size of real matrix multiplication per operation can be made larger than that of the known 4M algorithm, for example.

[0101] Furthermore, compared to the known 4M algorithm, the accelerator 1 can increase the matrix multiplication size per operation. Even if the real matrix multiplication size is increased, the capacity of the scratchpad memory 6 does not exceed the capacity required to read matrices A, B, and C. Therefore, compared to the known 1M algorithm, the capacity of the scratchpad memory 6 can be reduced, leading to cost savings.

[0102] By performing the processing equivalent to squaring i outside the PE group, i-swap10 can configure the PE array (PE group) itself using the real matrix multiplication. Also, by performing the processing equivalent to squaring i outside the PE group, flag propagation within the PE group is no longer necessary, reducing circuit implementation costs.

[0103] In addition, this accelerator 1 multiplies the real parts of matrix A and matrix B and adds them together (A × B real parts + C), and also multiplies the imaginary parts of matrix B with the imaginary parts swapped between them (A × B imaginary parts + C).

[0104] FIG. 20 is a diagram for explaining the processing of matrix A and matrix B in the accelerator 1 according to an embodiment.

[0105] 20, for each element constituting matrix A, the element of each column is multiplied by i and added to the adjacent column (see symbol P1). Consider the matrix product of matrix A and matrix B after such rearrangement.

[0106] Here, the complex number A=a r +ja i , complex number B=b r +jb iThen, the product of complex numbers (A × B) can be calculated as follows. For ease of distinction, let the imaginary unit be j.

[0107] Matrix multiplication can be broken down into the products of each element. By focusing on the calculation of one element, the matrix product A × B can be calculated using the following formula: (a r +ja i )×(b r +jb i ) =(a r +ja i )×b r +(a r +ja i )×jb i =(a r b r +ja i b r )+(ja r b i +j 2 a i b i ) =(a r b r -a i b i )+(a r b r +a r b i )j ···(1)

[0108] Looking at the above equation (1), the multiplication of matrix A and the real part of matrix B (a r b r ,a r b r ) and multiplication of matrix A with the imaginary part and real part swapped and the imaginary part of matrix B (a r b i ,a i b i ) is included.

[0109] This allows the product of complex matrices A and B to be calculated by a first operation (A×B real part + C) that includes multiplication of matrix A by the real part of matrix B and addition of matrix C, and a second operation (A×B imaginary part + C) that includes multiplication of matrix A with the imaginary part swapped by the real part of matrix B and addition of matrix C.

[0110] In addition, in this accelerator 1, by multiplying the real parts of matrix A and matrix B and adding matrix C (A × B real parts + C), and by multiplying the imaginary parts of matrix B with the imaginary parts swapped between them (A × B imaginary parts + C), it is possible to limit the target of rearrangement to matrix B. In addition, the size of matrix multiplication that can be performed at one time can be made larger than that of the 4M algorithm.

[0111] (D) Other The disclosed technology is not limited to the above-described embodiment, and can be implemented in various modifications without departing from the spirit of the present embodiment.

[0112] For example, in the above-described embodiment, the accelerator 1 may perform a real matrix multiplication calculation. In this calculation, a flag is set in i-swap 10 to not swap the real part and the imaginary part (for example, flag = 0), and then the calculation of the real matrix multiplication can be performed in the same way as a known accelerator.

[0113] In the above-described embodiment, for simplicity, an example is shown in which each LMP 7 has a two-dimensional structure in which each PE 8 processes one row and one column, but the present invention is not limited to this.

[0114] Furthermore, each configuration and each process of this embodiment can be selected as needed, or can be combined appropriately.

[0115] Furthermore, the above disclosure will enable those skilled in the art to implement and manufacture the present embodiment. [Explanation of symbols]

[0116] 1 Accelerator (matrix multiplier) 2 ACC Block 3. Memory 4 ACC Core 5 Core with arithmetic unit 6 Scratchpad Memory 7 LMP 8 PE 9a, 9b Timing adjustment block 10 i-swap (swap section) 11,12 Selector 13 Sign Reversal Block

Claims

1. A matrix multiplication calculator having a memory and a plurality of calculator elements, In a calculation of a matrix product using a first complex matrix, a second complex matrix, and a third complex matrix, Separating the complex numbers included in the second complex matrix into real parts and imaginary parts; loading the real part and the imaginary part of the complex number into the plurality of operator elements; performing a first multiplication of the first complex matrix by the real part of the second complex matrix, and adding the third complex matrix to the result of the first multiplication; swapping the imaginary part and the real part for each complex number in the first complex matrix; a second multiplication of the first complex matrix with the imaginary part and the real part swapped by the imaginary part of the second complex matrix, and an addition of the third complex matrix to the result of the second multiplication; A matrix multiplication calculator comprising:

2. a swapping unit that swaps the imaginary part and the real part of each complex number in the first complex matrix by multiplying the real part and the imaginary part of the first complex matrix by an imaginary unit i, respectively; 2. The matrix multiplication calculator according to claim 1, wherein:

Images