A processor for matrix inversion and matrix multiplication

Abstract

The present invention provides a processor for matrix inversion and matrix multiplication, comprising a processing unit (PU) array, a divider array, a memory, and a finite state machine; the processing unit (PU) array is used to perform matrix multiplication and matrix inversion operations; the processing unit (PU) array includes a systolic array. The systolic array in the present invention can implement both matrix multiplication and LU decomposition of matrices and inversion of upper and lower triangular matrices, thereby implementing arbitrary matrix inversion, rather than using two different hardware resources to implement matrix multiplication and matrix inversion separately, thereby significantly improving hardware utilization.

Description

Technical Field

[0001] The present invention relates to a processor for matrix inversion and matrix multiplication. Background Art

[0002] In recent years, with the rapid development of mobile communications technology, humanity has entered the 5G era. The primary goal of communications technology development is to increase communication speeds. However, due to issues such as nonlinear effects in optical fibers, which are not applicable to the traditional Shannon channel model, the lack of comprehensive theoretical research is one of the main factors limiting further increases in communication speeds. Furthermore, the poor robustness of many traditional solutions to communications problems is another bottleneck in the development of communications technology. Deep learning, which relies on fitting target mappings using large amounts of data, offers advantages such as high robustness and strong learning capabilities. Therefore, deep learning-based solutions to communication problems, such as channel estimation, have attracted widespread attention. However, due to the high complexity of matrix inversion operations, these solutions also face difficulties in hardware implementation.

[0003] Since matrix multiplication and matrix inversion operations cannot be derived from each other algorithmically, that is, we cannot use matrix multiplication to obtain the inverse of a matrix, nor can we obtain the product of matrices through matrix inversion, in many existing solutions that require both matrix multiplication and matrix inversion functions, we can only design matrix multiplication modules and matrix inversion modules separately. Currently, the widely used matrix multiplication modules are all implemented based on systolic arrays. For matrix inversion modules, due to the high complexity of the algorithm based on the adjoint matrix, the inversion of large-scale matrices currently widely used in the communications field is based on the LU decomposition + upper and lower triangular matrix inversion algorithm. Currently, there is no design that reuses the same hardware resources to implement matrix multiplication and matrix inversion. Summary of the Invention

[0004] Purpose of the Invention: Solutions to communication problems based on deep learning are currently attracting widespread attention due to their advantages, such as strong learning ability, good robustness, and strong portability. In such solutions, in addition to the matrix multiplication operations required for forward reasoning, matrix inversion operations are often required during the training process to better fit the communication problem and improve accuracy. This will bring extremely high complexity to the hardware implementation. The present invention addresses this bottleneck problem and proposes an effective solution that can reuse limited hardware resources to implement both matrix multiplication and matrix inversion operations, thereby improving hardware utilization.

[0005] The present invention specifically provides a processor for matrix inversion and matrix multiplication, comprising a processing unit PU array, a divider array, a memory and a finite state machine;

[0006] The processing unit PU array is used to perform matrix multiplication and matrix inversion operations;

[0007] When performing matrix multiplication and matrix inversion operations, the processing unit PU array first divides the matrix into two or more sub-blocks, and the sub-blocks are arranged in two dimensions in space. Since the PU array performs different operations for different sub-blocks, the sub-blocks are numbered as follows to facilitate the subsequent description:

[0008] When the processing unit PU array performs the following real matrix multiplication operation C=A×B, the total number of rows BLOCK_ROW_NUM and the total number of columns BLOCK_COL_NUM of the sub-blocks are related to the number of rows and columns of the right matrix B:

[0009]

[0010]

[0011]

[0012]

[0013] Among them, R m×n represents the m×n real number space; a i,p represents the element in the i-th row and p-th column of matrix A; b p, represents the element in the pth row and jth column of matrix B; c i,j Represents the element in the i-th row and j-th column of matrix C.

[0014] The total number of rows BLOCK_ROW_NUM and the total number of columns BLOCK_COL_NUM of the sub-blocks are related to the number of rows and columns of the right matrix B, and are expressed by the following formula:

[0015] BLOCK_ROW_NUM=n / N, BLOCK_ROW_NUM=k / N

[0016] Wherein, N is the number of rows or columns of the processing unit PU array (in the present invention, the number of rows of the PU array is equal to the number of columns);

[0017] When the processing unit PU array performs matrix inversion operation D=E -1 When , the total number of rows BLOCK_ROW_NUM and the total number of columns BLOCK_COL_NUM of the sub-blocks are related to the number of rows of the original matrix E, which can be expressed by the following formula:

[0018] BLOCK_ROW_NUM=BLOCK_ROW_NUM=m / N

[0019] Among them, E and D are square matrices respectively, E∈R m×m , D∈R m×m .

[0020] The sub-block currently processed by the processing unit PU array is identified by the row number block_row_idx and column number block_col_idx where the sub-block is located; the iteration of the sub-block is performed from top to bottom and from left to right, with columns taking precedence over rows. The first sub-block to be iterated is the sub-block of the first column, and the sub-blocks of each column are iterated from top to bottom, then the second column, until the last column is iterated; for sub-blocks in the same column, the output of the previous row is the input of the next row, and the output of the sub-block of the last row is the final output.

[0021] The processing unit PU array includes an N×N systolic array PU ij , 1≤i,j≤N;

[0022] The systolic array PU ij It is pulsating in the vertical direction and broadcasting in the horizontal direction;

[0023] The processing unit PU array includes a processing unit PU, which is a basic component of the processing unit PU array. en is an enable signal of the processing unit PU, and X in and Y in There are two data input ports;

[0024] There is a register reg inside the processing unit PU. When the processing unit PU array performs a matrix multiplication operation, the register reg stores an element b of the right matrix B. i, , 1≤i≤n,1≤j≤k, the data Y output by the processing unit PU out Z1 and data input port Y in The sum of the input data, Z1 refers to the data input port X in The product X of the input data and the data stored in register reg in ·reg, corresponding to formula (1), abbreviated as multiplication and addition;

[0025] When the processing unit PU array performs a matrix inversion operation, there are two cases. In the first case, the register reg stores an output data of the divider, and the processing unit PU inputs the data to port Y. in The input data is output after subtracting Z2, where Z2 refers to the data input port X in The product of the input data and the data stored in register reg corresponds to formula (2), which is simply expressed as multiplication and subtraction; in the second case, the processing unit PU inputs the data into port Y in The input data is stored in register reg and the output data is input to port Y at the same time. in The input data corresponds to formula (3), which is simply referred to as transmission; the selection signal sel is used to select PU ij What needs to be done:

[0026] Y out =Y in +X in ·reg; (1)

[0027] Y out =Y in -X in ·reg; (2)

[0028] Y out =Y in , reg=Y in ; (3)

[0029] Among them, when the matrix is multiplied, PU ij Execute operation (1), the columns of the right matrix are pre-stored in the rows of the processing unit PU array, and the elements of the left matrix A are taken from the X of the processing unit PU array in the following order: in Port Parallel Input:

[0030] Take N=4 as an example:

[0031] The first clock cycle: [a 1, ,0,0,0],

[0032] The second clock cycle: [a 2, , a 1,2 ,0,0],

[0033] The third clock cycle: [a 3, , a 2, , a 1, ,0],

[0034] The 4th clock cycle: [a 4, , a 3, , a 2, , a 1, ],

[0035] The 5th clock cycle: [a 5, , a 4, , a 3, , a 2, ], ...

[0037] The (m+3)th clock cycle: [0, 0, 0, a m,4 ].

[0038] The processor implements matrix inversion based on LU (lower-upper triangular) decomposition and the inverse algorithm of the lower-upper triangular matrix. Among them, the LU decomposition is based on the right-looking algorithm. When performing LU decomposition, the processing unit PU executes operation (2) or (3), and the division in (3) is implemented by an external divider array; from the perspective of the column j to be updated, 1 ≤ j ≤ N, when the column j is updated and becomes an updated column, it starts from the updated column 1 and accepts data update processing of the updated column k on the column j to be updated from left to right, 1 ≤ k < j. Then, the elements below the pivot element of the column j are divided by the pivot element, and the column j becomes an updated column. Therefore, when the PU array of processing units performs LU decomposition, it is a pipeline with N levels. One row of the array stores one updated column, and the columns to be updated are sequentially input from above the array. The specific implementation steps include:

[0039] In the fill pipeline stage, the first column of the matrix enters the PU array of processing units in parallel as the column j to be updated. Since there is no updated column before the first column, the first row of the processing unit PU performs operation (3) on the first column and stores the result in the first row of the processing unit PU, which becomes an updated column and performs data update on the columns to be updated flowing through the first column later, that is, operation (2); then the second column of the matrix enters the PU array of processing units as the column j to be updated. The first row of the processing unit PU performs data update on the column j to be updated. When the data reaches the second row of the processing unit PU, after the column j to be updated performs operation (3), the result is stored in the second row of the processing unit PU, which becomes an updated column until all the processing units PU are filled; in the pipeline stage, when the column j to be updated passes through the PU ij where 1 ≤ i < k, 1 ≤ j ≤ N, it will be processed by the i-th updated column; the fill pipeline and the pipeline stage alternate. Each iteration obtains N updated columns. After iterating more than twice, all the updated columns are finally obtained; when the column j to be updated passes through the PU ij it is not processed by all the elements of the i-th updated column, but only needs to be processed by the elements below the pivot element of the i-th updated column;

[0040] The inverse algorithm of the lower-upper triangular matrix is based on the Gauss elimination method. When performing the inverse of the lower-upper triangular matrix, the processing unit PU executes operation (2) or (3); the upper triangular matrix is input in parallel by rows from the X in port of the PU array, and the later row is delayed by one cycle compared to the previous row; for the lower triangular matrix, it is first transposed into an upper triangular matrix, and after performing the inverse according to the Gauss elimination method, it is transposed again to obtain the output of the original lower triangular matrix.

[0041] The function of the divider array is for the data input port Y of one row of the processing unit PU inThe input data performs a division operation; the divider array includes an N×1 divider and two groups of selectors;

[0042] The two groups of selectors are respectively used to generate two inputs of the divider, namely the divisor and the dividend; the first group of selectors is used to generate the divisor of the divider array, the size of the first group of selectors is N×1, each selector in the first group of selectors has N inputs, and can be replaced by a log2(N) level two-to-one selector, row_sel is the control signal of the first group of selectors, and the first group of selectors share the same row_sel;

[0043] The second group of selectors is used to generate the dividend of the divider array. The second group of selectors contains only one N-input selector. The N inputs are the broadcast inputs X of each row of the processing unit PU array. in , the second group of selectors and the first group of selectors share the same row_sel, and the N dividers share the same dividend.

[0044] The memory includes an LU decomposition intermediate result memory and a sub-block intermediate result memory;

[0045] The LU decomposition intermediate result memory is used to store the LU decomposition data input port X in The input data is because when LU is decomposed, the processing unit PU of some sub-blocks cannot obtain X from the sub-block. in , so this part of data needs to be stored;

[0046] The sub-block intermediate result memory is used to store the intermediate results between sub-blocks. Except for the last sub-block in each column, which directly outputs the result as the final output, the outputs of other sub-blocks need to be stored as the data input port Y of the next sub-block. in The memory includes N FIFOs, fifo_r and fifo_w are read enable signals and write enable signals respectively, and the N FIFOs share the same read enable signal and write enable signal. The processing unit PU array also includes three groups of selectors, which are used to generate PU ij Data input port X in and Y in Input data, 1≤i,j≤N;

[0047] The first set of selectors is used to generate PU ij Data input port X in Input data; the processing unit PU array is broadcast in the horizontal direction, and each row of processing units PU shares the same X inThe size of the first group of selectors is N×1. Each selector in the first group of selectors has two inputs, where the first input is the custom sequence data_in2, and the second input is related to the number of rows corresponding to the selector. The second input of the i-th row selector in the first group of selectors is PU ii Input Y in , 1≤i≤N; mode_sel is the control signal of the selector, the first group of selectors share the same mode_sel; the output of the i-th selector in the first group of selectors is the corresponding PU ij Input X in ;

[0048] The second set of selectors is used to generate PU ij Data input port Y in Input data; The processing unit PU array is pulsating in the vertical direction, so each processing unit PU needs a selector. The second group of selectors is N×N. Each selector in the second group of selectors has two inputs. For the selectors from the 2nd row to the Nth row, the first input is the output of the jth divider, and the second input is the PU (-1) The output of , 2≤i≤N,1≤j≤N; the first input of the first row selector in the second group of selectors is the output of the j-th divider, and the second input is the output of the j-th selector in the third group of selectors; div_sel0[i] and div_sel1 are the control signals of the second group of selectors;

[0049] For the i-th row selector in the second group of selectors, columns 1 to 1 share the same control signal div_sel0[i], 1≤i≤N;

[0050] For the i-th row selector in the second group of selectors, the i+1th column to the Nth column share the same signal div_sel1[i], 1≤i≤N-1;

[0051] The output of the selector in the i-th row and j-th column of the second group of selectors is the input Y of the corresponding PUij in ;

[0052] The third group of selectors is used to adapt the input of the first row of different sub-blocks. In the block division, for the first block of each column, the input of the first row processing unit PU is the custom sequence data_in1, while the input of the first row processing unit PU of other blocks is the output of the memory. The selector needs to make different choices for different blocks; the third group of selectors is only connected to the data input port Y of the first row processing unit PU. inThe third group of selectors is 1×N. Each selector in the third group has two inputs. The first input is the custom sequence data_in1, and the second input is related to the number of columns of the selector. The second input of the j-th column selector in the third group of selectors is the output of the j-th FIFO, 1≤j≤N; data_sel is the control signal of the selector, and the third group of selectors shares the same data_sel.

[0053] The finite state machine includes a total of 8 states, namely: 1 start state for initialization, waiting for matrix inversion or matrix multiplication instructions, 1 end state for outputting results, 2 states for upper and lower triangular matrix inversion, 2 states for matrix multiplication, and 2 states for LU decomposition;

[0054] The start state is used for initialization and waiting for a matrix inversion or matrix multiplication instruction;

[0055] The end state is used to output the result;

[0056] The inversion of the upper and lower triangular matrices has two states, S1 and S2. In the S1 state, a row in the processing unit PU array performs a division operation, and the other rows perform multiplication and subtraction operations. The number of rows performing the division operation is controlled by div_sel. In the S2 state, each row of the processing unit PU array performs a multiplication and subtraction operation. The state transition of each sub-block is: S1→S2→S1→S2→...→S1→S2→S2→...→S2→S2→S2, that is, S1 and S2 alternate until all processing units PU have performed one division operation, and then the S2 state loops loop_num times. The number of loops loop_num is related to the size of the matrix matrix_size, which is expressed by the following formula:

[0057] loop_num=(matrix_size-N×(block_row_idx+1))

[0058] After each block loop, the output data contains N×N elements of the final inverse matrix, and the remaining data is input to the next block for iteration;

[0059] The first state of matrix multiplication is to input the right matrix into the processing unit PU array in parallel by row and store it in the reg of the processing unit PU; the second state is to input the left matrix into the processing unit PU array in parallel by row for multiplication and addition. The state transition of each block is: S4→S4→...→S4→S4→S5→S5→...→S5→S5→S5. After S4 is cycled 2N-1 times, the S5 state is cycled loop_num times. The number of loops loop_num is related to the number of columns of the left matrix matrix_a_col_size, which is expressed by the following formula:

[0060] loop_num=(matrix_a_col_size-N×(block_row_idx+1))

[0061] The LU decomposition has two states, S6 and S7. In the S6 state, one row in the processing unit PU array performs a division operation, and the other rows perform multiplication and subtraction operations. In the S7 state, each row in the processing unit PU array performs a multiplication and subtraction operation. The state transition of each sub-block is: S6→S7→S6→S7→...→S6→S7→S6→...→S7→S7→S7, that is, S6 and S7 alternate until all processing units PU have performed one division operation, and then the S7 state loops loop_num times. The number of loops loop_num is related to the size of the matrix matrix_size, which is expressed by the following formula:

[0062] loop_num=(matrix_size-N×(block_row_idx+1)).

[0063] The present invention supports block iteration of matrices, that is, dividing a matrix into several sub-blocks to solve multiplication and inversion operations of larger-scale matrices.

[0064] Beneficial Effects: This solution proposes a new systolic array-based processor for matrix multiplication and matrix inversion. Compared with existing solutions, the systolic array in this design can perform both matrix multiplication and LU decomposition of matrices and inversion of upper and lower triangular matrices, thus enabling arbitrary matrix inversion. This significantly improves hardware utilization, rather than requiring two separate hardware resources for matrix multiplication and matrix inversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0066] Figure 1a This is the PU input and output pin diagram.

[0067] Figure 1b It is the internal architecture diagram of the PU.

[0068] Figure 2 It is the overall architecture diagram of the present invention.

[0069] Figure 3 It is the state transition diagram. Detailed implementation manners

[0070] As Figure 2 shown, the present invention provides a processor for matrix inversion and matrix multiplication, including a processing unit PU (Process Unit) array, a divider array, a memory, and a finite state machine. Among them, the size N of the PU array is taken as 4; PU ij (1 ≤ i ≤ N, 1 ≤ j ≤ N) represents a processing unit PU; DIV i (1 ≤ i ≤ N) represents a divider; MEM j (1 ≤ j ≤ N) represents a FIFO; data_in1 and data_in2 are two data input ports of the processing unit PU array; the selection signal mode_sel is used to select the data input from the data input port data_in2 or the data input port Y of PU ii (1 ≤ i ≤ N) as the input X of the processing unit PU in , and each row of the processing unit PU shares the same mode_Sel signal; the selection signal mode_sel is used to select the data input from the data input port data_in1 or the output data of MEM in (1 ≤ j ≤ k) as the input X of the processing unit PU j ; the selection signal row_sel is used to select the row number of the processing unit PU that needs to perform the division operation; div_sel0 and div_sel1 are used to select the output data of the processing unit PU <0OO0078>(1 ≤ i < N, 1 ≤ j ≤ N) or the output data of the divider DIV ij (1 ≤ i ≤ N) and input it into the data input port Y of the processing unit PU i (1 ≤ i < N, 1 ≤ j ≤ N) () . in

[0071] First of all, this solution supports block iteration of matrices, that is, dividing the matrix into two or more sub-blocks to solve the multiplication and inversion operations of larger-scale matrices. Since for different sub-blocks, the processing unit PU array will have different operations, for the convenience of subsequent description, the following settings are made for the numbering of sub-blocks:

[0072] The sub-blocks are arranged in two dimensions in space. When the processing unit PU array performs matrix multiplication operation C=A×B, where A, B, and C are matrices respectively, specifically, A∈R m×n , B∈R n×k , C∈R m×k The total number of rows BLOCK_ROW_NUM and the total number of columns BLOCK_COL_NUM of the sub-blocks are related to the number of rows and columns of the right matrix B, which can be expressed as follows:

[0073] BLOCK_ROW_NUM=n / N, BLOCK_ROW_NUM=k / N

[0074] Where N is the number of rows or columns of the processing unit PU array (the number of rows of the PU array in the present invention is equal to the number of columns). -1 When A and D are square matrices, A∈R m×m , D∈R m×m The total number of rows BLOCK_ROW_NUM and the total number of columns BLOCK_COL_NUM of the sub-blocks are related to the number of rows of the original matrix A (the present invention only considers the narrow inverse matrix, so the number of rows and columns of the original matrix A are equal), which can be expressed by the following formula:

[0075] BLOCK_ROW_NUM=BLOCK_ROW_NUM=m / N

[0076] The subblock currently being processed by the PU array is identified by the row number, block_row_idx, and column number, block_col_idx, of the subblock. Subblock iteration proceeds from top to bottom and left to right, with columns taking precedence over rows. That is, the subblocks in the first column are iterated first, followed by the subblocks in each column from top to bottom, then the subblocks in the second column, and so on until the last column is iterated. For subblocks in the same column, the output of the previous row becomes the input for the next row, and the output of the subblock in the last row becomes the final output.

[0077] Processing unit PU array:

[0078] The processing unit PU array is the core part of the present invention, which includes an N×N systolic array PU ij (1≤i,j≤N) and three sets of selectors. Systolic array PU ij (1≤i,j≤N) is pulsating in the vertical direction and broadcast in the horizontal direction. PU (Process Unit) is the basic component unit of the PU array, such as Figure 1a 、 Figure 1bAs shown in the figure, it is the input / output pin diagram and internal architecture of the processing unit PU. En is the enable signal of the processing unit PU, Xin and Yin are two data input ports. There is a register inside the processing unit PU array, denoted as reg, and the data stored therein will perform corresponding processing on Xin and Yin. The processing unit PU is used to execute three operations: multiply-add, multiply-subtract, and transfer, which respectively correspond to formulas (1), (2), and (3):

[0079] Y out =Y in +X in ·reg; (1)

[0080] Y out =Y in -X in ·reg; (2)

[0081] Y out =Y in , reg = Y in ; (3)

[0082] Among them, when performing matrix multiplication, PU ij (1 ≤ i, j ≤ N) executes operation (1). Each column of the right matrix is pre-stored in each row of the processing unit PU array, and the elements of the left matrix are input in parallel by row from the X in port of the processing unit PU array, and the next row is delayed by one cycle compared to the previous row. This design realizes matrix inversion based on LU (lower-upper triangular) decomposition + upper and lower triangular matrix inversion algorithm. Among them, LU decomposition is based on the right-looking algorithm. Therefore, when performing LU decomposition, the processing unit PU executes operation (2) or (3), and the division in (3) is implemented by an external divider array. From the perspective of the column j to be updated (1 ≤ j ≤ N), when the update of column j is completed and it becomes an updated column, it starts from the updated column 1, and from left to right, the updated column k (1 ≤ k < j) performs data update processing on the column j to be updated. Then, the elements below the pivot element of column j are divided by the pivot element, and column j becomes an updated column. Therefore, when the processing unit PU array performs LU decomposition, it is a multi-stage pipeline of N levels. One row of the array stores an updated column, and the columns to be updated are input sequentially from above the array. The specific implementation steps are as follows:

[0083] In the filling pipeline stage, the first column of the matrix (column 1 to be updated) enters the processing unit PU array in parallel. Since there is no updated column before the first column, the first row processing unit PU performs operation (3) on the column and stores the result in the row processing unit PU, which becomes the updated column. The data of the columns to be updated that flow through here are updated, that is, operation (2). Then the second column of the matrix (column 2 to be updated) enters the processing unit PU array, and the first row processing unit PU updates the data on it. When the data reaches the second row processing unit PU, the column 2 to be updated performs operation (3), and the result is stored in the second row processing unit PU, which becomes the updated column, until all the processing units PU are filled. In the pipeline stage, the column k to be updated passes through the PU ij (1≤i<k,1≤j≤N), it will be processed by the i-th (1≤i<k) updated column. Filling pipeline and pipeline stage are performed alternately, and each iteration can get N updated columns. After several iterations, all updated columns are finally obtained. It should be noted that when the column k to be updated passes through PU ij (1≤i<k, 1≤j≤N), not all elements of the i-th updated column are processed, but only the elements below the column pivot are processed.

[0084] The inversion of the upper and lower triangular matrices is based on the Gaussian elimination method. Therefore, when inverting the upper and lower triangular matrices, the processing unit PU performs operation (2) or (3). The upper triangular matrix is obtained from the X of the PU array by row. in The port is input in parallel, and the next row is delayed by one cycle compared to the previous row. For the lower triangular matrix, first transpose it to become an upper triangular matrix, and then invert it according to the above method and transpose it again to get the output of the original lower triangular matrix. sel is used to select PU ij (1≤i,j≤N) The operation to be performed, en is the enable signal.

[0085] Three sets of selectors are used to generate PU ij Two inputs X (1≤i,j≤N) in and Y in The first set of selectors is used to generate PU ij (1≤i,j≤N) input X in Since the processing unit PU of some sub-blocks cannot obtain the input X in the sub-block in Therefore, this part of the data can only be stored in advance and then input from the custom sequence input port so that the processing unit PU can obtain the correct input. Since the processing unit PU array is broadcast in the horizontal direction, each row of processing units PU shares the same X in, Therefore, the size of the first group of selectors is N×1. Each selector in this group has two inputs. The first input is the custom sequence data_in2, and the second input is related to the number of rows corresponding to the selector. The second input of the i-th (1≤i≤N) row selector is PU ii Input Y (1≤i≤N) in mode_sel is the control signal of the selector. The selectors in this group share the same mode_sel signal. The output of the i-th selector is the corresponding PU ij (1≤i,j≤N) input X in .

[0086] The second set of selectors is used to generate PU ij Input Y (1≤i,j≤N) in Since the processing unit PU array is pulsating in the vertical direction, each processing unit PU requires a selector. The second set of selectors is N×N. Each selector in this set also has two inputs. For the selectors from the 2nd row to the Nth row (2≤i≤N,1≤j≤N), the first input is the output of the jth divider, and the second input is the PU. (-1) (2≤i≤N,1≤j≤N); the first input of the first row selector (i=1,1≤j≤N) is the output of the jth divider, and the second input is the output of the jth selector in the third group of selectors. div_sel0[i](1≤i≤N) and div_sel1(1≤i≤N-1) are the control signals of the second group of selectors. For the i-th (1≤i≤N) row selector, the 1st column to the i-th (1≤i≤N) column share the same control signal div_sel0[i]. For the i-th (1≤i≤N-1) row selector, the i+1th column to the Nth column share the same signal div_sel1[i](1≤i≤N-1). The output of the i-th row j-th column selector is the input Y of the corresponding PUij(1≤i,j≤N) in .

[0087] The third set of selectors is used to adapt the input of the first row of different blocks. In the block division, for the first block of each column, the input of the first row processing unit PU is the custom sequence data_in1, while the input of the first row processing unit PU of other blocks is the output of the FIFO (First Input First Output, first in first out chip), so the selector needs to make different choices for different blocks. Since the third set of selectors is only compatible with the input Y of the first row processing unit PU, inThe selectors in this group are 1×N. Each selector has two inputs: the first is the custom sequence data_in1, and the second is related to the number of columns in the selector. The second input of the jth (1≤j≤N) column selector is the output of the jth FIFO. data_sel is the control signal for the selector, and the selectors in this group share the same data_sel signal.

[0088] Divider Array:

[0089] The function of the divider array is to divide the input Y of a row of processing units PU in Perform division operations. It consists of an N×1 divider and two groups of selectors. Since it is too expensive to make each PU include a divider, and it is observed that only one row of processing units PU in the processing unit PU array needs to perform division operations at the same time, this design designs a group of N×1 dividers outside the processing unit PU array to complete the division operation in the matrix inversion process. The two groups of selectors are used to generate the two inputs of the divider, namely the divisor and the dividend. The first group of selectors is used to generate the divisor of the divider array, and its size is N×1. Each selector has N inputs and can be replaced by a log2(N) level two-to-one selector. Row_sel is the control signal of the selector, and this group of selectors shares the same row_sel signal. The second group of selectors is used to generate the dividend of the divider array. It only contains one N-input selector, and the N inputs are the broadcast inputs X for each row of the processing unit PU array. in This selector and the first group of selectors share the same row_sel signal. The N dividers share the same dividend.

[0090] Memory:

[0091] The memory is divided into two parts: LU decomposition intermediate result memory and sub-block intermediate result memory. As mentioned above, during LU decomposition, some sub-blocks cannot obtain the correct X from the block. in Therefore, it is necessary to store this data in advance so that these blocks can obtain the correct input. The block intermediate result memory is used to store the intermediate results between sub-blocks. Except for the last sub-block in each column, which directly outputs the final output, the outputs of other sub-blocks need to be stored as the output Y of the next sub-block. in This memory consists of N FIFOs. The size of the FIFOs directly determines the size of the matrices that the processing unit array can process. For example, for a 200×200 matrix multiplication, the FIFO size must be at least N×(200-N)×2B. fifo_r and fifo_w are the read and write enable signals, respectively. The N FIFOs share the same read and write enable signals.

[0092] Finite State Machine:

[0093] like Figure 3 The state transition diagram is shown in Figure 2. The finite state machine has a total of 8 states: 1 start state for initialization, 1 end state, 2 states for upper and lower triangular matrix inversion, 2 states for matrix multiplication, and 2 states for LU decomposition.

[0094] There are two states for inverting the upper and lower triangular matrices: S1 and S2. In the S1 state, a row in the processing unit PU array performs a division operation, and the other rows perform multiplication and subtraction operations. The number of rows performing the division operation is controlled by div_sel; in the S2 state, each row of the processing unit PU array performs a multiplication and subtraction operation. The state transition of each sub-block is: S1→S2→S1→S2→...→S1→S2→S2→...→S2→S2→S2, that is, S1 and S2 alternate until all PUs have performed one division operation, and then the S2 state is cycled several times. The number of loops, loop_num, is related to the size of the matrix, matrix_size, and is expressed by the following formula:

[0095] loop_num=(matrix_size-N×(block_row_idx+1))

[0096] After each block loop is completed, N×N elements of the output data are the elements of the final inverse matrix, and the remaining data are input into the next block for iteration.

[0097] Matrix multiplication: The first state is to input the right matrix into the processing unit PU array in parallel by row and store it in the reg of the processing unit PU; the second state is to input the left matrix into the processing unit PU array in parallel by row for multiplication and addition. The state transition of each block is: S4→S4→...→S4→S4→S5→S5→...→S5→S5→S5. After S4 loops 2N-1 times, the S5 state loops loop_num times. The number of loops loop_num is related to the number of columns of the left matrix matrix_a_col_size, which is expressed by the following formula:

[0098] loop_num=(matrix_a_col_size-N×(block_row_idx+1))

[0099] LU decomposition has two states, S6 and S7. In the S6 state, one row in the processing unit PU array performs a division operation, and the other rows perform multiplication and subtraction operations. In the S7 state, each row of the processing unit PU array performs a multiplication and subtraction operation. The state transition of each sub-block is: S6→S7→S6→S7→...→S6→S7→S6→...→S7→S7→S7, that is, S6 and S7 alternate until all processing units PU have performed one division operation, and then the S7 state loops loop_num times. The number of loops loop_num is related to the size of the matrix matrix_size, which is expressed by the following formula:

[0100] loop_num=(matrix_size-N×(block_row_idx+1))

[0101] Example

[0102] This embodiment provides a device for implementing a neural network training algorithm, such as the Levenberg-Marquardt algorithm, which involves both matrix multiplication and matrix inversion. The device is applied to a neural network training accelerator. The device includes a processing unit (PU) array, a divider array, a memory, and a finite state machine.

[0103] The processing unit PU array is used to perform matrix multiplication and matrix inversion operations in the Levenberg-Marquardt neural network training algorithm;

[0104] The size of the processing unit PU array is arbitrary, so the size of the PU array can be adjusted according to the existing resources of the neural network training accelerator without affecting the implementation of matrix multiplication and matrix inversion operations.

[0105] Given an arbitrary-sized processing unit (PU) array, the present invention can perform matrix multiplication and inversion on matrices of any size. When a neural network training accelerator is training a neural network with 200 training parameters (including neural network weights and biases), the Levenberg-Marquardt algorithm requires the accelerator to compute the inverse of a 200×200 matrix. Large-scale neural networks can have tens of thousands of training parameters, so the present invention can utilize PU arrays of any size to perform matrix inversion and matrix multiplication on matrices of any size, demonstrating its significant practicality.

[0106] The acceleration device provided in this embodiment can reuse the same piece of hardware to implement matrix inversion and matrix multiplication operations. For a neural network training accelerator with one matrix multiplication module and one matrix inversion module, when the accelerator performs a matrix multiplication operation, the matrix inversion module is idle, and when it is used to perform a matrix inversion operation, the matrix multiplication module is idle, so the hardware utilization rate is relatively low. However, when the neural network training accelerator using the present invention performs matrix multiplication and matrix inversion operations, the processing unit PU array is in a working state, thus not only greatly reducing the consumption of hardware resources (for example, it can save CPU or GPU hardware resources), but also improving the resource utilization rate of the neural network training accelerator.

[0107] The present invention provides a processor for matrix inversion and matrix multiplication. There are numerous methods and approaches for implementing this technical solution. The above is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A processor for matrix inversion and matrix multiplication, characterized in that It includes a processing unit PU array, a divider array, a memory, and a finite state machine; The processing unit PU array is used to perform matrix multiplication operations and matrix inversion operations; When the processing unit PU array performs matrix multiplication operations and matrix inversion operations, the matrix is first divided into more than two sub-blocks, and the spatial arrangement of the sub-blocks is two-dimensional; the numbering of the sub-blocks is set as follows: When the processing unit PU array performs the following real matrix multiplication operation C = A × B, the total number of rows BLOCK_ROW_NUM, the total number of columns BLOCK_COL_NUM of the sub-blocks are related to the number of rows and columns of the right matrix B: Among them, R m×n represents the m×n real number space; a i,p represents the element in the i-th row and p-th column of matrix A; b p,j represents the element in the pth row and jth column of matrix B; c i,j Represents the element in the i-th row and j-th column of the matrix C; The total number of rows BLOCK_ROW_NUM, the total number of columns BLOCK_COL_NUM of the sub-blocks are related to the number of rows and columns of the right matrix B, and are expressed by the following formula: BLOCK_ROW_NUM = n / N, BLOCK_ROW_NUM = k / N Where, N is the number of rows or columns of the processing unit PU array; When the processing unit PU array performs matrix inversion operation D=E -1 When , the total number of rows BLOCK_ROW_NUM and the total number of columns BLOCK_COL_NUM of the sub-blocks are related to the number of rows of the original matrix E, which can be expressed by the following formula: BLOCK_ROW_NUM = BLOCK_ROW_NUM = m / N Among them, E and D are square matrices respectively, E∈R m×m , D∈R m×m ; The processing unit PU array includes an N×N systolic array PU ij , 1≤i,j≤N; The systolic array PU ij It is pulsating in the vertical direction and broadcasting in the horizontal direction; The processing unit PU array includes a processing unit PU, en is an enable signal of the processing unit PU, X in and Y in There are two data input ports; There is a register reg inside the processing unit PU. When the processing unit PU array performs a matrix multiplication operation, the register reg stores an element b of the right matrix B. i,j , 1≤i≤n,1≤j≤k, the data Y output by the processing unit PU out Z1 and data input port Y in The sum of the input data, Z1 refers to the data input port X in The product X of the input data and the data stored in register reg in ·reg, corresponding to formula (1), abbreviated as multiplication and addition; When the processing unit PU array performs a matrix inversion operation, there are two cases. In the first case, the register reg stores an output data of the divider, and the processing unit PU inputs the data to port Y. in The input data is output after subtracting Z2, where Z2 refers to the data input port X in The product of the input data and the data stored in register reg corresponds to formula (2), which is simply expressed as multiplication and subtraction; in the second case, the processing unit PU inputs the data into port Y in The input data is stored in register reg and the output data is input to port Y at the same time. in The input data corresponds to formula (3), which is simply referred to as transmission; the selection signal sel is used to select PU ij What needs to be done: Y out =Y in +X in ·reg; (1) Y out =Y in -X in ·reg; (2) AND out =And in ,reg=Y in ; (3) Among them, when the matrix is multiplied, PU ij Execute operation (1), the columns of the right matrix are pre-stored in the rows of the processing unit PU array, and the elements of the left matrix A are sequentially stored one by one from the X of the processing unit PU array. in Port parallel input.

2. A processor for matrix inversion and matrix multiplication according to claim 1, characterized in that: The sub-block currently processed by the processing unit PU array is identified by the row number block_row_idx and column number block_col_idx of the sub-block; the iteration of the sub-blocks is carried out in the order from top to bottom, from left to right, with columns prior to rows. The first column of sub-blocks is iterated first, and the sub-blocks in each column are in the order from top to bottom, then the second column, until the iteration of the last column is completed; for the sub-blocks in the same column, the output of the upper row is the input of the lower row, and the output of the sub-block in the last row is the final output.

3. A processor for matrix inversion and matrix multiplication according to claim 2, characterized in that: The processor implements matrix inversion based on LU upper and lower triangular decomposition and upper and lower triangular matrix inversion algorithms. Among them, the LU upper and lower triangular decomposition is based on the right-looking algorithm. When performing LU upper and lower triangular decomposition, the processing unit PU executes operation (2) or (3), and the division in (3) is implemented by an external divider array; from the perspective of the column j to be updated, 1 ≤ j ≤ N, when the column j is updated and becomes an updated column, it is to start from the updated column 1 and accept the data update processing of the updated column k to the column j to be updated from left to right, 1 ≤ k < j, and then the elements below the pivot of the column j are divided by the pivot, and the column j becomes an updated column. Therefore, when the processing unit PU array performs LU upper and lower triangular decomposition, it is a pipeline of N levels, and one row of the array stores one updated column, and the columns to be updated are input from above the array in turn. The specific implementation steps include: In the filling pipeline stage, the first column of the matrix enters the processing unit PU array in parallel as the column to be updated 1. Since there is no updated column before the first column, the first row processing unit PU performs operation (3) on the first column and stores the result in the first row processing unit PU, which becomes the updated column. The data of the column to be updated that flows through the first column is updated, that is, operation (2); then the second column of the matrix enters the processing unit PU array as the column to be updated 2. The first row processing unit PU updates the data of the column to be updated 2. When the data reaches the second row processing unit PU, the column to be updated 2 performs operation (3), and the result is stored in the second row processing unit PU, which becomes the updated column, until all the processing units PU are filled; in the pipeline stage, the column to be updated k passes through the PU ij When 1≤i<k, 1≤j≤N, it will be processed by the i-th updated column; the filling pipeline and the pipeline stage are performed alternately, and N updated columns are obtained in each iteration. After two or more iterations, all updated columns are finally obtained; The upper and lower triangular matrix inversion algorithm is based on Gaussian elimination method. When the upper and lower triangular matrices are inverted, the processing unit PU performs operation (2) or (3); the upper triangular matrix is obtained from the X of the PU array by row. in The ports are input in parallel, and the latter row is delayed by one cycle compared to the former row. For the lower triangular matrix, it is first transposed into an upper triangular matrix, and after inversion is obtained by Gaussian elimination, it is transposed again to obtain the output of the original lower triangular matrix.

4. A processor for matrix inversion and matrix multiplication according to claim 3, characterized in that: The function of the divider array is to provide a data input port Y for a row of processing units PU. in The input data performs a division operation; the divider array includes an N×1 divider and two groups of selectors; The two groups of selectors are respectively used to generate the two inputs of the divider, namely the divisor and the dividend; the first group of selectors is used to generate the divisor of the divider array. The size of the first group of selectors is N × 1, and each selector in the first group of selectors has N inputs and can be replaced by a log2(N)-level two-to-one selector. row_sel is the control signal of the first group of selectors, and the first group of selectors shares the same row_sel; The second group of selectors is used to generate the dividend of the divider array. The second group of selectors contains only one N-input selector. The N inputs are the broadcast inputs X of each row of the processing unit PU array. in , the second group of selectors and the first group of selectors share the same row_sel, and the N dividers share the same dividend.

5. A processor for matrix inversion and matrix multiplication according to claim 4, characterized in that: The memory includes an LU decomposition intermediate result memory and a sub-block intermediate result memory; The LU decomposition intermediate result memory is used to store the LU decomposition data input port X in Input data; The sub-block intermediate result memory is used to store the intermediate results between sub-blocks. Except for the last sub-block in each column, which directly outputs the result as the final output, the outputs of other sub-blocks need to be stored as the data input port Y of the next sub-block. in Input data; the memory includes N FIFOs, fifo_r and fifo_w are read enable signals and write enable signals respectively, and the N FIFOs share the same read enable signal and write enable signal.

6. A processor for matrix inversion and matrix multiplication according to claim 5, characterized in that: The processing unit PU array also includes three groups of selectors, which are used to generate PU ij Data input port X in and Y in Input data, 1≤i,j≤N; The first set of selectors is used to generate PU ij Data input port X in Input data; the processing unit PU array is broadcast in the horizontal direction, and each row of processing units PU shares the same X in The size of the first group of selectors is N×1. Each selector in the first group of selectors has two inputs, where the first input is the custom sequence data_in2, and the second input is related to the number of rows corresponding to the selector. The second input of the i-th row selector in the first group of selectors is PU ii Input Y in , 1≤i≤N; mode_sel is the control signal of the selector, the first group of selectors share the same mode_sel; the output of the i-th selector in the first group of selectors is the corresponding PU ij Input X in ; The second set of selectors is used to generate PU ij Data input port Y in Input data; The processing unit PU array is pulsating in the vertical direction, so each processing unit PU needs a selector. The second group of selectors is N×N. Each selector in the second group of selectors has two inputs. For the selectors from the 2nd row to the Nth row, the first input is the output of the jth divider, and the second input is the PU (i-1)j The output of , 2≤i≤N,1≤j≤N; the first input of the first row selector in the second group of selectors is the output of the j-th divider, and the second input is the output of the j-th selector in the third group of selectors; div_sel0[i] and div_sel1 are the control signals of the second group of selectors; For the i-th row selector in the second group of selectors, columns 1 to 1 share the same control signal div_sel0[i], 1≤i≤N; For the i-th row selector in the second group of selectors, the i+1th column to the Nth column share the same signal div_sel1[i], 1≤i≤N-1; The output of the selector in the i-th row and j-th column of the second group of selectors is the input Y of the corresponding PUij in ; The third group of selectors is used to adapt the input of the first row of different sub-blocks. In the block division, for the first block of each column, the input of the first row processing unit PU is the custom sequence data_in1, while the input of the first row processing unit PU of other blocks is the output of the memory; the third group of selectors is only connected to the data input port Y of the first row processing unit PU in The third group of selectors is 1×N. Each selector in the third group has two inputs. The first input is the custom sequence data_in1, and the second input is related to the number of columns of the selector. The second input of the j-th column selector in the third group of selectors is the output of the j-th FIFO, 1≤j≤N; data_sel is the control signal of the selector, and the third group of selectors shares the same data_sel.

7. A processor for matrix inversion and matrix multiplication according to claim 6, characterized in that: The finite state machine includes a total of 8 states, namely: 1 start state for initialization, waiting for matrix inversion or matrix multiplication instructions, 1 end state for outputting results, 2 states for upper and lower triangular matrix inversion, 2 states for matrix multiplication, and 2 states for LU upper and lower triangular decomposition; The start state is used for initialization and waiting for a matrix inversion or matrix multiplication instruction; The end state is used to output the result; The inversion of the upper and lower triangular matrices has two states, S1 and S2. In the S1 state, a row in the processing unit PU array performs a division operation, and the other rows perform multiplication and subtraction operations. The number of rows performing the division operation is controlled by div_sel. In the S2 state, each row of the processing unit PU array performs a multiplication and subtraction operation. The state transition of each sub-block is: S1→S2→S1→S2→...→S1→S2→S2→...→S2→S2→S2, that is, S1 and S2 alternate until all processing units PU have performed one division operation, and then the S2 state loops loop_num times. The number of loops loop_num is related to the size of the matrix matrix_size, which is expressed by the following formula: loop_num=(matrix_size-N×(block_row_idx+1)) After each block loop, the output data contains N×N elements of the final inverse matrix, and the remaining data is input to the next block for iteration; The first state of matrix multiplication is to input the right matrix into the processing unit PU array in parallel by row and store it in the reg of the processing unit PU; the second state is to input the left matrix into the processing unit PU array in parallel by row for multiplication and addition. The state transition of each block is: S4→S4→...→S4→S4→S5→S5→...→S5→S5→S5. After S4 is cycled 2N-1 times, the S5 state is cycled loop_num times. The number of loops loop_num is related to the number of columns of the left matrix matrix_a_col_size, which is expressed by the following formula: loop_num=(matrix_a_col_size-N×(block_row_idx+1)) The LU upper and lower triangular decomposition has two states, S6 and S7. In the S6 state, one row in the processing unit PU array performs a division operation, and the other rows perform multiplication and subtraction operations. In the S7 state, each row of the processing unit PU array performs a multiplication and subtraction operation. The state transition of each sub-block is: S6→S7→S6→S7→...→S6→S7→S6→...→S7→S7→S7, that is, S6 and S7 alternate until all processing units PU have performed one division operation, and then the S7 state loops loop_num times. The number of loops loop_num is related to the size of the matrix matrix_size, which is expressed by the following formula: loop_num=(matrix_size-N×(block_row_idx+1)).

Images