Sparse matrix accelerator based on block-based packing algorithm

Abstract

The application provides a sparse matrix accelerator based on a block-based packing algorithm, comprising an algorithm preprocessing module, a calculation module and a sorting module. The algorithm preprocessing module can obtain a sparse matrix comprising a flow matrix and a stationary matrix, block the flow matrix and the stationary matrix according to a preset numerical value, then perform shift and merge processing on the first non-zero element, the second non-zero element and the zero element in the matrix respectively to obtain a dense flow matrix and a dense stationary matrix, and save the address of the first non-zero element before shifting and the address of the second non-zero element before and after shifting. The calculation module can perform multiplication and addition calculation on the dense flow matrix and the dense stationary matrix to obtain a partial sum result and a first address result of the partial sum result. The sorting module can sort the partial sum result according to the first address result to obtain a result matrix. The above accelerator can improve the density of the matrix, at the same time, correct column addition can be realized, and the calculation efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sparse matrix computation, and in particular to a sparse matrix accelerator based on a block-based packing algorithm. BACKGROUND

[0002] Sparse General Matrix Multiply (SpGEMM) is an important computing kernel that is ubiquitous in Deep Neural Networks (DNN) models. When artificial intelligence applications run on GPUs and accelerators, SpGEMM affects the performance of the applications, and therefore the sparse matrix needs to be processed. Sparse input matrices can be compressed into denser input matrices by offline preprocessing padding algorithms, and then they are mapped onto a contracted array to perform SpGEMM to improve execution efficiency. Each offline preprocessing padding algorithm designs a dedicated hardware to perform computation, such as a collision-aware algorithm and a dedicated hardware structure, and a block-based SorPack packing algorithm and a dedicated hardware structure.

[0003] The collision-aware algorithm selects sparse column combinations to form denser columns without changing the row positions of non-zero elements, but in deep neural network applications, column merging for large matrices is very time-consuming, and the data flow in the hardware is designed for Sparse Matrix Vector Multiply (SpMV) and is not optimized for SpGEMM. The block-based SorPack packing algorithm divides the matrix into a streaming matrix and a stationary matrix to improve the utilization of processing units, then divides the streaming matrix and the stationary matrix into blocks, and rearranges the data in the data blocks of the stationary matrix in descending order according to the number of non-zero values in each column, while rearranging the data in the data blocks of the streaming matrix in descending order according to the number of non-zero values in each row, and then performing insertion and merging of the data blocks. To ensure the correctness of the computation, the algorithm also provides two auxiliary matrices to mark the rows and columns of the packed matrix, but the packing algorithm causes the accumulation values of the same column to be calculated in different cycles, increasing the complexity and execution performance of the hardware. At the same time, the packing algorithm does not tolerate collisions, so the actual packed matrix density is not the highest, and because of the descending arrangement operation, it is also very time-consuming when packing and merging large sparse matrices, thereby reducing the computational efficiency. SUMMARY

[0004] The present application provides a sparse matrix accelerator based on a block-based packing algorithm to solve the problem of low computational efficiency of sparse matrices.

[0005] The first aspect of the present application provides a sparse matrix accelerator based on a block-based packing algorithm, comprising:

[0006] An algorithm preprocessing module configured to:

[0007] Obtain a sparse matrix from an external memory; the sparse matrix includes a flow matrix and a stationary matrix, the flow matrix is a matrix for flow processing, and the stationary matrix is a matrix for fixed storage;

[0008] Perform block processing on the flow matrix and the stationary matrix according to a preset value respectively to obtain a plurality of block flow matrices and a plurality of block stationary matrices; elements in the block flow matrix include first non-zero elements and zero elements, and elements in the block stationary matrix include second non-zero elements and zero elements;

[0009] Perform shift processing on the first non-zero elements and the second non-zero elements respectively, and perform merging processing on the zero elements to obtain a dense flow matrix and a dense stationary matrix;

[0010] Generate a flow auxiliary matrix according to the dense flow matrix to save the addresses of the first non-zero elements before shift processing, and generate a stationary auxiliary matrix and a first auxiliary bitmap according to the dense stationary matrix to save the addresses of the second non-zero elements before shift processing and the addresses of the second non-zero elements after shift processing;

[0011] Perform a bitwise AND operation on elements in the flow auxiliary matrix to obtain a second auxiliary bitmap, and perform a bitwise AND operation on elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap;

[0012] A calculation module configured to:

[0013] Perform multiplication operation and addition calculation on the dense flow matrix and the dense stationary matrix according to the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap to obtain a partial sum result and a first address result of the partial sum result; the first address result includes a row address and a column address;

[0014] An ordering module configured to:

[0015] Order the partial sum result according to the first address result to obtain a result matrix.

[0016] Optionally, the block processing on the flow matrix and the stationary matrix according to a preset value respectively to obtain a plurality of block flow matrices and a plurality of block stationary matrices includes:

[0017] Segmenting the flow matrix in the direction of columns to obtain a plurality of block flow matrices;

[0018] The stationary matrix is segmented in a row direction to obtain a plurality of the segmented stationary matrices.

[0019] Optionally, the separately performing the shift processing on the first non-zero element and the second non-zero element, and performing the merging processing on the zero element to obtain the dense flow matrix and the dense stationary matrix comprises:

[0020] performing non-conflict row shift processing on the first non-zero element to obtain a shifted flow matrix; the non-conflict row shift processing is to shift the first non-zero element in a row direction;

[0021] merging zero rows in the shifted flow matrix to obtain a dense flow matrix, the zero row being a row with all zero elements;

[0022] performing non-conflict column shift processing on the second non-zero element to obtain a first shifted stationary matrix; the non-conflict column shift processing is to shift the second non-zero element in a column direction;

[0023] performing conflict shift processing on the second non-zero element in the first shifted stationary matrix to obtain a second shifted stationary matrix; the conflict shift processing is to simultaneously shift the second non-zero element in the first shifted stationary matrix in a row direction and a column direction;

[0024] merging zero columns in the second shifted stationary matrix to obtain a dense stationary matrix, the zero column being a column with all zero elements.

[0025] Optionally, the dense flow matrix comprises at least one first row vector, and the dense stationary matrix comprises at least one first column vector; a plurality of the calculation modules are connected in a mesh shape, arranged in rows and columns, and the calculation modules in a same row are configured to:

[0026] performing multiplication and addition calculation on a same first row vector of a same dense flow matrix and a different first column vector of a same dense stationary matrix by using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap to obtain a partial sum result and a first address result of the partial sum result;

[0027] the calculation modules in a same column are configured to:

[0028] performing multiplication and addition calculation on a next dense flow matrix and a next dense stationary matrix by using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap to obtain a partial sum result and a first address result of the partial sum result;

[0029] The calculation module of the same row is further configured to:

[0030] After the preset period, the next first row vector of the same dense flow matrix and the different first column vector of the same dense stationary matrix are multiplied and added using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap, and the third auxiliary bitmap, to obtain a partial sum result and a first address result of the partial sum result.

[0031] Optionally, the flow auxiliary matrix includes at least one second row vector, and the stationary auxiliary matrix includes at least one second column vector; an element in the flow auxiliary matrix includes a first element, which is a first address of the first non-zero element in the block flow matrix before shift processing, and an element in the stationary auxiliary matrix includes a second element, which is a second address of the second non-zero element in the block stationary matrix before shift processing; a length of the first auxiliary bitmap is an integer multiple of a value obtained by subtracting one from the preset value, an element in the first auxiliary bitmap includes a third element and a fourth element, the third element is a third address of the second non-zero element in the block stationary matrix before the conflict shift processing, and the fourth element is a fourth address of the second non-zero element in the dense stationary matrix after the conflict shift processing, the first address is a row address, the second address is a column address, and the third address and the fourth address are row addresses.

[0032] Optionally, the calculation module includes:

[0033] The shift adjustment module is configured to:

[0034] adjust positions of the first non-zero elements in the first row vector according to the first address, the second address, the third address, and the fourth address, so that the first non-zero elements in the first row vector and the second non-zero elements in the first column vector one-to-one correspond to addresses before shift processing;

[0035] The multiplication module is configured to:

[0036] respectively perform multiplication processing on the first non-zero elements in the first row vector and the corresponding second non-zero elements, to obtain a dot product result and a second address result of the dot product result, the second address result including a row address and a column address;

[0037] The algorithm preprocessing module is further configured to:

[0038] performing a bitwise AND operation on the first elements in the second row vectors two by two to obtain a second auxiliary bit map;

[0039] performing a bitwise AND operation on the second elements in the second column vectors two by two to obtain a third auxiliary bit map;

[0040] a sum tree module, configured to:

[0041] performing an AND operation on the second auxiliary bit map and the third auxiliary bit map to obtain a sum result;

[0042] performing a sum calculation on the dot product result by a sum tree according to the sum result to obtain a partial sum result and a first address result of the partial sum result.

[0043] Optionally, the multiplication module comprises:

[0044] a storage submodule, configured to:

[0045] save the second non-zero element and the second address in the dense stationary matrix;

[0046] a multiplier submodule, configured to:

[0047] obtain the first non-zero element and the first address in the dense flow matrix, and obtain the second non-zero element and the second address in the storage submodule;

[0048] perform a multiplication processing on the first non-zero element and the second non-zero element to obtain a dot product result and a second address result of the dot product result.

[0049] Optionally, the sum tree module comprises:

[0050] a selection submodule, configured to:

[0051] select a sum mode in a sum tree according to the sum result, so as to perform a sum calculation on the dot product result by the sum mode.

[0052] Optionally, further comprising:

[0053] a first storage module, configured to:

[0054] obtain and save the dense flow matrix, the dense stationary matrix, the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bit map, the second auxiliary bit map and the third auxiliary bit map generated by the algorithm preprocessing module;

[0055] a second storage module, configured to:

[0056] acquire and save the partial sum result and the first address result of the partial sum result.

[0057] Optionally, the calculation module further comprises:

[0058] The output module is configured to:

[0059] acquire the partial sum result and the first address result of the partial sum result;

[0060] output the partial sum result and the first address result of the partial sum result to the second storage module.

[0061] The accelerator provided by the application is a sparse matrix accelerator based on a block-based packing algorithm, comprising an algorithm preprocessing module, a calculation module and a sorting module. The algorithm preprocessing module is used to acquire a sparse matrix comprising a flow matrix and a stationary matrix, and perform block processing on the flow matrix and the stationary matrix according to preset numerical values respectively, so as to obtain a block flow matrix and a block stationary matrix; perform shift and merge processing on a first non-zero element in the block flow matrix and a second non-zero element in the block stationary matrix and a zero element in the matrix respectively, so as to obtain a dense flow matrix and a dense stationary matrix; generate a flow auxiliary matrix to save the address of the first non-zero element before shifting, and generate a stationary auxiliary matrix and a first auxiliary bitmap to save the address of the second non-zero element before and after shifting; perform a bitwise AND operation on the elements in the flow auxiliary matrix to obtain a second auxiliary bitmap, and perform a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap. The calculation module is used to perform multiplication and addition calculation on the dense flow matrix and the dense stationary matrix according to the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, so as to obtain a partial sum result and a first address result of the partial sum result. The sorting module is used to sort the partial sum result according to the first address result, so as to obtain a result matrix. The accelerator provided by the application can improve the density of the matrix and the calculation efficiency by performing block processing on the sparse matrix and performing shift and merge processing by the packing algorithm. Through the flow auxiliary matrix, the stationary auxiliary matrix and the three auxiliary bitmaps, correct column-wise addition can be realized, so that the accelerator is faster and more efficient, and the problem of low calculation efficiency of the sparse matrix can be solved. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, other drawings can also be obtained by those skilled in the art without any creative effort on the premise of not paying any creative effort.

[0063] Figure 1 A schematic diagram of the sparse general matrix multiplication provided by the application is shown in the figure;

[0064] Figure 2 A sparse matrix accelerator structure schematic diagram provided by the block-based packing algorithm of the present application;

[0065] Figure 3 A schematic diagram of splitting the flow matrix and the stationary matrix when the preset value is equal to 4 provided by the present application;

[0066] Figure 4 A schematic diagram of obtaining the first shifted stationary matrix when the preset value is equal to 4 provided by the present application;

[0067] Figure 5 A schematic diagram of obtaining the second shifted stationary matrix when the preset value is equal to 4 provided by the present application;

[0068] Figure 6 A schematic diagram of generating the flow auxiliary matrix and the stationary auxiliary matrix provided by the present application;

[0069] Figure 7 A way of summing the elements in the second row vector or the second column vector when the preset value is equal to 4 provided by the present application;

[0070] Figure 8 A schematic diagram of the calculation process of the addition tree module when the preset value is equal to 4 provided by the present application;

[0071] Figure 9 A schematic diagram of the calculation module structure when the preset value is equal to 4 provided by the present application;

[0072] Figure 10 A schematic diagram of the structure of the multiplication module provided by the present application;

[0073] Figure 11 A schematic diagram of the sparse matrix accelerator structure provided by the present application, which contains multiple calculation modules. DETAILED DESCRIPTION

[0074] The embodiments will be described in detail below with reference to the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of systems and methods consistent with some aspects of the present application as detailed in the appended claims.

[0075] For general matrix multiplication (GEMM), assuming that A is an m x p matrix and B is a p x n matrix, then the m x n matrix C is the product of the matrices A and B, denoted as Z = XY, where the i-th row, j-th column element in the matrix Z can be expressed as:

[0076]

[0077] Figure 1 For the example of SpGEMM, where Figure 1 (a) in is a 6x8 sparse matrix X, Figure 1 (b) in is an 8x8 sparse matrix Y, Figure 1 (c) is the product of two matrices. The gray blocks in each matrix represent zero elements, and other blocks marked with capital letters represent non-zero elements. The non-zero elements in the same row of the X matrix are represented using the same letter, and the non-zero elements in the same column of the Y matrix are represented using the same letter.

[0078] When calculating the result of the first row and the first column of the result matrix using formula (1), as shown in Figure 1 (a), the cases of 0x0, 0xnon-zero value, and non-zero valuex0 will occur. Since 0 multiplied by any number is equal to 0, the addition of zero elements is unnecessary, which will affect the execution efficiency of the hardware, and a large number of zero elements will also occupy a lot of storage space, thereby leading to low matrix calculation efficiency and high hardware requirements.

[0079] To solve the problem of low efficiency of sparse matrix calculation, an embodiment of the present application provides a sparse matrix accelerator based on a block-based packing algorithm, as shown in Figure 2 The accelerator comprises an algorithm preprocessing module, a calculation module, and a sorting module.

[0080] The algorithm preprocessing module can be an algorithm model stored in the CPU, and the algorithm preprocessing module is configured to:

[0081] S210: Obtain a sparse matrix from an external memory.

[0082] In the embodiment of the present application, the input sparse matrix is defined as a stream matrix and a stationary matrix, respectively, where the stream matrix is a matrix for stream processing, and the stationary matrix is a matrix for fixed storage, that is, in subsequent calculations, the elements in the stream matrix flow between the calculation modules, and the elements in the stationary matrix are fixed in the calculation modules.

[0083] S220: Perform block processing on the stream matrix and the stationary matrix according to a preset value to obtain a plurality of block stream matrices and a plurality of block stationary matrices.

[0084] The block processing refers to dividing the stream matrix or the stationary matrix along the row or column direction according to the preset value to obtain a plurality of blocks of the same size, wherein the block with a row number or a column number less than the preset value is also separately regarded as a matrix. The elements in the block stream matrix include first non-zero elements and zero elements, and the elements in the block stationary matrix include second non-zero elements and zero elements.

[0085] S230: performing shift processing on the first non-zero element and the second non-zero element respectively, and performing merging processing on the zero element, to obtain a dense flow matrix and a dense stationary matrix.

[0086] Compared with other packing algorithms, the time occupied is longer and the storage space is larger when packing according to the number of non-zero elements in the row or column. By performing shift and merging processing on the sparse matrix through the packing algorithm after blocking the sparse matrix, the sparse matrix can be made dense, the density of the matrix can be improved, and thus the storage space can be reduced and the calculation efficiency can be improved.

[0087] S240: generating a flow auxiliary matrix according to the dense flow matrix to save the address of the first non-zero element before shift processing, and generating a stationary auxiliary matrix and a first auxiliary bitmap according to the dense stationary matrix to save the address of the second non-zero element before shift processing and the address of the second non-zero element after shift processing.

[0088] Since the one-to-one correspondence between the elements after shift processing is broken, the addresses of the first non-zero element and the second non-zero element before shift processing and the addresses after shift processing need to be recorded. The address of the first non-zero element before shift processing is the address of the first non-zero element in the blocked flow matrix, and the address of the second non-zero element before shift processing is the address of the second non-zero element in the blocked stationary matrix.

[0089] S250: performing a bitwise AND operation on the elements in the flow auxiliary matrix to obtain a second auxiliary bitmap, and performing a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap.

[0090] The obtained second auxiliary bitmap includes all possible addition modes of the first non-zero element in the dense flow matrix, and the third auxiliary bitmap includes all possible addition modes of the second non-zero element in the dense stationary matrix, so that correct addition results can be obtained in the following addition calculation.

[0091] The calculation module is configured to perform multiplication and addition calculation on the dense flow matrix and the dense stationary matrix according to the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, to obtain a partial sum result and a first address result of the partial sum result. The first address result includes a row address and a column address.

[0092] The sorting module is configured to sort the partial sum result according to the first address result to obtain a result matrix. Since the sorting module can be obtained by using the existing technology, it will not be described in detail in the present application.

[0093] The accelerator provided in the application can improve the density of the matrix by block processing of the sparse matrix and displacement and merging processing by the packing algorithm, can reduce the number of handling times in calculation, and improve the calculation efficiency. The address before displacement of the non-zero element for non-conflict displacement is saved by the flow auxiliary matrix and the stationary auxiliary matrix, the address before and after displacement of the non-zero element for conflict displacement is saved by the first auxiliary bitmap, and the correct addition result is obtained by the second auxiliary bitmap and the third auxiliary bitmap, so that the correct column addition can be realized, and the speed and efficiency of the accelerator are higher.

[0094] In some embodiments, performing block processing on the flow matrix and the stationary matrix according to a preset value respectively to obtain a plurality of block flow matrices and a plurality of block stationary matrices comprises: dividing the flow matrix in the column direction to obtain a plurality of block flow matrices; and dividing the stationary matrix in the row direction to obtain a plurality of block stationary matrices. See Figure 3 , Figure 3 For an example of dividing the flow matrix and the stationary matrix, Figure 3 (a) in the figure is a 6-row 8-column flow matrix, Figure 3 (b) in the figure is an 8-row 8-column stationary matrix. The flow matrix in the figure is divided in the column direction, Figure 3 every 4 columns into a small matrix, and the two obtained matrices are the block flow matrices, that is, (c) in the figure. Figure 3 The stationary matrix in the figure is divided in the row direction, Figure 3 every 4 rows into a small matrix, and the two obtained matrices are the block stationary matrices, that is, (d) in the figure. Figure 3

[0095] The block processing in the application is different from other block processing methods of any shape, and only one direction of the sparse matrix is divided. It can be understood that, Figure 3 the preset value of the block processing in the figure is equal to 4, and the fewer the number of elements in a row / column, the fewer the merging cases. By setting the preset value to 4, it is beneficial to displacement and merging processing, and can also reduce the difficulty of the addition tree. In the following description, the case of the preset value being 4 is exemplified. In some embodiments, the preset value can also be other values less than 6.

[0096] In some embodiments, performing displacement processing on the first non-zero element and the second non-zero element respectively, and performing merging processing on the zero element to obtain a dense flow matrix and a dense stationary matrix comprises:

[0097] S410: performing non-conflict row displacement processing on the first non-zero element to obtain a displacement flow matrix.

[0098] S420: merging zero rows in the displacement flow matrix to obtain a dense flow matrix.​

[0099] S430: Perform a non-conflicting column shift on the second non-zero element to obtain the first shift-stable matrix.

[0100] S440: Perform conflict shift processing on the second non-zero element in the first shift stationary matrix to obtain the second shift stationary matrix.

[0101] S450: Merge the zero columns in the second shift stationary matrix to obtain a dense stationary matrix. The zero columns are columns with all zero elements.

[0102] It should be noted that non-conflicting row shifting involves shifting the first non-zero element along the row direction, and non-conflicting column shifting involves shifting the second non-zero element along the column direction. The zero row represents a row containing only zero elements, and the zero column represents a column containing only zero elements. Conflicting shifting involves shifting the second non-zero element in the first shift-stable matrix simultaneously along both the row and column directions.

[0103] See Figures 4-5 In the diagram, gray squares without letters represent zero elements, elements with letters represent non-zero elements, and non-zero elements in the same row / column are represented by the same letter. Figure 4 In the middle (a), the block flow matrix is... Figure 4 In (b), the block stationary matrix is ​​given. Figure 4 In (a), the first non-zero element is shifted along the row direction, meaning only the row address of the first non-zero element is changed. The resulting matrix is ​​a shift stream matrix, which is... Figure 4 (c) Merging Figure 4 The dense flow matrix can be obtained by finding the rows in (c) where all elements are zero. Figure 4 In (b), the second non-zero element is shifted along the column direction, meaning only the column address of the second non-zero element is changed. The resulting matrix is ​​the first shift-stationary matrix, which is... Figure 4 (d) in the middle.

[0104] It should be noted that during the non-conflicting shift process, due to the position where the second non-zero element is moved, there may be other non-zero elements present, such as... Figure 4 (d) or Figure 5 The elements at the vertical box in (a) are therefore, in some embodiments, a conflict shift operation needs to be performed on the first shift-stable matrix, and the second non-zero elements are shifted simultaneously in both the row and column directions to obtain the second shift-stable matrix, such as... Figure 5 The process from (a) to (c) is merged. Figure 5 By taking the columns in (c) that are all zero elements, we can obtain a dense stationary matrix.

[0105] It is understood that a dense flow matrix includes at least one first row vector, and a dense stationary matrix includes at least one first column vector. See also Figure 11 Multiple computing modules are connected in a network and set according to rows and columns. The computing modules in the same row are configured to: use the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap, and the third auxiliary bitmap to perform multiplication and summation operations on the same first row vector of the same dense flow matrix and different first column vectors of the same dense stationary matrix to obtain the partial sum result and the first address result of the partial sum result. Then, the results of multiplying the same row of the dense flow matrix with different columns of the dense stationary matrix can be calculated separately.

[0106] The computation module in the same column is configured to perform multiplication and summation operations on the next dense flow matrix and the next dense stationary matrix using the flow auxiliary matrix, stationary auxiliary matrix, first auxiliary bitmap, second auxiliary bitmap and third auxiliary bitmap to obtain the partial sum result and the first address result of the partial sum result, and then the results of multiplying different block flow matrices and block stationary matrices can be calculated respectively.

[0107] The calculation module in the same row is also configured to: after a preset period, use the flow auxiliary matrix, stationary auxiliary matrix, first auxiliary bitmap, second auxiliary bitmap and third auxiliary bitmap to perform multiplication and summation on the next first row vector of the same dense flow matrix and the different first column vectors of the same dense stationary matrix to obtain the partial sum result and the first address result of the partial sum result, and then calculate the result of multiplying the next row of the dense flow matrix with different columns of the dense stationary matrix respectively.

[0108] The flow-aided matrix includes at least one second row vector, and the stationary-aided matrix includes at least one second column vector. Elements in the flow-aided matrix include a first element, which is the first address of the first non-zero element in the block flow matrix before shifting. Elements in the stationary-aided matrix include a second element, which is the second address of the second non-zero element in the block stationary matrix before shifting, where the first address is the row address and the second address is the column address. See also... Figure 6 , Figure 6 (a) and Figure 6 In (b), the numbers 0, 1, 2, 3… represent the row and column addresses of the non-zero elements in the flow matrix and stationary matrix, respectively. Figure 6 (c) and Figure 6 In the example (d), dense flow matrix and dense stationary matrix are respectively examples. Figure 6 (e) and Figure 6 In the matrix (f), the flow-aided matrix and the stationary matrix are respectively. Figure 6 The number in block (e) is the row address of the stream matrix, which is the row address of the first non-zero element in the stream matrix before it was shifted.Figure 6 The number in block (f) is the column address of the second nonzero element in the stationary matrix, that is, the column address of the second nonzero element in the stationary matrix before shifting.

[0109] Since the second nonzero element in the stationary matrix is subjected to conflict shifting processing, the row address and the column address of some second elements may change, therefore, in the embodiment of the application, the row address of the second nonzero element whose row address and column address both change is recorded by a first auxiliary bitmap. The elements in the first auxiliary bitmap include a third element and a fourth element, the third element is the third address of the second nonzero element in the blocked stationary matrix before conflict shifting processing, and the fourth element is the fourth address of the second nonzero element in the dense stationary matrix after conflict shifting processing, the third address and the fourth address are row addresses.

[0110] The length of the first auxiliary bitmap is an integer multiple of a number obtained by subtracting one from a preset value, since the stationary matrix is divided by taking 4 as the preset value, therefore, in conflict shifting processing, the second nonzero element in the same column that needs to be subjected to shifting operation will not exceed 3 at most, therefore, in the embodiment of the application, the length of the first auxiliary bitmap is set to 12 bits, and three shifting signals can be saved at most in every 6 bits, wherein, the first 6 bits are used to save the row address of the second nonzero element before shifting, that is, the third address, and the last 6 bits are used to save the row address of the second nonzero element after shifting, that is, the fourth address. By setting the first auxiliary bitmap of 12 bits, the second nonzero element after shifting can be restored to the position before shifting in calculation, so as to ensure the accuracy of calculation.

[0111] Referring to Figure 9 In some embodiments, the calculation module includes a shifting adjustment module, a multiplication module and an addition tree module. The first nonzero element can be marked as Str_Data, the first address is row_id, the second nonzero element is Sta_Data, and the second address is col_id.

[0112] The shifting adjustment module is configured to adjust the position of the first nonzero element Str_Data in the first row vector according to the first address row_id, the second address col_id, the third address and the fourth address, so that the first nonzero element Str_Data in the first row vector and the second nonzero element Sta_Data in the first column vector correspond one by one according to the addresses before shifting, so as to calculate the product of the first nonzero element Str_Data and the second nonzero element Sta_Data according to the positions before shifting by the multiplication module.

[0113] The number of multiplication modules in each calculation module is equal to a preset value, and the multiplication modules are configured to perform multiplication processing on the first non-zero element Str_Data in the first row vector and the corresponding second non-zero element Sta_Data to obtain a dot product result Mul_data and a second address result row_id&col_id of the dot product result. It can be understood that the second address result includes a row address and a column address.

[0114] Referring to Figure 10 , the multiplication module includes a storage submodule and a multiplier submodule. The storage submodule is configured to save the second non-zero element Sta_Data in the dense stationary matrix and the second address col_id. In some embodiments, the storage submodule can be a register. The multiplier submodule is configured to obtain the first non-zero element Str_Data in the dense flow matrix and the first address row_id, and obtain the second non-zero element Sta_Data in the storage submodule and the second address col_id, and perform multiplication processing on the first non-zero element Str_Data and the second non-zero element Sta_Data to obtain a dot product result Mul_data and a second address result row_id&col_id of the dot product result.

[0115] Since the first non-zero element and the second non-zero element are subjected to shift and merge processing, the obtained dot product result cannot be simply added according to formula (1), and must be selected to obtain the correct addition result, so the second auxiliary bit map and the third auxiliary bit map are needed to ensure the accuracy of the calculation result. In some embodiments, the algorithm preprocessing module is further configured to perform bitwise AND operation on the first elements in the second row vector two by two to obtain a second auxiliary bit map, and perform bitwise AND operation on the second elements in the second column vector two by two to obtain a third auxiliary bit map. When the matrix is divided by 4 as the reference, since the number of elements in the second row vector and the second column vector is 4, there are 14 addition modes in total, referring to Figure 7 , ①, ②, ③, and ④ represent elements in the first row vector or the first column vector, 0 indicates that the addresses of the two elements are different, and 1 indicates that the addresses of the two elements are the same, Figure 7 Each result in Figure 7 is a possible addition mode of the second auxiliary bit map or the third auxiliary bit map. Exemplarily, Figure 7 ①+②+③+④ in indicates that the four input dot product results are sequentially added.

[0116] Referring to Figure 8The addition tree module is configured to perform an AND operation on the second auxiliary bitmap and the third auxiliary bitmap to obtain an addition result, and perform an addition calculation on the dot product result by the addition tree according to the addition result to obtain a partial sum result Data and a first address result row_id & col_id of the partial sum result. Then, the partial sum result Data and the first address result row_id & col_id of the partial sum result can be sent to the output module described below. Since there are only 14 kinds of addition methods when the preset value is equal to 4, the addition tree structure is simple, and the hardware overhead is reduced. In some embodiments, the addition tree module includes a selection sub-module configured to select an addition method in the addition tree according to the addition result, so as to perform an addition calculation on the dot product result by the addition method.

[0117] It should be further pointed out that although the first address result and the second address result are both marked as row_id & col_id in the present application, the first address result and the second address result actually calculated and output are not the same.

[0118] Again referring to Figure 11 In some embodiments, the accelerator further includes a first storage module and a second storage module, which can be SRAM memories. The first storage module is configured to obtain and save the dense flow matrix, the dense stationary matrix, the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap generated by the algorithm preprocessing module. The second storage module is configured to obtain and save the partial sum result and the first address result.

[0119] Again referring to Figure 9 In some embodiments, the calculation module further includes an output module, which can be a router. The output module is configured to obtain the partial sum result and the first address result, and output the partial sum result and the first address result to the second storage module.

[0120] According to the technical scheme, the embodiment of the application provides a sparse matrix accelerator based on a block-based packing algorithm, which comprises an algorithm preprocessing module, a calculation module and a sorting module. The algorithm preprocessing module is used for obtaining a sparse matrix comprising a flow matrix and a stationary matrix, performing block processing on the flow matrix and the stationary matrix according to preset numerical values respectively, so as to obtain a block flow matrix and a block stationary matrix; performing shift and merge processing on a first non-zero element in the block flow matrix, a second non-zero element in the block stationary matrix and a zero element in the matrix respectively, so as to obtain a dense flow matrix and a dense stationary matrix; generating a flow auxiliary matrix to save the address of the first non-zero element before the shift, and generating a stationary auxiliary matrix and a first auxiliary bitmap to save the address of the second non-zero element before the shift; performing a bitwise AND operation on the elements in the flow auxiliary matrix to obtain a second auxiliary bitmap, and performing a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap. The calculation module is used for performing multiplication operation and addition calculation on the dense flow matrix and the dense stationary matrix according to the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, so as to obtain a partial sum result and a first address result of the partial sum result. The sorting module is used for sorting the partial sum result according to the first address result, so as to obtain a result matrix. The accelerator provided by the application can improve the density of the matrix by block processing on the sparse matrix and shift and merge processing by the packing algorithm, and further improve the calculation efficiency. Through the flow auxiliary matrix, the stationary auxiliary matrix and the three auxiliary bitmaps, correct column-wise addition can be realized, so that the accelerator is faster and more efficient, and the problem of low calculation efficiency of the sparse matrix can be solved.

[0121] The similar parts among the embodiments provided by the application can be referred to each other, the specific embodiments provided above are only several examples under the general concept of the application, and do not limit the protection scope of the application. Any other embodiments extended by the person skilled in the art without creative labor according to the application scheme are within the protection scope of the application.

Claims

1. A sparse matrix accelerator based on a block-based packing algorithm, characterized in that, The method comprises the following steps: An algorithm preprocessing module is configured to: Obtain a sparse matrix from an external memory; The sparse matrix comprises a flow matrix and a stationary matrix, the flow matrix is a matrix for flow processing, and the stationary matrix is a matrix for fixed storage; Respectively perform block processing on the flow matrix and the stationary matrix according to a preset value to obtain a plurality of block flow matrices and a plurality of block stationary matrices; The elements in the block flow matrix comprise first non-zero elements and zero elements, and the elements in the block stationary matrix comprise second non-zero elements and zero elements; Respectively perform shift processing on the first non-zero elements and the second non-zero elements, and perform merging processing on the zero elements to obtain a dense flow matrix and a dense stationary matrix; Generate a flow auxiliary matrix according to the dense flow matrix to save the addresses of the first non-zero elements before shift processing, and generate a stationary auxiliary matrix and a first auxiliary bitmap according to the dense stationary matrix to save the addresses of the second non-zero elements before shift processing and the addresses of the second non-zero elements after shift processing; Perform a bitwise AND operation on the elements in the flow auxiliary matrix to obtain a second auxiliary bitmap, and perform a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap; A calculation module is configured to: Perform multiplication operation and addition calculation on the dense flow matrix and the dense stationary matrix according to the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap to obtain a partial sum result and a first address result of the partial sum result; the first address result comprises a row address and a column address; A sorting module is configured to: Sort the partial sum result according to the first address result to obtain a result matrix.

2. The block-based packed algorithm sparse matrix accelerator of claim 1, wherein, The respective block processing on the flow matrix and the stationary matrix according to a preset value to obtain a plurality of block flow matrices and a plurality of block stationary matrices comprises: Segmenting the flow matrix in the column direction to obtain a plurality of block flow matrices; Segmenting the stationary matrix in the row direction to obtain a plurality of block stationary matrices.

3. The block-based packed algorithm sparse matrix accelerator of claim 1, wherein, The respective shift processing on the first non-zero elements and the second non-zero elements, and the merging processing on the zero elements to obtain a dense flow matrix and a dense stationary matrix comprises: Performing non-conflict row shift processing on the first non-zero elements to obtain a shifted flow matrix; the non-conflict row shift processing is to shift the first non-zero elements in the row direction; Merging zero rows in the shifted flow matrix to obtain a dense flow matrix, wherein the zero row is a row with all zero elements; Performing non-conflict column shift processing on the second non-zero elements to obtain a first shifted stationary matrix; the non-conflict column shift processing is to shift the second non-zero elements in the column direction; Performing non-conflict row shift processing on the first non-zero elements to obtain a shifted flow matrix; the non-conflict row shift processing is to shift the first non-zero elements in the row direction; Merging zero rows in the shifted flow matrix to obtain a dense flow matrix, wherein the zero row is a row with all zero elements; Performing non-conflict column shift processing on the second non-zero elements to obtain a first shifted stationary matrix; the non-conflict column shift processing is to shift the second non-zero elements in the column direction; performing conflict shift processing on the second non-zero element in the first shift stationary matrix to obtain a second shift stationary matrix; the conflict shift processing is to simultaneously shift the second non-zero element in the first shift stationary matrix in a row direction and a column direction; merging zero columns in the second shift stationary matrix to obtain a dense stationary matrix, the zero columns being columns with all zero elements.

4. The block-based packed algorithm sparse matrix accelerator of claim 1, wherein, The dense flow matrix includes at least one first row vector, and the dense stationary matrix includes at least one first column vector; a plurality of the calculation modules are connected in a mesh, arranged by rows and columns, and the calculation modules in the same row are configured to: performing multiplication and addition calculation on the same first row vector of the same dense flow matrix and the different first column vectors of the same dense stationary matrix by using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, to obtain a partial sum result and a first address result of the partial sum result; the calculation modules in the same column are configured to: performing multiplication and addition calculation on the next dense flow matrix and the next dense stationary matrix by using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, to obtain a partial sum result and a first address result of the partial sum result; the calculation modules in the same row are further configured to: performing multiplication and addition calculation on the next first row vector of the same dense flow matrix and the different first column vectors of the same dense stationary matrix by using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap after a preset period, to obtain a partial sum result and a first address result of the partial sum result.

5. The block-based packed algorithm sparse matrix accelerator of claim 4, wherein, The flow auxiliary matrix includes at least one second row vector, and the stationary auxiliary matrix includes at least one second column vector; an element in the flow auxiliary matrix includes a first element, which is a first address of the first non-zero element in the block flow matrix before shift processing, and an element in the stationary auxiliary matrix includes a second element, which is a second address of the second non-zero element in the block stationary matrix before shift processing; the length of the first auxiliary bitmap is an integer multiple of a value obtained by subtracting one from the preset value; The elements in the first auxiliary bitmap include a third element and a fourth element, the third element being a third address of the second non-zero element in the block stationary matrix before conflict shift processing, and the fourth element being a fourth address of the second non-zero element in the dense stationary matrix after the conflict shift processing, the first address being a row address, the second address being a column address, and the third address and the fourth address being row addresses.

6. The block-based packed algorithm sparse matrix accelerator of claim 5, wherein, The calculation module includes: a shift adjustment module configured to: According to the first address, the second address, the third address and the fourth address, the position of the first non-zero element in the first row vector is adjusted, so that the first non-zero element in the first row vector and the second non-zero element in the first column vector correspond one by one according to the address before the shift processing; The multiplication operation module is configured to: Perform multiplication operation processing on the first non-zero element in the first row vector and the corresponding second non-zero element respectively to obtain a dot product result and a second address result of the dot product result, the second address result including a row address and a column address; The algorithm preprocessing module is further configured to: Perform bitwise AND operation on the first elements in the second row vector two by two to obtain a second auxiliary bit map; Perform bitwise AND operation on the second elements in the second column vector two by two to obtain a third auxiliary bit map; The addition tree module is configured to: Perform AND operation on the second auxiliary bit map and the third auxiliary bit map to obtain a sum result; Perform sum calculation on the dot product result through the addition tree according to the sum result to obtain a partial sum result and a first address result of the partial sum result.

7. The block-based packed algorithm sparse matrix accelerator of claim 6, wherein, The multiplication operation module includes: The storage submodule is configured to: Save the second non-zero element and the second address in the dense stationary matrix; The multiplier submodule is configured to: Obtain the first non-zero element and the first address in the dense flow matrix, and obtain the second non-zero element and the second address in the storage submodule; Perform multiplication operation processing on the first non-zero element and the second non-zero element to obtain a dot product result and a second address result of the dot product result.

8. The block-based packed algorithm sparse matrix accelerator of claim 6, wherein, The addition tree module includes: The selection submodule is configured to: Select a sum calculation method in the addition tree according to the sum result, so as to perform sum calculation on the dot product result through the sum calculation method.

9. The block-based packed algorithm sparse matrix accelerator of claim 6, wherein, Further comprising: The first storage module is configured to: Obtain and save the dense flow matrix, the dense stationary matrix, the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bit map, the second auxiliary bit map and the third auxiliary bit map generated by the algorithm preprocessing module; The second storage module is configured to: Obtain and save the partial sum result and the first address result.

10. The block-based packed algorithm sparse matrix accelerator of claim 9, wherein, The calculation module further includes: The output module is configured to: Obtain the partial sum result and the first address result; Output the partial sum result and the first address result to the second storage module.

Images