Sparse matrix accelerator based on block packaging algorithm

Abstract

The invention provides a sparse matrix accelerator based on a block packaging algorithm. The sparse matrix accelerator comprises an algorithm preprocessing module, a calculation module and a sorting module. The algorithm preprocessing module can obtain a sparse matrix comprising a stream matrix and a stationary matrix, block the stream matrix and the stationary matrix according to a preset numerical value, and then respectively perform shift merging processing on a first non-zero element, a second non-zero element and a zero element in the matrix to obtain a dense stream matrix and a dense stationary matrix; and storing the address of the first non-zero element before shifting and the addresses of the second non-zero element before and after shifting. The calculation module can perform multiplication and addition calculation on the dense flow matrix and the dense stationary matrix to obtain a partial sum result and a first address result of the partial sum result. The sorting module can sort the part and the result according to the first address result to obtain a result matrix. The accelerator can improve the density degree of a matrix, can realize correct same-column addition, and further can improve the calculation efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of sparse matrix computing, and in particular to a sparse matrix accelerator based on a block packing algorithm. Background Art

[0002] Sparse General Matrix Multiply (SpGEMM) is an important computing kernel that is ubiquitous in Deep Neural Networks (DNN) models. When artificial intelligence applications run on GPUs and accelerators, SpGEMM affects the performance of the application, so sparse matrices need to be processed. The sparse input matrices can be compressed into denser input matrices through offline preprocessing filling algorithms, and then they are mapped to contraction arrays to perform SpGEMM to improve execution efficiency. Each offline preprocessing filling algorithm designs specialized hardware for calculations, such as collision-aware algorithms and dedicated hardware structures; block-based SorPack packing algorithms and dedicated hardware structures.

[0003] The collision-aware algorithm selects sparse columns to combine into denser columns without changing the row positions of non-zero elements. However, in deep neural network applications, column merging of large matrices is very time-consuming, and the data flow in the hardware is designed for sparse matrix-vector multiplication (SpMV), and is not optimized for SpGEMM. In order to improve the utilization of processing units, the block-based SorPack packing algorithm divides the matrix into a flow matrix and a stationary matrix, and then divides the flow matrix and the stationary matrix into blocks, and rearranges the data in the data block of the stationary matrix in descending order according to the number of non-zero values ​​in each column, and rearranges the data in the data block of the flow matrix in descending order according to the number of non-zero values ​​in each row, and then inserts and merges the data blocks. For the correctness of the calculation, the algorithm also gives two auxiliary matrices to mark the rows and columns of the packed matrix, but its packing algorithm causes the accumulated values ​​of the same column to be calculated in different cycles, which increases the complexity of its hardware and execution performance. At the same time, because its packing algorithm does not tolerate collisions, the density of the matrix actually packed is not the highest. In addition, due to the descending sorting operation, it is also time-consuming to pack and merge large sparse matrices, which leads to low computational efficiency. Summary of the invention

[0004] The present application provides a sparse matrix accelerator based on a block packing algorithm to solve the problem of low efficiency in sparse matrix calculation.

[0005] In a first aspect, the present application provides a sparse matrix accelerator based on a block packing algorithm, comprising:

[0006] An algorithm preprocessing module, wherein the algorithm preprocessing module is configured to:

[0007] Acquire a sparse matrix from an external memory; the sparse matrix includes a flow matrix and a stationary matrix, the flow matrix is ​​a matrix for stream processing, and the stationary matrix is ​​a matrix for fixed storage;

[0008] Performing block processing on the flow matrix and the stationary matrix respectively according to preset values ​​to obtain a plurality of block flow matrices and a plurality of block stationary matrices; the elements in the block flow matrix include a first non-zero element and a zero element, and the elements in the block stationary matrix include a second non-zero element and a zero element;

[0009] Performing shift processing on the first non-zero element and the second non-zero element respectively, and performing merging processing on the zero element, so as to obtain a dense flow matrix and a dense stationary matrix;

[0010] Generate a flow auxiliary matrix according to the dense flow matrix to store the address of the first non-zero element before the shift process, and generate a stationary auxiliary matrix and a first auxiliary bitmap according to the dense stationary matrix to store the address of the second non-zero element before the shift process and the address of the second non-zero element after the shift process;

[0011] performing a bitwise AND operation on the elements in the stream auxiliary matrix to obtain a second auxiliary bitmap, and performing a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap;

[0012] A computing module, wherein the computing module is configured as:

[0013] According to the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, the dense flow matrix and the dense stationary matrix are multiplied and added to obtain a partial sum result and a first address result of the partial sum result; the first address result includes a row address and a column address;

[0014] A sorting module, wherein the sorting module is configured to:

[0015] The partial sum results are sorted according to the first address results to obtain a result matrix.

[0016] Optionally, performing block processing on the flow matrix and the stationary matrix according to preset values ​​to obtain a plurality of block flow matrices and a plurality of block stationary matrices comprises:

[0017] Splitting the flow matrix in a column direction to obtain a plurality of block flow matrices;

[0018] The stationary matrix is ​​divided in a row direction to obtain a plurality of block stationary matrices.

[0019] Optionally, performing shift processing on the first non-zero element and the second non-zero element respectively, and performing merging processing on the zero element to obtain a dense flow matrix and a dense stationary matrix includes:

[0020] Performing a non-conflicting row shift process on the first non-zero element to obtain a shift flow matrix; the non-conflicting row shift process is shifting the first non-zero element along the row direction;

[0021] Merging zero rows in the shifted flow matrix to obtain a dense flow matrix, wherein the zero rows are rows whose elements are all zero elements;

[0022] Performing a non-conflict column shift process on the second non-zero element to obtain a first shifted stationary matrix; the non-conflict column shift process is shifting the second non-zero element along the column direction;

[0023] Performing a conflict shift process on the second non-zero element in the first shifted stationary matrix to obtain a second shifted stationary matrix; the conflict shift process is to shift the second non-zero element in the first shifted stationary matrix simultaneously along the row direction and the column direction;

[0024] The zero columns in the second shifted stationary matrix are merged to obtain a dense stationary matrix, wherein the zero columns are columns whose elements are all zero elements.

[0025] Optionally, the dense flow matrix includes at least one first row vector, and the dense stationary matrix includes at least one first column vector; the plurality of computing modules are connected in a mesh, and are arranged in rows and columns, and the computing modules in the same row are configured as follows:

[0026] Using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, multiplication and addition operations are performed on the same first row vector of the same dense flow matrix and different first column vectors of the same dense stationary matrix to obtain a partial sum result and a first address result of the partial sum result;

[0027] The computing modules in the same column are configured as follows:

[0028] Using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, perform multiplication and addition calculations on the next dense flow matrix and the next dense stationary matrix to obtain a partial sum result and a first address result of the partial sum result;

[0029] The computing modules in the same row are also configured as:

[0030] After a preset period, the flow auxiliary matrix, the smooth auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap are used to perform multiplication and addition operations on the next first row vector of the same dense flow matrix and the different first column vectors of the same dense smooth matrix to obtain a partial sum result and a first address result of the partial sum result.

[0031] Optionally, the flow auxiliary matrix includes at least one second row vector, and the smooth auxiliary matrix includes at least one second column vector; the elements in the flow auxiliary matrix include a first element, which is the first address of the first non-zero element in the block flow matrix before the shift processing, and the elements in the smooth auxiliary matrix include a second element, which is the second address of the second non-zero element in the block smooth matrix before the shift processing; the length of the first auxiliary bitmap is an integer multiple of the value obtained by subtracting one from the preset value, and the elements in the first auxiliary bitmap include a third element and a fourth element, the third element is the third address of the second non-zero element in the block smooth matrix before the conflict shift processing, and the fourth element is the fourth address of the second non-zero element in the dense smooth matrix after the conflict shift processing, the first address is a row address, the second address is a column address, and the third address and the fourth address are row addresses.

[0032] Optionally, the computing module includes:

[0033] A shift adjustment module, wherein the shift adjustment module is configured to:

[0034] adjusting the position of the first non-zero element in the first row vector according to the first address, the second address, the third address, and the fourth address, so that the first non-zero element in the first row vector corresponds to the second non-zero element in the first column vector in one-to-one correspondence according to the addresses before the shift processing;

[0035] multiplication operation modules, the number of which is equal to the preset value, and the multiplication operation modules are configured as follows:

[0036] Performing multiplication processing on the first non-zero element and the corresponding second non-zero element in the first row vector respectively to obtain a dot product result and a second address result of the dot product result, wherein the second address result includes a row address and a column address;

[0037] The algorithm preprocessing module is also configured to:

[0038] Performing a bitwise AND operation on the first elements in the second row vector in pairs to obtain a second auxiliary bitmap;

[0039] Performing a bitwise AND operation on the second elements in the second column vector in pairs to obtain a third auxiliary bitmap;

[0040] An addition tree module, wherein the addition tree module is configured to:

[0041] performing an AND operation on the second auxiliary bitmap and the third auxiliary bitmap to obtain a sum result;

[0042] An addition calculation is performed on the dot product result through an addition tree according to the addition result to obtain a partial sum result and a first address result of the partial sum result.

[0043] Optionally, the multiplication operation module includes:

[0044] A storage submodule, wherein the storage submodule is configured as follows:

[0045] Saving the second non-zero element and the second address in the dense stationary matrix;

[0046] A multiplier submodule, wherein the multiplier submodule is configured to:

[0047] Obtaining the first non-zero element and the first address in the dense flow matrix, and obtaining the second non-zero element and the second address in the storage submodule;

[0048] A multiplication operation is performed on the first non-zero element and the second non-zero element to obtain a dot product result and a second address result of the dot product result.

[0049] Optionally, the addition tree module includes:

[0050] Select a submodule, the selection submodule is configured as follows:

[0051] An addition method in an addition tree is selected according to the addition result, so as to perform addition calculation on the dot product result by using the addition method.

[0052] Optionally, also include:

[0053] A first storage module, wherein the first storage module is configured as:

[0054] Acquire and save the dense flow matrix, dense stationary matrix, flow auxiliary matrix, stationary auxiliary matrix, first auxiliary bitmap, second auxiliary bitmap and third auxiliary bitmap generated by the algorithm preprocessing module;

[0055] A second storage module, wherein the second storage module is configured as:

[0056] The partial sum result and the first address result are obtained and saved.

[0057] Optionally, the computing module further includes:

[0058] An output module, wherein the output module is configured to:

[0059] Obtain the partial sum result and the first address result;

[0060] The partial sum result and the first address result are output to the second storage module.

[0061] The present application provides a sparse matrix accelerator based on a block packing algorithm, including an algorithm preprocessing module, a computing module and a sorting module. The algorithm preprocessing module is used to obtain a sparse matrix including a flow matrix and a stationary matrix, and respectively perform block processing on the flow matrix and the stationary matrix according to preset values ​​to obtain a block flow matrix and a block stationary matrix; respectively perform shift and merge processing on the first non-zero element in the block flow matrix and the second non-zero element in the block stationary matrix and the zero element in the matrix to obtain a dense flow matrix and a dense stationary matrix; generate a flow auxiliary matrix to save the address of the first non-zero element before shifting, and generate a stationary auxiliary matrix and a first auxiliary bitmap to save the address of the second non-zero element before and after shifting; perform a bitwise AND operation on the elements in the flow auxiliary matrix to obtain a second auxiliary bitmap, and perform a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap. The computing module is used to perform multiplication and addition operations on dense flow matrices and dense stable matrices according to the flow auxiliary matrix, the stable auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap to obtain partial sum results and the first address results of the partial sum results. The sorting module is used to sort the partial sum results according to the first address result to obtain a result matrix. The accelerator provided in the present application can improve the density of the matrix and thus improve the computing efficiency by dividing the sparse matrix into blocks and performing shift and merge processing through a packing algorithm. Through the flow auxiliary matrix, the stable auxiliary matrix and the three auxiliary bitmaps, correct same-column addition can be achieved, making the accelerator faster and more efficient, and solving the problem of low computing efficiency of sparse matrices. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0063] Figure 1 A schematic diagram of sparse general matrix multiplication provided for this application;

[0064] Figure 2 A schematic diagram of the sparse matrix accelerator structure based on the block packing algorithm provided in this application;

[0065] Figure 3 A schematic diagram of segmenting the flow matrix and the stationary matrix when the preset value is equal to 4 provided in this application;

[0066] Figure 4 A schematic diagram of the process of obtaining a first shifted stationary matrix when the preset value is equal to 4 provided in the present application;

[0067] Figure 5 A schematic diagram of the process of obtaining a second shifted stationary matrix when the preset value is equal to 4 provided in the present application;

[0068] Figure 6 Schematic diagram of the process of generating flow auxiliary matrix and stable auxiliary matrix provided for this application;

[0069] Figure 7 A method for summing the elements in the second row vector or the second column vector when the preset value provided in this application is equal to 4;

[0070] Figure 8 A schematic diagram of the calculation process of the addition tree module when the preset value provided by this application is equal to 4;

[0071] Fig. 9 A schematic diagram of the calculation module structure provided by this application when the preset value is equal to 4;

[0072] Fig.10 A schematic diagram of the structure of the multiplication operation module provided in this application;

[0073] Fig.11 A schematic diagram of the sparse matrix accelerator structure provided for this application, which includes multiple computing modules. DETAILED DESCRIPTION

[0074] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0075] For general matrix multiplication (GEMM), assuming that A is an m×p matrix and B is a p×n matrix, then the m×n matrix C is called the product of the matrices A and B, denoted as Z=XY, where the i-th row and j-th column element in the matrix Z can be expressed as:

[0076]

[0077] Figure 1 is an example of SpGEMM, where Figure 1 (a) in the figure is a 6×8 sparse matrix X. Figure 1 (b) in the figure is a 8×8 sparse matrix Y. Figure 1 (c) in the figure is the product of two matrices. In each matrix, the gray squares represent zero elements, and the other squares marked with capital letters represent non-zero elements. Non-zero elements in the same row of the X matrix are represented by the same letter, and non-zero elements in the same column of the Y matrix are represented by the same letter.

[0078] When using formula (1) to calculate the result of the first row and first column of the result matrix, such as Figure 1 As shown in (a), there will be 0 times 0, 0 times a non-zero value, and non-zero value times 0. Since 0 times any number equals 0, multiplication and addition with zero elements is unnecessary, which will affect the execution efficiency of the hardware. At the same time, many zero elements will also occupy a lot of storage space, which will lead to low matrix calculation efficiency and high hardware requirements.

[0079] In order to solve the problem of low efficiency of sparse matrix calculation, the embodiment of the present application provides a sparse matrix accelerator based on a block packing algorithm, see Figure 2 ,The accelerator includes: algorithm preprocessing module, computing module and sorting module.

[0080] The algorithm preprocessing module may be an algorithm model stored in the CPU, and the algorithm preprocessing module is configured as follows:

[0081] S210: Acquire a sparse matrix from an external memory.

[0082] In an embodiment of the present application, the input sparse matrix is ​​defined as a flow matrix and a stationary matrix, respectively, wherein the flow matrix is ​​a matrix for streaming processing, and the stationary matrix is ​​a matrix for fixed storage, that is, in subsequent calculations, the elements in the flow matrix flow between the computing modules, and the elements in the stationary matrix are fixed in the computing modules.

[0083] S220: performing block processing on the flow matrix and the stationary matrix according to preset values ​​to obtain a plurality of block flow matrices and a plurality of block stationary matrices.

[0084] Block processing refers to dividing the flow matrix or stationary matrix along the row or column direction according to the preset value to obtain several blocks of the same size, where the blocks with less than the preset number of rows or columns are also treated as a matrix separately. The elements in the block flow matrix include the first non-zero element and the zero element, and the elements in the block stationary matrix include the second non-zero element and the zero element.

[0085] S230: Perform shift processing on the first non-zero element and the second non-zero element respectively, and perform merging processing on the zero element to obtain a dense flow matrix and a dense stationary matrix.

[0086] Compared with other packing algorithms that sort according to the number of non-zero elements in rows or columns during packing, which takes a long time and requires a large storage space, this application divides the sparse matrix into blocks and then performs shift and merge processing through a packing algorithm, which can make the sparse matrix dense and increase the density of the matrix, thereby reducing storage space and improving computing efficiency.

[0087] S240: Generate a flow auxiliary matrix according to the dense flow matrix to save the address of the first non-zero element before the shift process, and generate a stationary auxiliary matrix and a first auxiliary bitmap according to the dense stationary matrix to save the address of the second non-zero element before the shift process and the address of the second non-zero element after the shift process.

[0088] Since the one-to-one correspondence between elements is broken after the shift process, it is necessary to record the addresses of the first non-zero element and the second non-zero element before and after the shift process. The address of the first non-zero element before the shift process is the address of the first non-zero element in the block flow matrix, and the address of the second non-zero element before the shift process is the address of the second non-zero element in the block stationary matrix.

[0089] S250: performing a bitwise AND operation on the elements in the flow auxiliary matrix to obtain a second auxiliary bitmap, and performing a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap.

[0090] The second auxiliary bitmap obtained includes all possible summation methods of the first non-zero elements in the dense flow matrix, and the third auxiliary bitmap includes all possible summation methods of the second non-zero elements in the dense stationary matrix, so that the correct summation result can be obtained in the following summation calculation.

[0091] The computing module is configured to perform multiplication and addition operations on the dense streaming matrix and the dense stable matrix according to the streaming auxiliary matrix, the stable auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap, and the third auxiliary bitmap to obtain a partial sum result and a first address result of the partial sum result, wherein the first address result includes a row address and a column address.

[0092] The sorting module is configured to sort the partial sum results according to the first address result to obtain a result matrix. Since the sorting module is a module that can be obtained through the prior art, it is not described in detail in this application.

[0093] The accelerator provided by the present application can improve the density of the matrix by dividing the sparse matrix into blocks and performing shift and merge processing through a packing algorithm, which can reduce the number of transfers during calculation and improve the calculation efficiency. The addresses of non-zero elements before non-conflicting shifts are saved by the flow auxiliary matrix and the stable auxiliary matrix respectively, the addresses of non-zero elements before and after conflicting shifts are saved by the first auxiliary bitmap, and the correct addition result is obtained by the second auxiliary bitmap and the third auxiliary bitmap, so that the correct same-column addition can be achieved, making the accelerator faster and more efficient.

[0094] In some embodiments, performing block processing on the flow matrix and the stationary matrix according to preset values ​​to obtain multiple block flow matrices and multiple block stationary matrices includes: dividing the flow matrix in the column direction to obtain multiple block flow matrices; dividing the stationary matrix in the row direction to obtain multiple block flow stationary matrices. Figure 3 , Figure 3 This is an example of partitioning the convection matrix and the stationary matrix. Figure 3 (a) in the figure is a flow matrix with 6 rows and 8 columns. Figure 3 (b) in the figure is a stationary matrix with 8 rows and 8 columns. Figure 3 The flow matrix in is divided into two blocks, and each 4 columns are divided into a small matrix. The two matrices obtained are the block flow matrices, that is, Figure 3 (c) in the row direction. Figure 3 The stationary matrix in is divided into blocks, and every 4 rows are divided into a small matrix. The two matrices obtained are block stationary matrices, that is, Figure 3 (d) in.

[0095] The block processing in this application is different from other arbitrary shape block processing methods, and only splits the sparse matrix in one direction. It can be understood that Figure 3 The preset value of the block processing is equal to 4. Since the fewer the number of elements in a row / column, the fewer the cases that can be merged, by setting the preset value to 4, it is conducive to shift merging processing and the difficulty of the addition tree can be reduced. In the following description, the case where the preset value is 4 is used as an example. In some embodiments, the preset value can also be other values ​​less than 6.

[0096] In some embodiments, performing shift processing on the first non-zero element and the second non-zero element, respectively, and performing merging processing on the zero element to obtain a dense flow matrix and a dense stationary matrix includes:

[0097] S410: Perform a non-conflicting row shift process on the first non-zero element to obtain a shift flow matrix.

[0098] S420: Merge zero rows in the shifted flow matrix to obtain a dense flow matrix.

[0099] S430: Perform non-conflicting column shift processing on the second non-zero element to obtain a first shifted stationary matrix.

[0100] S440: Perform conflict shift processing on the second non-zero element in the first shifted stationary matrix to obtain a second shifted stationary matrix.

[0101] S450: Merge the zero columns in the second shifted stationary matrix to obtain a dense stationary matrix, where the zero columns are columns whose elements are all zero elements.

[0102] It should be noted that the non-conflicting row shift process is to shift the first non-zero element along the row direction, and the non-conflicting column shift process is to shift the second non-zero element along the column direction. A zero row is a row whose elements are all zero elements, and a zero column is a column whose elements are all zero elements. The conflicting shift process is to shift the second non-zero element in the first shifted stationary matrix along both the row direction and the column direction.

[0103] See also Figure 4-Figure 5 In the figure, the gray grids without letters represent zero elements, the elements marked with letters represent non-zero elements, and the non-zero elements in the same row / column are represented by the same letter. Figure 4 (a) is the block flow matrix, Figure 4 (b) is a block stationary matrix. Figure 4 The first non-zero element of (a) is shifted along the row direction, that is, only the row address of the first non-zero element is changed, and the resulting matrix is ​​the shift flow matrix, that is, Figure 4 (c) in the Merger Figure 4 The rows in (c) of which all elements are zero elements can obtain a dense flow matrix. Figure 4 The second non-zero element of (b) is shifted along the column direction, that is, only the column address of the second non-zero element is changed, and the resulting matrix is ​​the first shift stationary matrix, that is, Figure 4 (d) in.

[0104] It should be noted that, in the process of performing non-conflicting shifts, due to the position where the second non-zero element is moved, there may be other non-zero elements, such as Figure 4 Medium (d) or Figure 5 Therefore, in some embodiments, it is necessary to perform a conflict shift operation on the first shifted stable matrix, and perform a shift operation on the second non-zero element in the row direction and the column direction at the same time to obtain a second shifted stable matrix, such as Figure 5 The process from (a) to (c) is combined Figure 5 The columns in (c) are all zero elements, and a dense stationary matrix can be obtained.

[0105] It is understood that a dense flow matrix includes at least one first row vector, and a dense stationary matrix includes at least one first column vector. Fig.11 , multiple computing modules are meshed and arranged in rows and columns. The computing modules in the same row are configured as follows: using the flow auxiliary matrix, the stable auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, multiply and add the same first row vector of the same dense flow matrix and the different first column vectors of the same dense stable matrix to obtain the partial sum result and the first address result of the partial sum result, and then the results of multiplying the same row of the dense flow matrix with different columns of the dense stable matrix can be calculated respectively.

[0106] The computing modules in the same column are configured to use the flow auxiliary matrix, the smooth auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap to perform multiplication and addition operations on the next dense flow matrix and the next dense smooth matrix to obtain the partial sum result and the first address result of the partial sum result, and then calculate the results of multiplication of different block flow matrices and block smooth matrices respectively.

[0107] The computing modules in the same row are also configured to: after a preset period, use the flow auxiliary matrix, the stable auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap to perform multiplication and addition operations on the next first row vector of the same dense flow matrix and the different first column vectors of the same dense stable matrix to obtain the partial sum result and the first address result of the partial sum result, and then calculate the results of multiplying the next row of the dense flow matrix with different columns of the dense stable matrix respectively.

[0108] The flow auxiliary matrix includes at least one second row vector, and the stationary auxiliary matrix includes at least one second column vector. The elements in the flow auxiliary matrix include a first element, which is the first address of the first non-zero element in the block flow matrix before the shift process; the elements in the stationary auxiliary matrix include a second element, which is the second address of the second non-zero element in the block stationary matrix before the shift process, wherein the first address is a row address and the second address is a column address. Figure 6 , Figure 6 (a) and Figure 6 The numbers 0, 1, 2, 3, etc. in (b) are the row and column addresses of the non-zero elements in the flow matrix and the stationary matrix. Figure 6 (c) and Figure 6 (d) in the figure are examples of dense flow matrix and dense stationary matrix, respectively. Figure 6 (e) and Figure 6 The (f) in the equation are the flow auxiliary matrix and the stationary auxiliary matrix, Figure 6 The number in the block (e) is the row address of the flow matrix, that is, the row address of the first non-zero element in the flow matrix before shifting. Figure 6 The number in the block (f) is the column address of the stationary matrix, that is, the column address before the second non-zero element in the stationary matrix is ​​shifted.

[0109] Since the second non-zero element in the stable matrix is ​​subjected to conflict shift processing, the row address and column address of some second elements may change. Therefore, in the embodiment of the present application, the row address of the second non-zero element whose row address and column address are changed is recorded by the first auxiliary bitmap. The elements in the first auxiliary bitmap include a third element and a fourth element, the third element is the third address of the second non-zero element in the block stable matrix before the conflict shift processing, the fourth element is the fourth address of the second non-zero element in the dense stable matrix after the conflict shift processing, and the third address and the fourth address are row addresses.

[0110] The length of the first auxiliary bitmap is an integer multiple of the value obtained by subtracting one from the preset value. Since the stable matrix is ​​divided by 4 preset values, when performing conflict shift processing, the second non-zero elements that need to be shifted in the 4 elements of the same column will not exceed 3 at most. Therefore, in the embodiment of the present application, the length of the first auxiliary bitmap is set to 12 bits, and every 6 bits can save up to three shift signals, wherein the first 6 bits are used to save the row address of the second non-zero element before shifting, that is, the third address, and the last 6 bits are used to save the row address of the second non-zero element after shifting, that is, the fourth address. By setting a 12-bit first auxiliary bitmap, the second non-zero element after shifting can be restored to the position before shifting during calculation to ensure the accuracy of the calculation.

[0111] See also Fig. 9 In some embodiments, the computing module includes a shift adjustment module, a multiplication operation module, and an addition tree module. The first non-zero element may be marked as Str_Data, the first address as row_id, the second non-zero element as Sta_Data, and the second address as col_id.

[0112] The shift adjustment module is configured to: adjust the position of the first non-zero element Str_Data in the first row vector according to the first address row_id, the second address col_id, the third address and the fourth address, so that the first non-zero element Str_Data in the first row vector and the second non-zero element Sta_Data in the first column vector correspond one-to-one according to the address before the shift processing, so as to calculate the product of the first non-zero element Str_Data and the second non-zero element Sta_Data according to the position before the shift through the following multiplication operation module.

[0113] The number of multiplication operation modules in each calculation module is equal to the preset value, and the multiplication operation module is configured to: perform multiplication processing on the first non-zero element Str_Data and the corresponding second non-zero element Sta_Data in the first row vector respectively to obtain the dot product result Mul_data and the second address result row_id&col_id of the dot product result. It can be understood that the second address result includes a row address and a column address.

[0114] See also Fig.10 The multiplication operation module includes a storage submodule and a multiplier submodule. The storage submodule is configured to: save the second non-zero element Sta_Data and the second address col_id in the dense stationary matrix. In some embodiments, the storage submodule may be a register. The multiplier submodule is configured to: obtain the first non-zero element Str_Data and the first address row_id in the dense flow matrix, and obtain the second non-zero element Sta_Data and the second address col_id in the storage submodule, and perform multiplication operation processing on the first non-zero element Str_Data and the second non-zero element Sta_Data to obtain the dot product result Mul_data and the second address result row_id&col_id of the dot product result.

[0115] Since the first non-zero element and the second non-zero element are shifted and merged, the dot product result cannot be simply added according to formula (1). Selection is required to obtain the correct sum result. Therefore, the second auxiliary bitmap and the third auxiliary bitmap are required to ensure the accuracy of the calculation result. In some embodiments, the algorithm preprocessing module is further configured to: perform a bitwise AND operation on the first elements in the second row vector in pairs to obtain a second auxiliary bitmap; perform a bitwise AND operation on the second elements in the second column vector in pairs to obtain a third auxiliary bitmap. When the matrix is ​​split based on 4, since the number of elements in the second row vector and the second column vector are both 4, there are a total of 14 ways of adding. See Figure 7 , ①, ②, ③, ④ represent the elements in the first row vector or the first column vector respectively, 0 means the addresses of the two elements are different, 1 means the addresses of the two elements are the same, Figure 7 Each result in represents a way of adding, Figure 7 The result in is all possible summation methods of the second auxiliary bitmap or the third auxiliary bitmap. Exemplarily, Figure 7 ①+②+③+④ in the figure means that the four dot product results of the input are added in sequence.

[0116] See also Figure 8, the addition tree module is configured to: perform an AND operation on the second auxiliary bitmap and the third auxiliary bitmap to obtain a sum result; perform a sum calculation on the dot product result through the addition tree according to the sum result to obtain the partial sum result Data and the first address result row_id&col_id of the partial sum result. After that, the partial sum result Data and the first address result row_id&col_id of the partial sum result can be sent to the output module described below. Since there are only 14 summing methods when the preset value is equal to 4, the addition tree structure is simple and the hardware overhead is reduced. In some embodiments, the addition tree module includes a selection submodule, and the selection submodule is configured to: select a summing method in the addition tree according to the summing result, so as to perform a summing calculation on the dot product result through the summing method.

[0117] It should also be noted that, although the first address result and the second address result are both marked as row_id&col_id in the present application, the first address result and the second address result actually calculated and outputted are not the same.

[0118] See again Fig.11 In some embodiments, the accelerator further includes a first storage module and a second storage module, the first storage module and the second storage module may be SRAM memories, the first storage module is configured to: obtain and save the dense flow matrix, dense stationary matrix, flow auxiliary matrix, stationary auxiliary matrix, first auxiliary bitmap, second auxiliary bitmap and third auxiliary bitmap generated by the algorithm preprocessing module. The second storage module is configured to: obtain and save the partial sum result and the first address result.

[0119] See again Fig. 9 In some embodiments, the computing module further includes an output module, which may be a router, and the output module is configured to: obtain a partial sum result and a first address result; and output the partial sum result and the first address result to a second storage module.

[0120] It can be seen from the above technical solutions that the embodiment of the present application provides a sparse matrix accelerator based on a block packing algorithm, including: an algorithm preprocessing module, a computing module and a sorting module. The algorithm preprocessing module is used to obtain a sparse matrix including a flow matrix and a stationary matrix, and respectively perform block processing on the flow matrix and the stationary matrix according to preset values ​​to obtain a block flow matrix and a block stationary matrix; respectively perform shift and merge processing on the first non-zero element in the block flow matrix and the second non-zero element in the block stationary matrix and the zero element in the matrix to obtain a dense flow matrix and a dense stationary matrix; generate a flow auxiliary matrix to save the address of the first non-zero element before shifting, and generate a stationary auxiliary matrix and a first auxiliary bitmap to save the address of the second non-zero element before shifting; perform a bitwise AND operation on the elements in the flow auxiliary matrix to obtain a second auxiliary bitmap, and perform a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap. The computing module is used to perform multiplication and addition operations on dense flow matrices and dense stable matrices according to the flow auxiliary matrix, the stable auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap to obtain partial sum results and the first address results of the partial sum results. The sorting module is used to sort the partial sum results according to the first address result to obtain a result matrix. The accelerator provided in the present application can improve the density of the matrix and thus improve the computing efficiency by dividing the sparse matrix into blocks and performing shift and merge processing through a packing algorithm. Through the flow auxiliary matrix, the stable auxiliary matrix and the three auxiliary bitmaps, correct same-column addition can be achieved, making the accelerator faster and more efficient, and solving the problem of low computing efficiency of sparse matrices.

[0121] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A sparse matrix accelerator based on a block packing algorithm, characterized in that: include: An algorithm preprocessing module, wherein the algorithm preprocessing module is configured to: Get sparse matrices from external memory; The sparse matrix includes a flow matrix and a stationary matrix, the flow matrix is ​​a matrix for stream processing, and the stationary matrix is ​​a matrix for fixed storage; Performing block processing on the flow matrix and the stationary matrix respectively according to preset values ​​to obtain a plurality of block flow matrices and a plurality of block stationary matrices; The elements in the block flow matrix include a first non-zero element and a zero element, and the elements in the block stationary matrix include a second non-zero element and a zero element; Performing shift processing on the first non-zero element and the second non-zero element respectively, and performing merging processing on the zero element, so as to obtain a dense flow matrix and a dense stationary matrix; Generate a flow auxiliary matrix according to the dense flow matrix to store the address of the first non-zero element before the shift process, and generate a stationary auxiliary matrix and a first auxiliary bitmap according to the dense stationary matrix to store the address of the second non-zero element before the shift process and the address of the second non-zero element after the shift process; performing a bitwise AND operation on the elements in the stream auxiliary matrix to obtain a second auxiliary bitmap, and performing a bitwise AND operation on the elements in the stationary auxiliary matrix to obtain a third auxiliary bitmap; A computing module, wherein the computing module is configured as: According to the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, the dense flow matrix and the dense stationary matrix are multiplied and added to obtain a partial sum result and a first address result of the partial sum result; the first address result includes a row address and a column address; A sorting module, wherein the sorting module is configured to: The partial sum results are sorted according to the first address results to obtain a result matrix.

2. The sparse matrix accelerator based on the block packing algorithm according to claim 1, characterized in that: The performing block processing on the flow matrix and the stationary matrix according to preset values ​​to obtain a plurality of block flow matrices and a plurality of block stationary matrices comprises: Splitting the flow matrix in a column direction to obtain a plurality of block flow matrices; The stationary matrix is ​​divided in a row direction to obtain a plurality of block stationary matrices.

3. The sparse matrix accelerator based on the block packing algorithm according to claim 1, characterized in that: The performing shift processing on the first non-zero element and the second non-zero element respectively, and performing merging processing on the zero element to obtain a dense flow matrix and a dense stationary matrix comprises: Performing a non-conflicting row shift process on the first non-zero element to obtain a shift flow matrix; the non-conflicting row shift process is shifting the first non-zero element along the row direction; Merging zero rows in the shifted flow matrix to obtain a dense flow matrix, wherein the zero rows are rows whose elements are all zero elements; Performing a non-conflict column shift process on the second non-zero element to obtain a first shifted stationary matrix; the non-conflict column shift process is shifting the second non-zero element along the column direction; Performing a conflict shift process on the second non-zero element in the first shifted stationary matrix to obtain a second shifted stationary matrix; the conflict shift process is to shift the second non-zero element in the first shifted stationary matrix simultaneously along the row direction and the column direction; The zero columns in the second shifted stationary matrix are merged to obtain a dense stationary matrix, wherein the zero columns are columns whose elements are all zero elements.

4. The sparse matrix accelerator based on block packing algorithm according to claim 1, characterized in that: The dense flow matrix includes at least one first row vector, and the dense stationary matrix includes at least one first column vector; the plurality of computing modules are connected in a mesh, arranged in rows and columns, and the computing modules in the same row are configured as follows: Using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, multiplication and addition operations are performed on the same first row vector of the same dense flow matrix and different first column vectors of the same dense stationary matrix to obtain a partial sum result and a first address result of the partial sum result; The computing modules in the same column are configured as follows: Using the flow auxiliary matrix, the stationary auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap, perform multiplication and addition calculations on the next dense flow matrix and the next dense stationary matrix to obtain a partial sum result and a first address result of the partial sum result; The computing modules in the same row are also configured as: After a preset period, the flow auxiliary matrix, the smooth auxiliary matrix, the first auxiliary bitmap, the second auxiliary bitmap and the third auxiliary bitmap are used to perform multiplication and addition operations on the next first row vector of the same dense flow matrix and the different first column vectors of the same dense smooth matrix to obtain a partial sum result and a first address result of the partial sum result.

5. The sparse matrix accelerator based on block packing algorithm according to claim 4, characterized in that: The flow auxiliary matrix includes at least one second row vector, and the stable auxiliary matrix includes at least one second column vector; the elements in the flow auxiliary matrix include a first element, which is the first address of the first non-zero element in the block flow matrix before the shift process, and the elements in the stable auxiliary matrix include a second element, which is the second address of the second non-zero element in the block stable matrix before the shift process; the length of the first auxiliary bitmap is an integer multiple of the value obtained by subtracting one from the preset value; The elements in the first auxiliary bitmap include a third element and a fourth element, the third element is the third address of the second non-zero element in the block stationary matrix before the conflict shift processing, the fourth element is the fourth address of the second non-zero element in the dense stationary matrix after the conflict shift processing, the first address is a row address, the second address is a column address, and the third address and the fourth address are row addresses.

6. The sparse matrix accelerator based on block packing algorithm according to claim 5, characterized in that: The computing module includes: A shift adjustment module, wherein the shift adjustment module is configured to: adjusting the position of the first non-zero element in the first row vector according to the first address, the second address, the third address, and the fourth address, so that the first non-zero element in the first row vector corresponds to the second non-zero element in the first column vector in one-to-one correspondence according to the addresses before the shift processing; multiplication operation modules, the number of which is equal to the preset value, and the multiplication operation modules are configured as follows: Performing multiplication processing on the first non-zero element and the corresponding second non-zero element in the first row vector respectively to obtain a dot product result and a second address result of the dot product result, wherein the second address result includes a row address and a column address; The algorithm preprocessing module is also configured to: Performing a bitwise AND operation on the first elements in the second row vector in pairs to obtain a second auxiliary bitmap; Performing a bitwise AND operation on the second elements in the second column vector in pairs to obtain a third auxiliary bitmap; An addition tree module, wherein the addition tree module is configured to: performing an AND operation on the second auxiliary bitmap and the third auxiliary bitmap to obtain a sum result; An addition calculation is performed on the dot product result through an addition tree according to the addition result to obtain a partial sum result and a first address result of the partial sum result.

7. The sparse matrix accelerator based on block packing algorithm according to claim 6, characterized in that: The multiplication operation module comprises: A storage submodule, wherein the storage submodule is configured as follows: Saving the second non-zero element and the second address in the dense stationary matrix; A multiplier submodule, wherein the multiplier submodule is configured to: Obtaining the first non-zero element and the first address in the dense flow matrix, and obtaining the second non-zero element and the second address in the storage submodule; A multiplication operation is performed on the first non-zero element and the second non-zero element to obtain a dot product result and a second address result of the dot product result.

8. The sparse matrix accelerator based on block packing algorithm according to claim 6, characterized in that: The addition tree module comprises: Select a submodule, the selection submodule is configured as follows: An addition method in an addition tree is selected according to the addition result, so as to perform addition calculation on the dot product result by using the addition method.

9. The sparse matrix accelerator based on block packing algorithm according to claim 6, characterized in that: Also includes: A first storage module, wherein the first storage module is configured as: Acquire and save the dense flow matrix, dense stationary matrix, flow auxiliary matrix, stationary auxiliary matrix, first auxiliary bitmap, second auxiliary bitmap and third auxiliary bitmap generated by the algorithm preprocessing module; A second storage module, wherein the second storage module is configured as: The partial sum result and the first address result are obtained and saved.

10. The sparse matrix accelerator based on block packing algorithm according to claim 9, characterized in that: The computing module also includes: An output module, wherein the output module is configured to: Obtain the partial sum result and the first address result; The partial sum result and the first address result are output to the second storage module.

Images