Data processing acceleration method and device for sparse matrix multiplication operator

Abstract

The invention provides a data processing acceleration method and device for a sparse matrix multiplication operator, and the method comprises the steps: obtaining the sparse matrix multiplication operator needing to be processed in a data processing process, compressing a sparse matrix, and enabling a storage form of the compressed sparse matrix to comprise a non-zero element array, a row pointer array, a global column index array and a local column index array. The global column indexes record the column indexes of non-zero elements in the sparse matrix in a group of continuous preset number of sparse rows divided by the sparse matrix, and the global column indexes in each column index array are not repeated. The local column index records a position index of each non-zero element in the global column index array. And accessing corresponding row data in the dense matrix by using the global column index and reading the row data to the temporary cache region. And accessing corresponding row data in the dense matrix in the temporary cache region by using the local column index. And calculating and accumulating products of the non-zero elements and corresponding row data in the dense matrix, and outputting a result matrix. The data processing speed can be increased, and the memory access overhead can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of sparse matrix multiplication, and in particular to a data processing acceleration method and device for a sparse matrix multiplication operator. Background Art

[0002] Sparse matrices are an important data structure in computer science, engineering, and scientific computing, especially when dealing with large-scale problems such as social network analysis, image processing, and solving linear equations. Sparse matrices are characterized by the fact that most of their elements are zero, which makes traditional matrix multiplication algorithms inefficient. Therefore, specialized algorithms and data structures can be used to improve efficiency when storing and computing.

[0003] SpMM (Sparse Matrix-Matrix Multiplication) is a multiplication operation between a sparse matrix and a dense matrix. In the sparse matrix multiplication (SpMM) calculation, although the compression method of the CSR (Compressed Sparse Row) format effectively reduces the storage requirements, its limitations in data access lead to low data locality. The CSR format stores non-zero elements by row, which may cause discontinuous memory access when processing the non-zero elements of each row, especially when the non-zero elements of the rows of the sparse matrix are unevenly distributed. Since the non-zero elements of the sparse matrix may be scattered between different rows and columns, the index lookup during the calculation process may cause the CPU to frequently jump from one memory address to another. When data access efficiency is low, the CPU will spend more time waiting for data to be loaded from the main memory to the cache, which reduces the cache hit rate and affects the computational efficiency of sparse matrix multiplication. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a data processing acceleration method and device for a sparse matrix multiplication operator to eliminate or improve the defect of the existing data compression method in the prior art that the memory access is too frequent during the operation of the sparse matrix multiplication operator, resulting in low data processing efficiency.

[0005] A first aspect of the present invention provides a method for accelerating data processing of a sparse matrix multiplication operator, the method comprising the following steps:

[0006] In the data processing process, a sparse matrix multiplication operator to be processed is obtained, wherein the sparse matrix multiplication operator includes a multiplication operation of a sparse matrix and a dense matrix;

[0007] The sparse matrix is ​​compressed to store the non-zero elements in the sparse matrix and their corresponding positions; the storage format of the compressed sparse matrix includes a non-zero element array, a row pointer array, a global column index array and a local column index array; wherein the non-zero element array records all non-zero elements in the sparse matrix in a row-first order; the row pointer array sequentially records the position of the first non-zero element of each row in the sparse matrix in the non-zero element array; the global column index array includes a plurality of column index arrays, each column index array includes a plurality of global column indexes to record the column index of the non-zero elements in a set of continuous preset number of sparse rows divided by the sparse matrix, and the global column index in each column index array is not repeated; the local column index array includes a plurality of local column indexes to record the position index of each non-zero element in the non-zero element array in the global column index array;

[0008] Using the global column index to access the row data in the dense matrix corresponding to the multiplication calculation performed with the global index, and reading the corresponding row data in the dense matrix into a temporary buffer area;

[0009] Using the local column index to access the row data corresponding to the multiplication calculation in the dense matrix in the temporary buffer area;

[0010] Access the corresponding relationship of the row data corresponding to the multiplication calculation in the dense matrix according to the local column index, calculate the product of the corresponding non-zero elements in the sparse matrix and the row data corresponding to the multiplication calculation in the dense matrix and accumulate them, and output the result matrix.

[0011] In some embodiments of the present invention, the method of establishing the global column index array includes:

[0012] Obtain a column index array of the sparse matrix to be processed, wherein the column index array records the column indices of non-zero elements in the sparse matrix;

[0013] Divide the column index array corresponding to all non-zero elements in the sparse matrix into a plurality of column index arrays, taking a continuous preset number of sparse rows in the sparse matrix as a group;

[0014] Traverse all non-zero column indices in each column index array, delete duplicate column indices, and obtain the global column index array corresponding to the sparse matrix.

[0015] In some embodiments of the present invention, calculating the product of corresponding non-zero elements in the sparse matrix and corresponding row data in the dense matrix and accumulating the product includes:

[0016] Record the product of the corresponding non-zero elements in the sparse matrix and the row data corresponding to the multiplication calculation in the dense matrix into the intermediate calculation hash table;

[0017] If the non-zero element and the corresponding row data to be multiplied in the subsequent multiplication calculation are the same as the non-zero element and the corresponding row data recorded in the intermediate operation hash table, directly call the product of the non-zero element and the corresponding row data;

[0018] If it does not exist, perform the multiplication and store the result in the hash table.

[0019] In some embodiments of the present invention, it further includes:

[0020] Before calculating the sparse matrix multiplication operator, if it is detected that the size of the sparse matrix or the dense matrix exceeds a preset threshold, the sparse matrix or the dense matrix is ​​divided into multiple matrix blocks, and the multiple matrix blocks are allocated to multiple threads for parallel calculation.

[0021] In some embodiments of the present invention, the size of the temporary buffer is dynamically adjusted according to the number of non-zero elements in the sparse matrix, the dimension of the dense matrix and the available memory size of the currently executed task.

[0022] In some embodiments of the present invention, using the global column index to access the corresponding row data in the dense matrix, and reading the corresponding row data in the dense matrix into a temporary buffer area, includes:

[0023] The temporary buffer area is divided into a plurality of sub-areas, a plurality of global column indexes access the corresponding row data in the dense matrix in parallel, and the corresponding row data in the dense matrix is ​​read into the corresponding plurality of sub-areas.

[0024] In some embodiments of the present invention, a hash table is used to store non-zero elements and their corresponding positions in the sparse matrix.

[0025] The second aspect of the present invention provides a data processing acceleration device for a sparse matrix multiplication operator, comprising a processor, a memory, and computer instructions stored in the memory, wherein the processor is used to execute the computer instructions, and when the computer instructions are executed, the device implements the steps of any of the methods described above.

[0026] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0027] A fourth aspect of the present invention provides a computer program product, comprising a computer program, which implements the steps of any of the methods described above when executed by a processor.

[0028] The beneficial effects of the present invention are at least:

[0029] The present invention provides a data processing acceleration method and device for a sparse matrix multiplication operator, the method comprising: obtaining a sparse matrix multiplication operator, wherein the compressed storage form of the sparse matrix comprises a non-zero element array, a row pointer array, a global column index array and a local column index array. The global column index records the column index of the non-zero elements in a group of continuous preset number of sparse rows divided by the sparse matrix, and the global column index is not repeated in each column index array. The local column index records the position index of each non-zero element in the global column index array. The global column index is used to access the corresponding row data in the dense matrix and read it into a temporary buffer area. The local column index is used to access the corresponding row data in the dense matrix in the temporary buffer area. The product of the corresponding non-zero element in the sparse matrix and the corresponding row data in the dense matrix is ​​calculated and accumulated, and the result matrix is ​​output. The present invention proposes a new compressed storage format for sparse matrices and a sparse matrix multiplication calculation method based on the new compressed storage format, which can significantly improve the data locality of sparse matrix multiplication calculation, reduce the frequency of access to data in the memory during sparse matrix multiplication calculation, reduce memory access overhead, and improve data processing efficiency.

[0030] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0031] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0033] Figure 1 The present invention is a flowchart of a method for accelerating data processing of a sparse matrix multiplication operator in one embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of an example of a sparse matrix in another embodiment of the present invention.

[0035] Figure 3 Schematic diagram of the CSR compression storage format corresponding to the sparse matrix in another embodiment of the present invention.

[0036] Figure 4 It is a schematic diagram of the operation process of a sparse matrix multiplied by a dense matrix (SpMM) operator in another embodiment of the present invention.

[0037] Figure 5 It is a schematic diagram of a Grouped Compressed Sparse Row Format (GCSR) in another embodiment of the present invention.

[0038] Figure 6 This is a calculation flow chart of SpMM based on the GCSR compression storage format in another embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0040] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0041] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0042] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0043] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0044] The sparse matrix multiplication operator is an operation of multiplying a sparse matrix by a dense matrix. A sparse matrix is ​​a matrix in which most elements are zero and only a small number of non-zero elements are present. In order to save storage space and improve computational efficiency, sparse matrices use special data structures (such as CSR, CSC, etc.) to store these non-zero elements. A dense matrix is ​​a matrix in which most elements are non-zero, and is usually stored using a traditional two-dimensional array or matrix representation method. CSR (Compressed Sparse Row) cannot fully exploit the data locality in a sparse matrix, resulting in low data access efficiency when calculating sparse matrix multiplication (SpMM). The present invention proposes a sparse matrix multiplication calculation method based on a new compressed storage format, which can improve the data locality of sparse matrix multiplication calculations, reduce the frequency of access to data in memory during sparse matrix multiplication calculations, reduce memory access overhead, and improve data processing efficiency.

[0045] An embodiment of the present invention provides a method and device for accelerating data processing of a sparse matrix multiplication operator, such as Figure 1 As shown, the method includes the following steps S101 to S105:

[0046] Step S101: obtaining a sparse matrix multiplication operator to be processed during data processing, where the sparse matrix multiplication operator includes multiplication operations of a sparse matrix and a dense matrix.

[0047] For example, when data is input into a neural network used to perform computer vision tasks or natural language processing tasks, a sparse matrix is ​​generated during the calculation process to store and process data information.

[0048] Step S102: compress the sparse matrix to store the non-zero elements and their corresponding positions in the sparse matrix. The storage form after the sparse matrix is ​​compressed includes a non-zero element array, a row pointer array, a global column index array and a local column index array. Among them, the non-zero element array records all non-zero elements in the sparse matrix in a row-first order. The row pointer array records the position of the first non-zero element of each row in the sparse matrix in the non-zero element array in sequence. The global column index array includes a plurality of column index arrays, each column index array includes a plurality of global column indexes to record the column index of the non-zero elements in a set of continuous preset number of sparse rows divided by the sparse matrix in the sparse matrix, and the global column index in each column index array is not repeated. The local column index array includes a plurality of local column indexes to record the position index of each non-zero element in the non-zero element array in the global column index array.

[0049] Step S103: using the global column index to access the row data in the dense matrix corresponding to the multiplication calculation with the global index, and reading the corresponding row data in the dense matrix into a temporary buffer area.

[0050] Step S104: using the local column index to access the row data corresponding to the multiplication calculation in the dense matrix in the temporary buffer area.

[0051] Step S105: access the corresponding relationship of the row data corresponding to the multiplication calculation in the dense matrix according to the local column index, calculate the product of the corresponding non-zero elements in the sparse matrix and the row data corresponding to the multiplication calculation in the dense matrix, accumulate them, and output the result matrix.

[0052] Furthermore, in step S102, the method for establishing a global column index array includes:

[0053] Get the column index array of the sparse matrix to be processed. The column index array records the column indices of non-zero elements in the sparse matrix.

[0054] The column index array corresponding to all non-zero elements in the sparse matrix is ​​divided into multiple column index arrays by taking a preset number of consecutive sparse rows in the sparse matrix as a group.

[0055] Traverse all non-zero column indices in each column index array, delete duplicate column indices, and obtain the global column index array corresponding to the sparse matrix.

[0056] Specifically, the global column index and the local column index form a double-layer index structure. The global column index records the column index of the non-zero element in the sparse matrix, and the local column index points to the position of the non-zero element in the global column index, that is, a global column index is equivalent to collecting the non-zero elements of the same column in a preset number of consecutive sparse rows in the sparse matrix.

[0057] The operation rule of the sparse matrix multiplication operator is to multiply the rows of the sparse matrix by the columns of the dense matrix element by element and sum them to obtain the result matrix. During its operation, the non-zero elements in the same column of the sparse matrix need to access the row data of the same row in the dense matrix. Therefore, the double-layer index structure composed of the global column index and the local column index proposed in this embodiment can reduce the frequent access to the dense matrix compared to the existing compression form, and improve the data processing efficiency.

[0058] In some embodiments, calculating the product of corresponding non-zero elements in the sparse matrix and corresponding row data in the dense matrix and accumulating the product includes:

[0059] The product of the corresponding non-zero elements in the sparse matrix and the corresponding row data in the dense matrix is ​​recorded in the intermediate calculation hash table.

[0060] If the non-zero element and the corresponding row data to be multiplied in the subsequent multiplication calculation are the same as the non-zero element and the corresponding row data recorded in the intermediate operation hash table, the product of the non-zero element and the corresponding row data is directly called.

[0061] If it does not exist, perform the multiplication and store the result in the hash table.

[0062] Among them, a hash table is a data structure used to quickly find and store data. It supports efficient insertion, deletion, and search operations by mapping data into an array of a fixed size. If elements in the matrix are frequently added or deleted, using this dynamic data structure can avoid reserving excess memory, thereby optimizing memory usage.

[0063] In some embodiments, it also includes:

[0064] Before calculating the sparse matrix multiplication operator, if it is detected that the size of the sparse matrix or the dense matrix exceeds a preset threshold, the sparse matrix or the dense matrix is ​​divided into a plurality of matrix blocks, and the plurality of matrix blocks are allocated to a plurality of threads for parallel calculation.

[0065] Specifically, the preset threshold is set to control memory usage and computational complexity. When the size of the matrix exceeds the preset threshold, the matrix is ​​divided into blocks and the matrix blocks are assigned to a single thread, so that multiple threads can calculate in parallel, which can reduce the computation time and improve the computation efficiency.

[0066] In some embodiments, the size of the temporary buffer is dynamically adjusted according to the number of non-zero elements in the sparse matrix, the dimension of the dense matrix, and the available memory size of the currently executed task.

[0067] In some embodiments, using a global column index to access corresponding row data in a dense matrix, and reading corresponding row data in the dense matrix into a temporary buffer area, includes:

[0068] The temporary buffer area is divided into a plurality of sub-areas, a plurality of global column indexes access corresponding row data in the dense matrix in parallel, and the corresponding row data in the dense matrix is ​​read into the corresponding plurality of sub-areas.

[0069] In some embodiments, a hash table is used to store non-zero elements and their corresponding positions in a sparse matrix.

[0070] Correspondingly, the above embodiments may be executed in a CPU or in a GPU.

[0071] When the CPU is executing, the main memory is used to store the data required for the program to run, such as sparse matrices, dense matrices, and the result matrix obtained. The temporary buffer area is the CPU's cache. The cache in the CPU is a fast storage area between the main memory and the CPU register, which is used to temporarily store frequently accessed data. The CPU is generally configured with multiple levels of cache (L1, L2, L3). In the above embodiment, the corresponding row data in the dense matrix is ​​accessed using the global column index, and the corresponding row data in the dense matrix is ​​read into the temporary buffer area. The local column index is then used to access the corresponding row data in the dense matrix in the temporary buffer area. Avoid re-reading from the main memory and reduce delays. The process of calculating the product of the corresponding non-zero elements in the sparse matrix and the corresponding row data in the dense matrix and accumulating them is completed in the register. The register is an ultra-high-speed storage unit inside the CPU, which is specifically used to store the currently executed instructions and related operands.

[0072] When executing in a GPU, the temporary cache area can be a high-speed scratchpad memory. A high-speed scratchpad memory is a hardware cache device equipped on a GPU, located between the global memory and the registers. The caching of data on the high-speed scratchpad memory requires manual programming by the programmer.

[0073] Another embodiment of the present invention provides a method and device for accelerating data processing of a sparse matrix multiplication operator. Specifically, this embodiment proposes a new sparse matrix compression method and a corresponding sparse matrix multiplication operator calculation method.

[0074] Sparse matrix multiplication is widely used in fields such as deep learning and scientific computing. In actual application scenarios, a large number of elements in the sparse matrix are zero elements, but these zero elements do not contribute to the calculation results. Therefore, the sparse matrix can be compressed and stored, that is, only the non-zero elements in the sparse matrix and the corresponding position information are stored. Compressed Sparse Row (CSR) is a commonly used sparse matrix compression storage format. The CSR format contains three arrays: non-zero elements, column indexes, and row pointers. The non-zero element array stores all non-zero elements in a row-first manner; the column index array records the column index of the non-zero elements in the original matrix; the row pointer array records the position of the first non-zero element of each row in the non-zero element array, and the last element in the row pointer array records the number of all non-zero elements. An example of a sparse matrix is Figure 2 As shown, Figure 2 The CSR format corresponding to the sparse matrix is ​​as follows Figure 3 shown.

[0075] Sparse matrix multiplication by dense matrix (SpMM) is a sparse operator commonly used in deep learning and scientific computing. For example, SpMM operators are often used in sparse large models. An example of a SpMM operator is A*B=C. Figure 4 As shown in the figure, matrix A is a sparse matrix, matrix B is a dense matrix, and output matrix C is a dense matrix. The CSR compression storage format cannot fully exploit the data locality in the sparse matrix, resulting in low data access efficiency when calculating the sparse matrix multiplication SpMM. Figure 4 For example, the non-zero element b in matrix A will be multiplied with all the elements of the second row of dense matrix B (rows start from 0) in sequence, so all the elements of the second row of dense matrix B will be accessed, and the non-zero elements f, m, r, and v in matrix A will also be multiplied with all the elements of the second row of dense matrix B in sequence. However, the CSR format is compressed and stored in a row-first manner. After accessing the element b in matrix A, the elements c, d, e, and f will be accessed in sequence, and the access of elements b and f will be separated by three elements c, d, and e; accordingly, after accessing the second row of dense matrix B, the third, fifth, and 0th rows of matrix B must be accessed in the middle before the second row of dense matrix B is accessed again. At this time, the second row of matrix B may have been swapped out from the cache device. Therefore, the characteristics of CSR row-first compression storage result in poor locality of data access to matrix B.

[0076] The present invention designs a new sparse matrix compression storage format, which is a grouped compressed sparse row format (Grouped Compressed Sparse Row, GCSR). The non-zero element array and row pointer array of the GCSR format are the same as those of the CSR format. The difference is that the GCSR format groups the sparse rows of the original sparse matrix, and then eliminates the repeated global column indexes of non-zero elements in the group through a global and local double-layer column index data structure, thereby improving the data access efficiency in sparse matrix multiplication calculations.

[0077] The specific design of the GCSR format (such as Figure 5 As shown in the figure, the non-zero element and row pointer array are consistent with the CSR format; then, the sparse rows of the sparse matrix are grouped, and each consecutive row is set as a group. Figure 2 Take sparse matrix as an example, Figure 5 The plan will Figure 2 Every 4 rows of the sparse matrix are set as a group, and the entire sparse matrix is ​​divided into 2 groups; then, the column indexes of all non-zero elements in each group are traversed once, thereby deleting duplicate column indexes and obtaining the global column index array of each group; finally, the column index of each row of non-zero elements is compared with the global column index array of the group to obtain the local column index of each row, which is used to identify the position of the column index of each non-zero element in the global column index array. In general, the GCSR format includes 4 arrays: non-zero elements, row pointers, global column indexes, and local column indexes.

[0078] The present invention takes the GPU processor as an example to explain the calculation process of the SpMM operator based on the GCSR format. The GPU graphics processor is a commonly used computing acceleration device. On the GPU, the scratch pad is a hardware cache device located between the global memory and the register. The cache of data on the scratch pad requires manual programming by the programmer. The calculation process of the SpMM operator based on the GCSR format (such as Figure 6 As shown) as follows:

[0079] (1) Use the global column index in the GCSR format to access the corresponding row data of the B matrix; (2) Read the row data of the B matrix accessed in step (1) from the global memory to the high-speed scratchpad memory for caching; (3) Use the local column index to access the row data of the B matrix cached in the high-speed scratchpad memory and read the data into the register; (4) Multiply the row data of the B matrix with the corresponding non-zero elements in the sparse matrix A, accumulate the intermediate results, and finally output the results to the matrix C. Figure 5 Only Figure 4 The calculation process of group 0 in the GCSR format is the same as that of group 1, but it is not Figure 6 Displayed in.

[0080] In order to verify the benefits of the GCSR format in data locality, the following verification method is adopted: First, assume that the non-zero elements in each row of the sparse matrix are approximately uniformly distributed and independent of each other, and its sparsity is approximately equal to the overall sparsity of the matrix r (r[0,1]), that is, the probability that each row element is zero is r. At the same time, suppose that in the SpMM operator, the sparse matrix A has m rows and k columns, the dense matrix B has k rows and n columns, and the output matrix C has m rows and n columns. Then the number of non-zero elements in each row of the sparse matrix A is N = k*(1-r). Further assume that the group size in the GCSR format is s, that is, each group contains s rows, then the expected size of the global column index array corresponding to each group is, where the integer i[0,s-1]. The expected size of the column index array corresponding to the s rows of the sparse matrix in the CSR format is sN. It is easy to know that <=sN. Based on the CSR format, the memory access overhead of the dense matrix B in the SpMM operator is mnN. And Figure 5 The calculation method of the SpMM operator based on the GCSR format is given in . The memory access overhead of the dense matrix B is mn() / s. It is easy to know that mn() / s<=mnN. Compared with the CSR format, the GCSR format can significantly reduce the memory access overhead of the dense matrix B in the SpMM operator. From the formula mn() / s<=mnN, it can be seen that when the sparsity r is lower, the GCSR format has a more significant effect in reducing the memory access overhead; when the group size s is larger, the GCSR format has a more significant effect in reducing the memory access overhead. In actual situations, the value of s is set according to the capacity of the high-speed temporary storage memory, which is usually much smaller than the number of rows m of the sparse matrix.

[0081] In summary, the present invention proposes a data processing acceleration method and device for a sparse matrix multiplication operator, the method comprising: obtaining a sparse matrix multiplication operator, compressing the sparse matrix, and the storage form of the compressed sparse matrix includes a non-zero element array, a row pointer array, a global column index array, and a local column index array. The global column index records the column index of the non-zero elements in the sparse matrix in a set of continuous preset number of sparse rows divided by the sparse matrix, and the global column index is not repeated in each column index array. The local column index records the position index of each non-zero element in the global column index array. The global column index is used to access the corresponding row data in the dense matrix and read it into a temporary buffer area. The local column index is used to access the corresponding row data in the dense matrix in the temporary buffer area. The product of the corresponding non-zero element in the sparse matrix and the corresponding row data in the dense matrix is ​​calculated and accumulated, and the result matrix is ​​output. The present invention proposes a new compression format for sparse matrices, which groups the sparse rows of the sparse matrix, sets a double-layer column index data structure of global column index and local column index in each group, and eliminates the repeated global column index of non-zero elements in the group. The SpMM operator calculation method based on this format reduces the frequent access to the main memory during the calculation process. Compared with the prior art, the SpMM operator calculation method based on the new format can reduce the frequency of access to data in the memory during calculation, reduce memory access overhead, and improve data processing efficiency.

[0082] Corresponding to the above method, the present invention also provides an apparatus / system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system implements the steps of the method described above.

[0083] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0084] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0085] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0086] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for accelerating data processing of a sparse matrix multiplication operator, characterized in that: The method comprises the following steps: In the data processing process, a sparse matrix multiplication operator to be processed is obtained, wherein the sparse matrix multiplication operator includes a multiplication operation of a sparse matrix and a dense matrix; The sparse matrix is ​​compressed to store the non-zero elements in the sparse matrix and their corresponding positions; the storage format of the compressed sparse matrix includes a non-zero element array, a row pointer array, a global column index array and a local column index array; wherein the non-zero element array records all non-zero elements in the sparse matrix in a row-first order; the row pointer array sequentially records the position of the first non-zero element of each row in the sparse matrix in the non-zero element array; the global column index array includes a plurality of column index arrays, each column index array includes a plurality of global column indexes to record the column index of the non-zero elements in a set of continuous preset number of sparse rows divided by the sparse matrix, and the global column index in each column index array is not repeated; the local column index array includes a plurality of local column indexes to record the position index of each non-zero element in the non-zero element array in the global column index array; Using the global column index to access the row data in the dense matrix corresponding to the multiplication calculation performed with the global index, and reading the corresponding row data in the dense matrix into a temporary buffer area; Using the local column index to access the row data corresponding to the multiplication calculation in the dense matrix in the temporary buffer area; Access the corresponding relationship of the row data corresponding to the multiplication calculation in the dense matrix according to the local column index, calculate the product of the corresponding non-zero elements in the sparse matrix and the row data corresponding to the multiplication calculation in the dense matrix and accumulate them, and output the result matrix.

2. The data processing acceleration method of the sparse matrix multiplication operator according to claim 1, characterized in that: The method of establishing the global column index array includes: Obtain a column index array of the sparse matrix to be processed, wherein the column index array records the column indices of non-zero elements in the sparse matrix; Divide the column index array corresponding to all non-zero elements in the sparse matrix into a plurality of column index arrays, taking a continuous preset number of sparse rows in the sparse matrix as a group; Traverse all non-zero column indices in each column index array, delete duplicate column indices, and obtain the global column index array corresponding to the sparse matrix.

3. The data processing acceleration method of the sparse matrix multiplication operator according to claim 1, characterized in that: Calculating the product of the corresponding non-zero elements in the sparse matrix and the corresponding row data in the dense matrix and accumulating them, including: Record the product of the corresponding non-zero elements in the sparse matrix and the row data corresponding to the multiplication calculation in the dense matrix into the intermediate calculation hash table; If the non-zero element and the corresponding row data to be multiplied in the subsequent multiplication calculation are the same as the non-zero element and the corresponding row data recorded in the intermediate operation hash table, directly call the product of the non-zero element and the corresponding row data; If it does not exist, perform the multiplication and store the result in the hash table.

4. The data processing acceleration method of the sparse matrix multiplication operator according to claim 1, characterized in that: Also includes: Before calculating the sparse matrix multiplication operator, if it is detected that the size of the sparse matrix or the dense matrix exceeds a preset threshold, the sparse matrix or the dense matrix is ​​divided into multiple matrix blocks, and the multiple matrix blocks are allocated to multiple threads for parallel calculation.

5. The data processing acceleration method of the sparse matrix multiplication operator according to claim 1, characterized in that: The size of the temporary buffer is dynamically adjusted according to the number of non-zero elements in the sparse matrix, the dimension of the dense matrix and the available memory size of the currently executed task.

6. The data processing acceleration method of the sparse matrix multiplication operator according to claim 1, characterized in that: Accessing corresponding row data in the dense matrix using the global column index, and reading corresponding row data in the dense matrix into a temporary buffer area, comprising: The temporary buffer area is divided into a plurality of sub-areas, a plurality of global column indexes access the corresponding row data in the dense matrix in parallel, and the corresponding row data in the dense matrix is ​​read into the corresponding plurality of sub-areas.

7. The data processing acceleration method of the sparse matrix multiplication operator according to claim 1, characterized in that: A hash table is used to store non-zero elements and their corresponding positions in the sparse matrix.

8. A data processing acceleration device for a sparse matrix multiplication operator, comprising a processor, a memory, and computer instructions stored in the memory, characterized in that: The processor is used to execute the computer instructions, and when the computer instructions are executed, the device implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images