An adaptive balanced partitioning method based on GPU-accelerated sparse-dense matrix multiplication

Abstract

The application discloses a self-adaptive balanced division method based on GPU acceleration sparse-dense matrix multiplication, and comprises the following steps: analyzing a sparse matrix, determining long rows affecting parallel performance; dividing the long rows into equal "parts"; storing the divided sparse matrix by using a new storage format; loading all nonzero elements in the "parts" to a shared memory on a GPU; consuming the nonzero elements in the shared memory and multiplying corresponding elements of a dense matrix; adding the parts from the same row and obtaining a final result. The application aims at the current situation that long rows lead to low parallel efficiency when a row division mechanism is used in GPU acceleration sparse-dense multiplication, and proposes a self-adaptive balanced division method based on the structure characteristics of a GPU. The "parts" are used to replace rows as parallel granularity, and the size of the "parts" is controlled within a reasonable range to ensure that a thread group has enough threads to effectively parallelize the nonzero element task, and the similar size of the "parts" avoids the load imbalance among the thread groups.

Description

Technical Field

[0001] This invention belongs to the field of high-performance parallel computing, and in particular relates to an adaptive balanced partitioning method based on GPU-accelerated sparse-dense matrix multiplication. Background Technology

[0002] Graphics Processing Units (GPUs) have a large number of computing cores, making them well-suited for large-scale parallel computing applications such as video image processing, machine learning, and large-scale sparse matrix operations. Modern GPUs employ a Single Instruction, Multithreaded (SIMT) model, where numerous threads concurrently execute the same instructions on different data. CUDA is the most popular parallel computing architecture, which allows developers to directly access GPU hardware resources and improve computational and memory access efficiency through a combination of hardware and software, thereby solving more complex problems.

[0003] Large-scale sparse matrix operations have always been a key research focus in the field of high-performance computing. Sparse-dense matrix multiplication is represented as C=AB, where A is a sparse matrix and B and C are dense matrices. Sparse-dense matrix multiplication is widely used in real-world applications and consumes a significant amount of time, such as in data mining and image neural networks (GNNs).

[0004] Currently, research projects using GPUs to accelerate sparse-dense matrix multiplication operations mainly include Tsinghua University's GE-SpMM, California State University's GraphBLAST, MIT's AsPT, Almería University's Fast-SpMM, and NVIDIA's official math library cuSPARSE.

[0005] Current solutions draw on the experience of GPU-accelerated SpMV (Sparse Vector Multiplication), using a row partitioning strategy as the basis. The idea behind this strategy is to use a row of the sparse matrix as the parallel granularity, that is, to hand over all non-zero elements of a row to a group of threads (warp) in the GPU for processing. The SpMM kernel based on row partitioning mainly includes a loading phase and a consumption phase. In the loading phase, each thread in the thread group loads a non-zero element from that row in parallel. In the consumption phase, each thread in the thread group is responsible for a column of the dense matrix, and performs a product operation between the loaded non-zero element and the corresponding element in that column.

[0006] The only difference between SpMV and SpMM operations is that the vector is replaced by a dense matrix with multiple columns. However, when designing the SpMM kernel, it is necessary to ensure the reuse of non-zero elements in the sparse matrix and to merge memory accesses of multiple columns in the dense matrix. These issues do not need to be considered in the SpMV kernel. In addition, during the consumption phase, each thread needs to process all non-zero elements in a row one by one. Therefore, the irregular distribution of the sparse matrix itself will exacerbate the load imbalance problem in the SpMM kernel.

[0007] While some work has attempted to address these issues, it still falls short of effectively utilizing GPU computational and memory resources when processing various types of sparse matrices. The main reasons include: the large differences in non-zero elements between rows of a sparse matrix make it difficult to guarantee load balancing among thread groups; for some long rows, GPUs lack sufficient on-chip memory to effectively ensure data reuse; when the number of non-zero elements in a row exceeds the number of threads in a thread group, multiple iterations are required to process all elements, resulting in low parallel efficiency; and some optimization methods require complex preprocessing, are only applicable to a few matrices, and lack general applicability. Summary of the Invention

[0008] The purpose of this invention is to provide an adaptive balanced partitioning method for GPU-accelerated sparse-dense matrix multiplication, which aims to solve the problems of uneven load caused by the difference in the number of non-zero elements between rows in the current GPU-accelerated sparse-dense multiplication method, which uses row partitioning mechanism to treat a row of the sparse matrix as the parallel granularity, and the problem that processing long rows requires multiple iterations, resulting in low parallel efficiency.

[0009] This invention is implemented as follows: An adaptive balanced partitioning method based on GPU-accelerated sparse-dense matrix multiplication includes the following steps:

[0010] Step 1: Analyze the sparse matrix to determine the long rows that affect parallel performance;

[0011] Step 2: Divide the long line into equal-sized "parts";

[0012] Step 3: Store the partitioned sparse matrix using the new storage format;

[0013] Step 4: Load all non-zero elements in the "part" into shared memory on the GPU;

[0014] Step 5: Consume the non-zero elements in the shared memory and multiply them by the corresponding elements of the dense matrix;

[0015] Step 6: Add the partial sums from the same row to get the final result.

[0016] Furthermore, in step one, by analyzing the structural characteristics of the sparse matrix itself, rows with a number of non-zero elements greater than the number of threads in a thread group are selected, and these rows are identified as long rows and used as candidates for further balanced partitioning.

[0017] Furthermore, step two involves dividing the long line into smaller "parts," the size of which is controlled to be smaller than the size of a thread group in the GPU.

[0018] Furthermore, in step three, the partitioned sparse matrix is ​​stored in a new storage format that uses "parts" instead of rows as the basic unit.

[0019] Furthermore, step four is the loading phase, where all non-zero elements in the "part" are loaded into shared memory on the GPU. The specific steps are as follows:

[0020] Step 1: Each thread in the thread group determines its assigned "part" location based on the new storage format;

[0021] Step 2: Each thread is responsible for a non-zero element in the "part", loading the value of the non-zero element and its column into shared memory.

[0022] Furthermore, step five is the consumption phase, which consumes the non-zero elements in the shared memory and multiplies them with the corresponding elements of the dense matrix. The specific steps are as follows:

[0023] Step 1: One thread in the thread group is responsible for calculating one column of the dense matrix;

[0024] Step 2: Calculate the position of the corresponding element in the dense matrix based on the column containing the non-zero element in the shared memory;

[0025] Step 3: Iterate through all non-zero elements and calculate the partial sum.

[0026] Furthermore, step six involves adding the partial sums from the same row to obtain the final result and returning it.

[0027] The beneficial effects of this invention are as follows:

[0028] (1) In this invention, “part” is used instead of “row” as the parallel granularity, and the size of “part” is controlled within a certain range according to hardware resources. The on-chip memory has enough space to ensure data reusability, and the thread group has enough threads to perform parallel computing tasks, thereby improving parallel efficiency.

[0029] (2) The “parts” obtained by the balanced partitioning method of the present invention are similar in size, thereby eliminating the load imbalance caused by the huge difference between rows in the past. The non-zero elements in the “parts” come from the same row of the sparse matrix, which can also ensure the merged memory access of the dense matrix, thereby improving the memory access efficiency.

[0030] (3) This invention does not require complex preprocessing, is applicable to different types of matrices, and has universality. Attached Figure Description

[0031] Figure 1 This is a flowchart of an adaptive balanced partitioning method based on GPU-accelerated sparse-dense matrix multiplication provided in an embodiment of the present invention.

[0032] Figure 2 This is an example of a sparse matrix storage format provided in the embodiments of the present invention.

[0033] Figure 3 This is an example of a GPU-based sparse-dense matrix multiplication kernel architecture provided in this embodiment of the invention.

[0034] Figure 4 This is an example of the effect of using the adaptive balanced partitioning method provided in the embodiments of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0036] Figure 1 The flowchart of the adaptive balanced partitioning method based on GPU-accelerated sparse-dense matrix multiplication of the present invention is shown in the figure. The present invention is implemented as follows: An adaptive balanced partitioning method based on GPU-accelerated sparse-dense matrix multiplication includes the following steps:

[0037] S101: Analyze the sparse matrix to determine the long rows that affect parallel performance;

[0038] S102: Divide the long line into equal-sized "parts";

[0039] S103: Store the partitioned sparse matrix using a new storage format;

[0040] S104: Load all non-zero elements in the “part” into shared memory on the GPU;

[0041] S105: Consume the non-zero elements in the shared memory and multiply them with the corresponding elements of the dense matrix;

[0042] S106: Add the partial sums from the same row to get the final result.

[0043] Furthermore, S101 analyzes the structural characteristics of the sparse matrix itself, selects rows with a number of non-zero elements greater than the number of threads in a thread group, identifies these rows as long rows, and uses them as candidates for further balanced partitioning.

[0044] Furthermore, S102 divides the long line into smaller “parts”, the size of which is controlled to be smaller than the size of a thread group in the GPU.

[0045] Furthermore, S103 stores the partitioned sparse matrix using a new storage format that replaces rows as the basic unit.

[0046] Furthermore, S104 is the loading phase, and the specific steps for loading all non-zero elements in the "part" into the shared memory on the GPU are as follows:

[0047] Step 1: Each thread in the thread group determines its assigned "part" location based on the new storage format;

[0048] Step 2: Each thread is responsible for a non-zero element in the "part", loading the value of the non-zero element and its column into shared memory.

[0049] Furthermore, S105 is the consumption phase, which consumes non-zero elements in the shared memory and performs multiplication with the corresponding elements of the dense matrix. The specific steps are as follows:

[0050] Step 1: One thread in the thread group is responsible for calculating one column of the dense matrix;

[0051] Step 2: Calculate the position of the corresponding element in the dense matrix based on the column containing the non-zero element in the shared memory;

[0052] Step 3: Iterate through all non-zero elements and calculate the partial sum.

[0053] Furthermore, S106 adds the partial sums from the same row to obtain the final result and returns it.

[0054] Example: The SNAP dataset is a sparse matrix benchmark containing 66 sparse matrices. These sparse matrices are formed from efficient graphs across various domains, which often require sparse-dense matrix operations in real-world scenarios. The total number of rows in these 66 sparse matrices ranges from 1005 to 4847571, with an average of 1.58 to 32.53 non-zero elements per row. This dataset has also been used for performance testing of previous GPU-accelerated sparse-dense matrix operation kernel designs, thus effectively validating the performance improvements of our proposed method. All sparse matrices are computed with dense matrices having 32, 64, 128, and 256 columns, respectively, to further illustrate the effectiveness of our method. The steps of the GPU-accelerated adaptive balanced partitioning method for sparse-dense matrix multiplication implemented on the SNAP dataset are as follows.

[0055] Step 1: Analyze the sparse matrix to determine the long rows that affect parallel performance;

[0056] Based on the number of threads in the GPU thread group, a threshold is set for the number of parallel non-zero element tasks. Lines with non-zero elements exceeding the GPU thread group size are identified as long lines, and these long lines are selected for subsequent balancing operations. For example... Figure 2On the left, assuming the GPU thread group size is 4, the number of non-zero elements in the first row exceeds 4, so we consider this row to be a long row and it needs to be divided.

[0057] Step 2: Divide the long line into equal-sized "parts";

[0058] Based on the GPU's thread group size, the long row is divided into equally sized "parts," and the size of each "part" must be smaller than the thread group size, such as... Figure 2 Above, the first row has 5 non-zero elements, so it needs to be divided into two "parts".

[0059] Step 3: Store the partitioned sparse matrix using the new storage format;

[0060] After balancing the matrix, "parts" replace rows as the basic units, such as... Figure 2 The new storage format shown below has four parameters: row_Ind represents the row where the "part" is located, part_Ind represents the starting position of the "part" (e.g., the starting position of the first part is 0), col_Ind represents the column where each non-zero element in the "part" is located, and value represents the value of each non-zero element.

[0061] Step 4: Load all non-zero elements in the "part" into shared memory on the GPU;

[0062] This step is the loading phase. Each thread in the thread group is responsible for loading one non-zero element. Since the number of all "parts" in the previous steps was kept smaller than the thread group size, all non-zero elements can be loaded in a single operation. In practice, the parameters of the non-zero elements handled by each thread are determined using a new sparse matrix storage format. All non-zero elements are loaded into shared memory for reuse in subsequent operations. Figure 3 As shown, the two “parts” from line 0, part1 and part2, are handed over to two different thread groups (warp in NVIDIA).

[0063] Step 5: Consume the non-zero elements in the shared memory and multiply them by the corresponding elements of the dense matrix;

[0064] This step is the consumption phase, in which each thread in the thread group is responsible for one column of the dense matrix. The non-zero elements in the shared memory from step four are multiplied by the corresponding elements in that column of the dense matrix, until each thread iterates and consumes all the non-zero elements in the shared memory, and finally a partial sum is obtained.

[0065] Step 6: Add the partial sums from the same row to get the final result and return it.

[0066] Since the "parts" from the same row are calculated in different thread groups, it is necessary to add the parts from different thread groups together to obtain the final sum. In the GPU, the values ​​of different thread groups are synchronized through atomic addition operations.

[0067] The performance of sparse-dense matrix operations on the GPU after the above balanced partitioning is as follows: Figure 4 As shown. Because this method replaces the original rows with "parts" as the parallel granularity, and the size of the "parts" is controlled within a reasonable range to ensure that the thread group has enough threads to parallelize all non-zero element tasks, it avoids issues such as... Figure 4 This issue arises when there are too many non-zero elements, requiring multiple iterations to process all non-zero element tasks, and also due to load imbalances between thread groups. Figure 4 This achieves a more rational allocation of computing tasks and computing resources, thereby achieving an acceleration effect that is more in line with the characteristics of the GPU architecture and can be efficiently parallelized.

[0068] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An adaptive balanced partitioning method based on GPU-accelerated sparse-dense matrix multiplication, characterized in that, Includes the following steps: Step 1: Analyze the sparse matrix to determine the long rows that affect parallel performance; Based on the number of threads in the GPU thread group, a threshold for the number of parallel non-zero element tasks is set. Lines with non-zero elements exceeding the GPU thread group size are identified as long lines, and these long lines are selected for subsequent balancing operations. Step 2: Divide the long line into equal parts; Based on the GPU's thread group size, the long row is balanced into equal-sized "parts", and the size of each "part" must be smaller than the size of the thread group; Step 3: Store the partitioned sparse matrix using the new storage format; After the matrix is ​​balanced and partitioned, "parts" are used instead of rows as the basic unit. The new storage format includes four parameters: row_Ind represents the row of the "part", part_Ind represents the starting position of the "part", col_Ind represents the column of each non-zero element in the "part", and value represents the value of each non-zero element. Step 4: Load all non-zero elements in the "part" into shared memory on the GPU; Each thread in the thread group is responsible for loading one non-zero element. The number of all "parts" is controlled to be less than the size of the thread group. All non-zero elements can be loaded in one load. The parameters of the non-zero elements that each thread is responsible for are determined by the new storage format. All non-zero elements are loaded into shared memory so that they can be reused in subsequent operations. Step 5: Consume the non-zero elements in the shared memory and multiply them by the corresponding elements of the dense matrix; Each thread in the thread group is responsible for calculating one column of the dense matrix. Based on the column containing the non-zero elements in the shared memory, the thread calculates the position of the corresponding element in the dense matrix. The thread iterates through the column until all non-zero elements are consumed, and then calculates the "partial" sum. Step 6: Add the "partial" sums from the same row to get the final result and return it.

Images