Compressed storage of sparse matrix and parallel processing method of vector multiplication thereof

Abstract

The application discloses a compressed storage method of a sparse matrix, which comprises the following steps: shifting non-zero elements in each row of the sparse matrix to the left in sequence from left to right, so that the non-zero elements in each row are arranged continuously on the left to obtain a first intermediate matrix; performing reverse sorting on the rows of the first intermediate matrix to obtain a second intermediate matrix; and intercepting the non-zero element part of the second intermediate matrix to obtain a compressed matrix of the sparse matrix. Based on the compressed matrix of the sparse matrix, the application simultaneously proposes a parallel processing method of sparse matrix vector multiplication, which realizes adaptive calculation task grouping of each row in the compressed matrix, generates at least one parallel calculation task to respectively execute multiplication operation of each in the at least one row vector group and a target vector, reduces the delay of parallel execution calculation of each row in the compressed matrix, and improves the parallel processing performance of the sparse matrix vector multiplication.

Description

Technical Field

[0001] This application relates to the field of high-performance computing, and more specifically, to a method for compressed storage of sparse matrices and parallel processing of their vector multiplication. Background Technology

[0002] A sparse matrix is ​​a matrix in which the number of zero elements far exceeds the number of non-zero elements, and the distribution of non-zero elements is irregular. Sparse matrix vector multiplication (SpMV) is widely used in high-performance computing and is one of the most time-consuming operations in various sparse matrix iterative solvers. Therefore, optimizing the performance of sparse matrix vector multiplication is of practical need and significance. Sparse matrix vector multiplication is usually performed based on the compressed storage format of the sparse matrix, and its computational performance is affected by the different data storage formats of the sparse matrix and the parallel processing methods in the processor.

[0003] Common sparse matrix compression storage formats include Coordinate Array (COO), Compressed Sparse Row (CSR), ELL, Diagonal Array (DIA), and Hybrid Storage Format (HYB), which combines COO and ELL. These storage formats either involve non-merging of memory access to non-zero elements and indices, resulting in inefficient memory access, or require extensive padding with zero elements to balance the number of non-zero elements in each row after compression, wasting significant computational and storage resources. These issues limit the parallel processing performance of operations such as sparse matrix vector multiplication on multi-core processors like GPUs. Summary of the Invention

[0004] To address the aforementioned issues, this application proposes a method for compressed storage of sparse matrices and parallel processing of their vector multiplication, which can significantly increase the efficiency of compressed storage of sparse matrices, save computational and storage resources, and improve the parallel processing performance of sparse matrix vector multiplication operations.

[0005] In a first aspect, embodiments of this application propose a method for compressed storage of sparse matrices, comprising:

[0006] Shift the non-zero elements in each row of the sparse matrix to the left in order from left to right, so that the non-zero elements in each row are arranged continuously on the left, and obtain the first intermediate matrix.

[0007] The rows of the first intermediate matrix are reversed according to the number of non-zero elements in each row from largest to smallest to obtain the second intermediate matrix. The non-zero element part of the second intermediate matrix is ​​then extracted to obtain the compressed matrix of the sparse matrix.

[0008] Following the column order from left to right, the non-zero elements in each column of the compressed matrix are stored in the first array from top to bottom. The column indices of the non-zero elements in each column in the sparse matrix are stored in the second array, and the number of non-zero elements in each column is stored in the third array. Following the row order from top to bottom, the number of non-zero elements in each row of the compressed matrix is ​​stored consecutively in the fourth array, and the row index of each row in the compressed matrix in the sparse matrix is ​​stored in the fifth array.

[0009] In an optional implementation, the first array, the second array, the third array, the fourth array, and the fifth array constitute a quintuple characterizing the storage features of the compression matrix.

[0010] Secondly, embodiments of this application also propose a parallel processing method for sparse matrix-vector multiplication, including:

[0011] Step S710: The sparse matrix is ​​compressed and stored according to the compression storage method described in the foregoing embodiment to obtain the compressed matrix of the sparse matrix and a first array, a second array, a third array, a fourth array and a fifth array characterizing the storage features of the compressed matrix;

[0012] Step S720: According to the row order of the compressed matrix, the rows of the compressed matrix are adaptively grouped into computational tasks to obtain at least one row vector group, and the number of row parallel threads, the total number of parallel threads, and the row starting address index required for parallel computation of each row vector group are obtained.

[0013] Step S730: Based on the number of parallel threads required for parallel computation of each row vector group, the total number of parallel threads, and the row start address index, generate at least one parallel computation task to perform the multiplication operation between each of the at least one row vector group and the target vector.

[0014] In an optional implementation, step S720 includes:

[0015] Step S7210: Pre-group each row of the compression matrix according to the row order and the predetermined number of grouped rows to obtain at least one row group;

[0016] Step S7220: Calculate the maximum number of non-zero elements and the minimum number of non-zero elements in each row group in the at least one row group, and determine the number of row parallel threads required for each row group to perform parallel computing based on the ratio of the maximum number of non-zero elements to the minimum number of non-zero elements.

[0017] Step S7230: The row groups with the same number of parallel threads required for parallel computing are merged into a row vector group to obtain the at least one row vector group.

[0018] In an optional implementation, determining the number of row-parallel threads required for parallel computation of each row group based on the ratio of the largest row's non-zero element count to the smallest row's non-zero element count includes:

[0019] The number of row-parallel threads required for parallel computation of each row group is determined based on whether the ratio of the maximum number of non-zero elements in a row to the minimum number of non-zero elements in a row is within a predetermined range.

[0020] In an optional implementation, step S730 includes:

[0021] Step S7310: In each parallel computing task, based on the number of parallel threads required for parallel computing in each row vector group, the total number of parallel threads, and the row starting address index, at least one parallel thread is allocated to each row in each row vector group, and the first column index and row address index of each row are determined for each parallel thread in the at least one parallel thread.

[0022] Step S7320: In one loop of each parallel thread, a non-zero element and the second column index of the non-zero element in the sparse matrix are read from the first array and the second array respectively according to the row address index, the first column index and the third array, and the multiplication operation of the non-zero element with the multiplier of the target vector corresponding to the second column index is performed.

[0023] Step S7330: In the next loop of each parallel thread, shift each parallel thread to the right by a predetermined number of columns in each row to obtain a new first column index corresponding to each parallel thread. Repeat step S7320 until the first column index exceeds the range of the number of non-zero elements in each row.

[0024] Step S7340: During the execution of one of the at least one parallel threads, the results of the multiplication operations completed by the at least one parallel thread are summed.

[0025] In an optional implementation, step S7340 further includes: restoring the summation result to the original result order of the multiplication operation of the sparse matrix according to the row index of each row in the compressed matrix stored in the fifth array in the sparse matrix.

[0026] In an optional implementation, step S7330 includes: when the index of the first column corresponding to each parallel thread exceeds the range of the number of non-zero elements in each row, saving the result of the multiplication operation completed by each parallel thread to shared memory.

[0027] In an optional implementation, step S7340 further includes: during the execution of one of the at least one parallel thread, reading the results of the multiplication operations completed by each parallel thread from the shared memory, summing them, and outputting the summation results to memory.

[0028] In an optional implementation, generating at least one parallel computing task in step S730 includes:

[0029] Based on the number of row-parallel threads required for parallel computation in each row vector group and the total number of parallel threads, determine the number of thread blocks to be launched for at least one GPU task and the number of parallel threads launched for each thread block;

[0030] The at least one GPU task is generated based on the number of thread blocks required to be launched for the at least one GPU task and the number of parallel threads launched for each thread block.

[0031] Compared to existing technologies, this application has at least the following beneficial effects: On the one hand, the sparse matrix compression storage method of this application's embodiments enables complete merged access to the compressed element array and column index array in memory when performing calculations such as sparse matrix vector multiplication, improving the data processing efficiency of sparse matrix vector operations on multi-core processors such as GPUs. Furthermore, the storage of non-zero elements in each column does not require padding with a large number of zero elements to balance the number of elements in rows or columns with uneven non-zero element counts, saving memory storage resources. On the other hand, by utilizing the advantages of the sparse matrix compression storage method of this application, and by adaptively allocating the number of parallel threads for each row of the compressed matrix, the parallel computing capability of threads can be fully utilized when performing sparse matrix vector operations on multi-core processors such as GPUs. This improves the performance of parallel thread computing for non-zero elements in rows with uneven counts in the compressed matrix, significantly reducing the execution latency of parallel operations in each row. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as limiting the scope of this application.

[0033] Figure 1 This is a flowchart illustrating a method for compressing and storing sparse matrices according to an embodiment of this application;

[0034] Figure 2 This is a schematic diagram of an exemplary sparse matrix according to an embodiment of this application;

[0035] Figure 3This is a schematic diagram of an intermediate matrix after an exemplary sparse matrix transformation according to an embodiment of this application;

[0036] Figure 4 This is a schematic diagram of a compressed matrix after an exemplary sparse matrix transformation according to an embodiment of this application;

[0037] Figure 5 This is a schematic diagram of an exemplary compressed storage format of a sparse matrix according to an embodiment of this application;

[0038] Figure 6 This is another schematic diagram of a compressed storage format of an exemplary sparse matrix according to an embodiment of this application;

[0039] Figure 7 This is a flowchart illustrating a parallel processing method for sparse matrix-vector multiplication according to an embodiment of this application.

[0040] Figure 8 This is a flowchart illustrating a parallel processing method for sparse matrix-vector multiplication according to another embodiment of this application;

[0041] Figure 9 This is a flowchart illustrating a parallel processing method for sparse matrix-vector multiplication according to another embodiment of this application;

[0042] Figure 10 This is a schematic diagram illustrating the principle of parallel thread processing for sparse matrix-vector multiplication according to an embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. However, it should be understood that the described embodiments are merely some exemplary embodiments of this application, and not all embodiments. Therefore, the following detailed description of the embodiments of this application is not intended to limit the scope of the claimed application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0044] It should be noted that the terms "first," "second," etc., in the specification and claims of this application are only used to distinguish and describe similar objects, and are not used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance.

[0045] As mentioned earlier, commonly used compressed storage formats for sparse matrices are insufficient to meet the parallel processing performance requirements of sparse matrix vector multiplication in high-performance computing. Therefore, this application first proposes a compressed storage method for sparse matrices, and then, based on this compressed storage method, provides a parallel processing method for sparse matrix vector multiplication to improve the parallel processing performance of sparse matrix vector multiplication in multi-core processors, thereby meeting the requirements of high-performance computing.

[0046] Example 1

[0047] Figure 1 This is a flowchart illustrating a sparse matrix compression storage method according to an embodiment of this application. Figure 1 As shown, the sparse matrix compression storage method proposed in this application includes the following steps:

[0048] In step S110, the non-zero elements in each row of the sparse matrix are shifted to the left in order from left to right, so that the non-zero elements in each row are arranged continuously on the left, thus obtaining the first intermediate matrix.

[0049] like Figure 2 As shown, assume the original sparse matrix is ​​an m×n two-dimensional matrix, where m and n are both integers greater than 1, and each row has a different number of non-zero elements and zero elements. To more intuitively describe the scheme described in this application, Figure 2 An exemplary m×13 sparse matrix A with a finite number of columns (n=13) is presented only schematically. s The 0th row contains 7 non-zero elements a1 to a7, the 1st row contains 9 non-zero elements b1 to b9, and so on. The (m-1)th row contains 8 non-zero elements s1 to s8, and the rest are zero elements.

[0050] In this step, first according to matrix A s Starting from left to right, each row in the matrix is ​​shifted one of its non-zero elements to the left, resulting in a continuous left-aligned arrangement of non-zero elements. This is equivalent to moving the zero elements from their central position to the right side of the matrix, leaving all non-zero elements on the left side, forming a matrix like this: Figure 3 The first intermediate matrix A1 is shown.

[0051] Step S120: Sort the rows of the first intermediate matrix in descending order of the number of non-zero elements in each row to obtain the second intermediate matrix. Extract the non-zero element portion of the second intermediate matrix to obtain the compressed matrix of the sparse matrix.

[0052] like Figure 3As shown, the number of non-zero elements in each row of the first intermediate matrix A1 is unequal, with the first row having the most non-zero elements b1 to b9, and the (m-3)th row having the fewest non-zero elements k1 to k2. In this step, the rows of the first intermediate matrix A1 need to be sorted in descending order of the number of non-zero elements in each row, forming the following... Figure 4 The second intermediate matrix A2 is shown below. After reversing the sorting, the first row with the most non-zero elements in the first intermediate matrix A1 becomes the 0th row of the second intermediate matrix A2, and the (m-3)th row with the fewest non-zero elements in the first intermediate matrix A1 becomes the (m-1)th row of the second intermediate matrix A2. From the perspective of the distribution of non-zero elements in the matrix, the arrangement of non-zero elements in the second intermediate matrix A2 resembles an inverted ladder. Extracting this non-zero element portion yields a compressed matrix A as shown in Figure 5. c .

[0053] Step S130: Store the non-zero elements of each column in the compressed matrix from top to bottom into the first array according to the column order from left to right; store the column index of each non-zero element in the sparse matrix into the second array; and store the number of each non-zero element in the third array. Store the number of each row of the compressed matrix into the fourth array according to the row order from top to bottom; and store the row index of each row in the compressed matrix into the fifth array.

[0054] In this step, the original sparse matrix A has already been obtained. s Compression matrix A c Next, the first step is to compress matrix A. c Non-zero elements are stored consecutively in the first array. For example... Figure 5 As shown, for the compression matrix A c For each non-zero element in the matrix, compress the matrix A sequentially from left to right. c The non-zero elements in each column are stored consecutively from top to bottom in the first array A[]. In one implementation, the first array A[] can be a one-dimensional array with a length equal to the number of all non-zero elements in the original sparse matrix. For example, the compressed matrix A is stored in the order described above. c The first array A[] obtained by storing non-zero elements in each column consecutively can be like this: Figure 5 As shown in the lower left corner.

[0055] In this step, in order to perform various operations such as vector multiplication (SpMV) on the non-zero elements of the original sparse matrix later, the compressed matrix A needs to be compressed sequentially in the same order as the non-zero elements in the first array. c The column indices corresponding to the non-zero elements in each column of the first array are stored consecutively in the second array. For example... Figure 5As shown, for the compression matrix A c The column index (also called column number) corresponding to each non-zero element in the original sparse matrix is ​​stored consecutively from top to bottom in the second array COL[]. In one implementation, the second array COL[] can be a one-dimensional array with a length equal to the number of all non-zero elements in the original sparse matrix. The compressed matrix A is then compressed in the order described above. c After storing the column indices of each non-zero element in the original sparse matrix, the resulting second array COL[] can be represented as follows: Figure 5 As shown in the lower right corner. Using compression matrix A c Taking the 0th column as an example, the non-zero elements b1, s1, a1, g1, c1, f1, d1, ..., e1, l1, k1 correspond to the original sparse matrix A. s The column indices are 0,0,1,0,1,0,0,…,2,1,2.

[0056] In this step, in order to perform parallel thread computation and adaptive task scheduling in various operations such as vector multiplication of non-zero elements in the original sparse matrix, it is also necessary to compress the matrix A according to the column order from left to right. c The number of non-zero elements in each column is stored consecutively in a third array. For example... Figure 6 As shown, the compression matrix A is arranged sequentially from left to right. c The number of non-zero elements in each column is stored consecutively in a third array CNT[]. In one implementation, the third array CNT[] can be of length A equal to the original sparse matrix. s A one-dimensional array with the most non-zero elements in a row, due to the compression matrix A c The 0th row corresponds to the original sparse matrix A. s The third array CNT[] contains the row with the most non-zero elements, therefore its length is equal to that of the compressed matrix A. c The number of non-zero elements in row 0. Figure 6 In the example shown, due to the compression matrix A c The number of non-zero elements in the 0th row is 9, meaning the length of the third array CNT[] is 9. Therefore, the compressed matrix A can be arranged in column order from left to right. c The number of non-zero elements in each column is stored sequentially in CNT[0] to CNT[8].

[0057] In this step, in order to perform parallel thread computation and adaptive task scheduling in various operations such as vector multiplication of non-zero elements in the original sparse matrix, it is also necessary to compress the matrix A in a top-to-bottom row order. c The number of non-zero elements in each row is stored consecutively in the fourth array. For example... Figure 6As shown, the compression matrix A can be arranged sequentially from top to bottom. c The number of non-zero elements in each row is stored consecutively in the fourth array RNNZ[]. In one implementation, the fourth array RNNZ[] can be of length A equal to the original sparse matrix. s A one-dimensional array of rows. Figure 6 In the example shown, due to the original sparse matrix A s The number of rows is m, meaning the length of the fourth array RNNZ[] is m. This allows the compressed matrix A to be arranged in a row-wise order from top to bottom. c The number of non-zero elements in each row is stored sequentially in RNNZ[0]~RNNZ[m-1].

[0058] In this step, in order to restore the row order of the original sparse matrix after performing various operations such as vector multiplication on the non-zero elements in the original sparse matrix, it is necessary to compress the matrix A in a top-to-bottom row order. c The row index corresponding to each row in the original sparse matrix is ​​stored consecutively in the fifth array. For example... Figure 6 As shown, the compression matrix A can be arranged sequentially from top to bottom. c The row indices corresponding to each row in the original sparse matrix are stored consecutively in the fifth array IPERM[]. In one implementation, the fifth array IPERM[] can be of length A equal to that of the original sparse matrix. s A one-dimensional array of rows. Figure 6 In the example shown, due to the original sparse matrix A s The number of rows is m, meaning the length of the fifth array IPERM[] is m. This allows the compressed matrix A to be arranged in a row-by-row order from top to bottom. c The row index corresponding to each row in the original sparse matrix is ​​stored sequentially in IPERM[0]~IPERM[m-1].

[0059] In some implementations, the first array A[], the second array COL[], the third array CNT[], the fourth array RNNZ[], and the fifth array IPERM[] are used to characterize the storage features of the compression matrix, and can be represented by a quintuple {A[],COL[],CNT[],RNNZ[],IPERM[]}.

[0060] The sparse matrix compression storage method of this application first shifts the non-zero elements of each row in the original sparse matrix to the left, so that the non-zero elements in each row are arranged consecutively on the left. Then, the rows are sorted in reverse order according to the number of non-zero elements in each row to obtain a second intermediate matrix. The non-zero elements in each column of the second intermediate matrix are compressed and stored into a first array in column order from left to right. At the same time, the column indices of the non-zero elements in the original sparse matrix are also stored into a second array in the same order. The number of non-zero elements in each column is stored consecutively into a third array, and the number of non-zero elements in each row is stored consecutively into a fourth array. This compression storage method enables complete merged access to the compressed element array and column index array in memory when performing calculations such as vector multiplication of sparse matrices, thereby improving the data processing efficiency of sparse matrix vector operations on multi-core processors such as GPUs. On the other hand, by storing the number of non-zero elements in each column of the second intermediate matrix to the third array and the number of non-zero elements in each row to the fourth array, the storage of non-zero elements in each column does not require filling a large number of zero elements to balance the number of elements in each row or column where the number of non-zero elements is uneven. At the same time, when performing sparse matrix vector operations on multi-core processors such as GPUs, the ability of thread parallel computing can be fully utilized to improve the performance of thread parallel computing on non-zero elements in each row of the compressed matrix. This advantage will be further explained in detail in subsequent embodiments.

[0061] Example 2

[0062] The compressed storage format of the original sparse matrix obtained by the sparse matrix compression storage method in the embodiments of this application can fully utilize the parallel computing capabilities of threads when performing sparse matrix vector operations on multi-core processors such as GPUs. Therefore, the embodiments of this application further propose a parallel processing method for sparse matrix vector multiplication, which can achieve parallel acceleration calculation of non-zero elements in each row of the compressed matrix through adaptive thread task scheduling, thereby improving the parallel processing performance of sparse matrix vector multiplication and minimizing the execution latency problem in the parallel operation of each row of data.

[0063] Figure 7 This is a flowchart illustrating a parallel processing method for sparse matrix-vector multiplication according to an embodiment of this application. Figure 7 As shown, the parallel processing method for sparse matrix-vector multiplication in this application includes the following steps:

[0064] Step S710: The sparse matrix is ​​compressed and stored according to the sparse matrix compression storage method of Embodiment 1 of this application to obtain the compressed matrix of the sparse matrix and a first array, a second array, a third array, a fourth array and a fifth array characterizing the storage features of the compressed matrix.

[0065] In this step, the original sparse matrix As Compressed storage yields the compressed matrix A of the original sparse matrix. c And a first array A[], a second array COL[], a third array CNT[], a fourth array RNNZ[], and a fifth array IPERM[] used to characterize the features of the compressed matrix. Among them, the first array A[] stores the non-zero elements in each column of the compressed matrix, the second array COL[] stores the column index of each non-zero element in the compressed matrix in the original sparse matrix, the third array CNT[] stores the number of non-zero elements in each column of the compressed matrix, the fourth array RNNZ[] stores the number of non-zero elements in each row of the compressed matrix, and the fifth array IPERM[] stores the row index of each row of the compressed matrix in the original sparse matrix.

[0066] In some implementations, the first array A[], the second array COL[], the third array CNT[], the fourth array RNNZ[], and the fifth array IPERM[] used to characterize the features of the compressed matrix can be represented by a quintuple {A[],COL[],CNT[],RNNZ[],IPERM[]}.

[0067] Step S720: According to the row order of the compressed matrix, the rows of the compressed matrix are adaptively grouped into computational tasks to obtain at least one row vector group, and the number of row parallel threads, the total number of parallel threads, and the row starting address index required for parallel computation of each row vector group are obtained.

[0068] In this embodiment, compression matrix A c Perform vector multiplication with the target vector to compress matrix A. c Each row in the matrix can be understood as a row vector, and parallel thread computations of vector multiplication can be performed on that row vector. Due to the compression matrix A... c In a compressed matrix, the number of non-zero elements in each row decreases from top to bottom. When performing vector multiplication between each row vector and the target vector in this compressed matrix, the varying number of non-zero elements leads to significant differences in computation latency for each row, impacting the parallel computing performance of sparse matrix vector multiplication. This is especially true when the ratio between the number of non-zero elements in the largest and smallest rows of the compressed matrix is ​​large, or when the ratio is close to 1 but the absolute values ​​of the numbers differ significantly, resulting in substantial differences in parallel computation latency for each row and further affecting the parallel computing performance of sparse matrix vector multiplication.

[0069] In this step, according to the compression matrix A c The row order of the compressed matrix A cThe row vectors are adaptively grouped for computational tasks, resulting in at least one row vector group. The rows of the compressed matrix can be divided into different row vector groups based on the distribution of the number of non-zero elements in each row. The number of parallel threads required for parallel computation in different row vector groups can vary. The number of parallel threads required for parallel computation in a row vector group refers to the number of parallel threads required for parallel computation in each row of the row vector group. The total number of parallel threads required for parallel computation in a row vector group refers to the total number of parallel threads required for parallel computation in all rows of the row vector group. Generally, the total number of parallel threads equals the product of the total number of rows in the row vector group and the number of parallel threads. The row start address index refers to the row address index of the first row of each row vector group in the compressed matrix when performing parallel computation tasks; it can also be called the starting offset address of the row vector group.

[0070] This step involves compressing matrix A. c The adaptive grouping of rows into computational tasks is to ensure that the number of parallel threads required for parallel computation in each row vector group is adapted to the distribution of the number of non-zero elements in each row of the row vector group. This makes the overall execution latency of parallel computation in each row not much different, thereby improving the overall parallel computation performance of sparse matrix vector multiplication.

[0071] In some implementations, such as Figure 8 As shown, step S720 may further include the following steps:

[0072] Step S7210: Pre-group each row of the compression matrix according to the row order and the predetermined number of grouped rows to obtain at least one row group.

[0073] In this step, the predetermined number of grouping rows can be determined based on the compression matrix A. c The number of rows in the compressed matrix A is determined by factors such as the size of the row count and thread scheduling parameters in multi-core processors like GPUs. As an example, the number of rows in the grouping matrix A can be chosen to be 256, 128, or 64. c The rows are pre-grouped to obtain several row groups.

[0074] Step S7220: Calculate the maximum number of non-zero elements and the minimum number of non-zero elements in each row group in the at least one row group, and determine the number of row parallel threads required for parallel computation in each row group based on the ratio of the maximum number of non-zero elements to the minimum number of non-zero elements in each row group.

[0075] In this step, it is assumed that the previous steps have already been based on the compression matrix A. c The row order is used to compress matrix A into groups of 256 rows. c The rows of matrix A are pre-grouped to obtain several row groups. Then, the compressed matrix A can be... cThe maximum number of non-zero elements in each row of each row group is obtained from the fourth array RNNZ[]. Then, the number of parallel threads required for parallel computation in each row of each row group can be determined based on the ratio of max_RNNZ / min_RNNZ.

[0076] In some implementations, the number of row-parallel threads required for parallel computation in each row of the row group can be determined based on whether the ratio of max_RNNZ / min_RNNZ is within a predetermined range.

[0077] Step S7230: The row groups with the same number of parallel threads required for parallel computing are merged into a row vector group to obtain the at least one row vector group.

[0078] In this step, after determining the number of row-parallel threads required for parallel computation of each row in each row group based on the ratio of max_RNNZ / min_RNNZ, row groups with the same number of row-parallel threads required for parallel computation of each row can be merged. That is, row groups with the same number of row-parallel threads are merged into a row vector group, and the same computation task can be used for parallel computation processing on the row vector group.

[0079] As an example, the ratio of max_RNNZ / min_RNNZ is divided into five ranges to determine the level of the number of row-parallel threads required for parallel computation in each row of a row group. The specific rules are explained below:

[0080] 1) When the minimum number of non-zero elements in a row group, min_RNNZ, is less than 4, set min_RNNZ = 4;

[0081] 2) If max_RNNZ / min_RNNZ>64, then each row in this row group requires 256 parallel threads to perform parallel computation, and the total number of parallel threads for all rows is 256*256.

[0082] 3) If max_RNNZ / min_RNNZ>16 and max_RNNZ / min_RNNZ<=64, then each row in this row group requires 64 parallel threads to perform parallel computation, and the total number of parallel threads for all rows is 64*256.

[0083] 4) If max_RNNZ / min_RNNZ>4 and max_RNNZ / min_RNNZ<=16, then each row in this row group requires 16 parallel threads to perform parallel computation, and the total number of parallel threads for all rows is 16*256.

[0084] 5) If max_RNNZ / min_RNNZ>1 and max_RNNZ / min_RNNZ<=4, then each row in this row group requires 4 parallel threads to perform parallel computation, and the total number of parallel threads for all rows is 4*256.

[0085] 6) If max_RNNZ / min_RNNZ<=1, then each row in this row group requires 1 parallel thread to perform parallel computation, and the total number of parallel threads for all rows is 1*256.

[0086] Subsequently, the compression matrix A is compressed according to the above rules. c All rows are divided into row groups with 256, 64, 16, 4, and 1 parallel threads respectively. Row groups with the same number of parallel threads are then merged. This generates five row vector groups, from row vector group 1 to row vector group 5, according to the above rules. The number of parallel threads per row, the total number of parallel threads, and the row start address index for each row vector group are shown in Table 1 below. Here, rows_PT256, rows_PT64, rows_PT16, rows_PT4, and rows_PT1 represent the total number of rows in row vector groups 1 to 5, respectively. row_Start_256, row_Start_64, row_Start_16, row_Start_4, and row_Start_1 represent the row start address indices for row vector groups 1 to 5, respectively.

[0087] Table 1

[0088]

[0089]

[0090] Step S730: Based on the number of parallel threads required for parallel computation of each row vector group, the total number of parallel threads, and the row start address index, generate at least one parallel computation task to perform the multiplication operation between each of the at least one row vector group and the target vector.

[0091] In this step, the compression matrix A is... c After adaptively grouping the computational tasks, at least one parallel computational task can be generated to perform the multiplication operation between each row vector group and the target vector. In one implementation, multiple parallel computational tasks can be launched on a multi-core processor such as a GPU to perform parallel accelerated computation of each row vector group. Each parallel computational task can determine the appropriate kernel function running parameters based on the number of row parallel threads, the total number of parallel threads, and the row start address index of the row vector group, and execute the parallel computation of each row vector group by running different kernel functions.

[0092] In one embodiment, the operating parameters include at least the number of thread blocks required to be launched by the GPU task and the number of parallel threads launched per thread block. In one embodiment, step S730 may include: generating the at least one GPU task based on the number of thread blocks required to be launched by the GPU task and the number of parallel threads launched per thread block.

[0093] As an example, a corresponding GPU computing task can be generated for each row vector group. The characteristics of the running parameters of each computing task are shown in Table 2 below:

[0094] Table 2

[0095]

[0096] In the table above, the number of thread blocks is the number of parallel thread blocks (Blocks) required to execute a GPU task. The number of block threads is the number of parallel threads started by each thread block.

[0097] In some implementations, such as Figure 9 As shown, step S730 may further include the following steps:

[0098] Step S7310: In each parallel computing task, based on the number of parallel threads required for parallel computing in each row vector group, the total number of parallel threads, and the row starting address index, at least one parallel thread is allocated to each row in each row vector group, and the first column index and row address index of each row are determined for each parallel thread in the at least one parallel thread.

[0099] In this step, for each parallel computing task, the first step is to allocate threads for parallel execution to each row in each row vector group from the started parallel thread block, based on the number of row-parallel threads required for parallel computing in each row vector group, the total number of parallel threads, and the row start address index. As an example, let's take a parallel computing task with 4 row-parallel threads as an example. Figure 10 This diagram illustrates the parallel computation of vector multiplication on a row vector group with a total number of rows_PT4 = 16 in a compressed matrix.

[0100] like Figure 10As shown, in this parallel computing task, each thread block starts 64 parallel threads, namely threads Thread0 to Thread63. The total number of rows in the row vector is rows_PT4 = 16, and the row start address index, i.e., the row address index of row 0 in the compression matrix, is row_Start_4. Each row in this row vector group executes 4 parallel threads, that is, the total number of parallel threads executed is 16*4 = 64. That is, starting one thread block can meet the parallel computing requirements of this row vector. Since each row in this row vector requires 4 parallel threads, 4 parallel threads can be initially allocated to each row in this row vector. Specifically, threads Thread0 through Thread3 are sequentially assigned to elements with indexes 0 through 3 in row 0 of the row vector, with the row address index being row_Start_4; threads Thread4 through Thread7 are sequentially assigned to elements with indexes 0 through 3 in row 1 of the row vector, with the row address index being row_Start_4+1; and so on, until threads Thread60 through Thread63 are sequentially assigned to elements with indexes 0 through 3 in row 15 of the row vector, with the row address index being row_Start_4+15.

[0101] Step S7320: In one loop of each parallel thread, a non-zero element and the second column index of the non-zero element in the sparse matrix are read from the first array and the second array respectively according to the row address index, the first column index and the third array, and the multiplication operation of the non-zero element with the multiplier of the target vector corresponding to the position of the second column index is performed.

[0102] In this step, for each loop of a parallel thread, the non-zero element to be executed for each parallel thread and its corresponding column index (column number) in the sparse matrix can be read from the first array A[] and the second array COL[] of the compressed matrix according to the row address index, the column index in each row corresponding to the parallel thread, and the number of non-zero elements in each column of the compressed matrix stored in the third array CNT[]. For example... Figure 10 As shown, for Thread0 to Thread3, non-zero elements b1, b2, b3, b4 are read from the first array A[], and the column indices 0, 2, 3, 5 corresponding to the non-zero elements b1, b2, b3, b4 in the sparse matrix are read from the second array COL[]. (See...) Figure 5 For threads Thread4 to Thread7, read the non-zero elements s1, s2, s3, s4 from the first array A[], and read the column indices 0, 1, 3, 4 of the non-zero elements s1, s2, s3, s4 in the sparse matrix from the second array COL[] (see...). Figure 5); other threads follow the same pattern.

[0103] Then, for each parallel thread, the multiplication operation is performed between the non-zero element and the multiplier in the target vector X corresponding to the column index position of the non-zero element in the sparse matrix. For example, in a loop executed by Thread0, the operation of non-zero element b1*X[0] is performed; in a loop executed by Thread1, the operation of non-zero element b2*X[2] is performed; in a loop executed by Thread2, the operation of non-zero element b3*X[3] is performed; in a loop executed by Thread3, the operation of non-zero element b4*X[5] is performed; and so on.

[0104] In step S7330, in the next loop of each parallel thread, each parallel thread is shifted to the right by a predetermined number of columns in each row to obtain a new first column index corresponding to each parallel thread. Step S7320 is repeated until the first column index exceeds the range of the number of non-zero elements in each row.

[0105] In this step, after each parallel thread completes one loop, in the next loop, the parallel thread needs to be shifted to the right by a predetermined number of columns in the current row. For example... Figure 10 As shown, this example uses a 4-column shift per parallel thread. For example, for threads Thread0 to Thread3, the next loop shifts from the positions of non-zero elements b1, b2, b3, b4 to the positions of non-zero elements b5, b6, b7, b8 respectively. It then continues to read non-zero elements b5, b6, b7, b8 from the first array A[] according to the aforementioned steps, and reads the column indices 6, 7, 8, 10 (see [link to example]) of the non-zero elements b5, b6, b7, b8 in the sparse matrix from the second array COL[]. Figure 5 After completing the current loop operation, thread Thread0 continues to shift four columns to the right, moving from the position of non-zero element b5 to the position of non-zero element b9. The other three threads, Thread1 through Thread3, terminate their loops because the column indices corresponding to the shifted elements exceed the range of non-zero elements in the current row. The execution of parallel threads for the other rows of this row vector group follows the same pattern.

[0106] Step S7340: During the execution of one of the at least one parallel threads, the results of the multiplication operations completed by the at least one parallel thread are summed.

[0107] In this step, for each row of multiple parallel threads, one of the parallel threads is selected, and after the loop operation is completed, the multiplication results of the group of parallel threads assigned to each row are summed. In one embodiment, step S7340, in the execution of one of the at least one parallel thread, further includes performing a matrix restoration operation on the summed result according to the fifth array, that is, according to the row index of each row in the compressed matrix stored in the fifth array in the sparse matrix, the summed result is restored to the original result order of the multiplication operation of the sparse matrix.

[0108] In one implementation, when the index of the first column corresponding to each parallel thread exceeds the range of the number of non-zero elements in each row, step S7340 saves the result of the multiplication operation completed by each parallel thread to shared memory.

[0109] In one implementation, step S7340, during the execution of one of the at least one parallel thread, reads the results of the multiplication operations completed by each parallel thread from the shared memory, sums them, and outputs the summation results to memory.

[0110] The parallel processing method for sparse matrix-vector multiplication in this application, based on the advantages of the sparse matrix compression storage method of this application, can achieve complete merged access of the first and second arrays of the compressed matrix in memory, thereby improving the data processing efficiency of sparse matrix-vector operations on multi-core processors such as GPUs. Furthermore, by adaptively allocating the number of parallel threads for each row of the compressed matrix, the parallel computing capabilities of threads can be fully utilized when performing sparse matrix-vector operations on multi-core processors such as GPUs. This improves the performance of parallel thread computation on non-zero elements in rows of the compressed matrix with uneven numbers, and significantly reduces the execution latency of parallel operations on each row.

[0111] The foregoing description of exemplary embodiments of this application should be understood as not limiting, but illustrative, and the scope of protection of this application is not limited thereto. It should be understood that those skilled in the art can make modifications and variations to the embodiments of this application without departing from the spirit and scope of this application, and such modifications and variations should be within the scope of protection of this application.

Claims

1. A parallel processing method for sparse matrix-vector multiplication, characterized in that, include: Step S710: Compress and store the sparse matrix according to the sparse matrix compression storage method to obtain the compressed matrix of the sparse matrix and the first array, second array, third array, fourth array and fifth array characterizing the storage characteristics of the compressed matrix; Step S720: According to the row order of the compressed matrix, the rows of the compressed matrix are adaptively grouped into computational tasks to obtain at least one row vector group, and the number of row parallel threads, the total number of parallel threads, and the row starting address index required for parallel computation of each row vector group are obtained. Step S730: Based on the number of parallel threads required for parallel computation of each row vector group, the total number of parallel threads, and the row start address index, generate at least one parallel computation task to perform the multiplication operation between each of the at least one row vector group and the target vector. The sparse matrix compression storage method includes: shifting the non-zero elements in each row of the sparse matrix to the left in a left-to-right order, so that the non-zero elements in each row are arranged continuously on the left, to obtain a first intermediate matrix; reversing the order of the rows of the first intermediate matrix in descending order of the number of non-zero elements in each row, to obtain a second intermediate matrix; extracting the non-zero element portion of the second intermediate matrix to obtain a compressed matrix of the sparse matrix; storing the non-zero elements in each column of the compressed matrix from top to bottom in a first array in a left-to-right column order, storing the column index of each column of the non-zero elements in the sparse matrix in a second array, and storing the number of each column of the non-zero elements in a third array; and storing the number of each row of the compressed matrix in a continuous row order from top to bottom in a fourth array, and storing the row index of each row of the compressed matrix in the sparse matrix in a fifth array.

2. The parallel processing method for sparse matrix-vector multiplication according to claim 1, characterized in that, Step S720 includes: Step S7210: Pre-group each row of the compression matrix according to the row order and the predetermined number of grouped rows to obtain at least one row group; Step S7220: Calculate the maximum number of non-zero elements and the minimum number of non-zero elements in each row group in the at least one row group, and determine the number of row parallel threads required for each row group to perform parallel computing based on the ratio of the maximum number of non-zero elements to the minimum number of non-zero elements. Step S7230: The row groups with the same number of parallel threads required for parallel computing are merged into a row vector group to obtain the at least one row vector group.

3. The parallel processing method for sparse matrix-vector multiplication according to claim 2, characterized in that, The step of determining the number of row-parallel threads required for parallel computation of each row group based on the ratio of the largest row's non-zero element count to the smallest row's non-zero element count includes: The number of row-parallel threads required for parallel computation of each row group is determined based on whether the ratio of the maximum number of non-zero elements in a row to the minimum number of non-zero elements in a row is within a predetermined range.

4. The parallel processing method for sparse matrix-vector multiplication according to any one of claims 1-3, characterized in that, Step S730 includes: Step S7310: In each parallel computing task, based on the number of parallel threads required for parallel computing in each row vector group, the total number of parallel threads, and the row starting address index, at least one parallel thread is allocated to each row in each row vector group, and the first column index and row address index of each row are determined for each parallel thread in the at least one parallel thread. Step S7320: In one loop of each parallel thread, a non-zero element and the second column index of the non-zero element in the sparse matrix are read from the first array and the second array respectively according to the row address index, the first column index and the third array, and the multiplication operation of the non-zero element with the multiplier of the target vector corresponding to the second column index is performed. Step S7330: In the next loop of each parallel thread, shift each parallel thread to the right by a predetermined number of columns in each row to obtain a new first column index corresponding to each parallel thread. Repeat step S7320 until the first column index exceeds the range of the number of non-zero elements in each row. Step S7340: During the execution of one of the at least one parallel threads, the results of the multiplication operations completed by the at least one parallel thread are summed.

5. The parallel processing method for sparse matrix-vector multiplication according to claim 4, characterized in that, Step S7340 further includes: restoring the summation result to the original result order of the multiplication operation of the sparse matrix according to the row index of each row in the compressed matrix stored in the fifth array.

6. The parallel processing method for sparse matrix-vector multiplication according to claim 5, characterized in that, Step S7330 includes: when the index of the first column corresponding to each parallel thread exceeds the range of the number of non-zero elements in each row, saving the result of the multiplication operation completed by each parallel thread to shared memory.

7. The parallel processing method for sparse matrix-vector multiplication according to claim 6, characterized in that, Step S7340 further includes: during the execution of one of the at least one parallel thread, reading the multiplication results completed by each parallel thread from the shared memory, summing them, and outputting the summation results to memory.

8. The parallel processing method for sparse matrix-vector multiplication according to claim 7, characterized in that, The step S730 of generating at least one parallel computing task includes: Based on the number of row-parallel threads required for parallel computation in each row vector group and the total number of parallel threads, determine the number of thread blocks to be launched for at least one GPU task and the number of parallel threads launched for each thread block; The at least one GPU task is generated based on the number of thread blocks required to be launched for the at least one GPU task and the number of parallel threads launched for each thread block.

9. The parallel processing method for sparse matrix-vector multiplication according to claim 1, characterized in that, The first array, the second array, the third array, the fourth array, and the fifth array constitute a quintuple characterizing the storage features of the compression matrix.

Images