Sparse matrix calculation method, device, equipment, storage medium and program product
By estimating the number of non-zero elements in the sparse matrix multiplication result matrix and utilizing multi-threaded computation and caching mechanisms, the problems of memory waste and low computational efficiency in sparse matrix multiplication are solved, achieving faster computation speed and smaller memory usage.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ALIBABA (CHINA) CO LTD
- Filing Date
- 2023-04-12
- Publication Date
- 2026-07-21
AI Technical Summary
Existing technologies cannot effectively predict the structure of the resulting matrix when calculating sparse matrix multiplication, resulting in wasted memory space and low computational efficiency.
By estimating the number of non-zero elements in each row of the sparse matrix multiplication result matrix, multiple threads are allocated to calculate and cache the result matrix, which is then compactly stored using intermediate memory space and finally copied to the target memory space.
It achieves faster computation speed and smaller memory footprint, improving the efficiency of sparse matrix multiplication.
Smart Images
Figure CN116561500B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of data processing technology, specifically relating to a method, apparatus, device, storage medium, and program product for calculating sparse matrices. Background Technology
[0002] In a matrix, if the number of elements with a value of 0 far exceeds the number of non-zero elements, and the distribution of non-zero elements is irregular, then the matrix is called a sparse matrix. The method for calculating the product of two sparse matrices to obtain another sparse result matrix is called sparse matrix multiplication. Sparse matrix multiplication is a key computational method used in many scientific and engineering applications, and can be applied, but is not limited to, image processing, linear algebra, and machine learning.
[0003] Since the output structure of the result matrix is unpredictable before performing sparse matrix multiplication, it is impossible to allocate appropriate memory space for the result matrix in advance. Related techniques can first precisely calculate the output structure of the result matrix for memory allocation, then calculate the result matrix and store it in the allocated memory space. However, this precise calculation consumes a significant amount of execution time. Alternatively, related techniques can estimate the upper limit of memory for each output row of the result matrix based on the number of elements in the first and second sparse matrices, and allocate memory space for the result matrix based on this upper limit. However, using this upper limit to determine the memory space for the result matrix may waste a large amount of memory. Summary of the Invention
[0004] This application proposes a method, apparatus, device, storage medium, and program product for calculating sparse matrices, which results in a matrix that occupies less memory space and has a faster calculation speed.
[0005] The first aspect of this application proposes a method for calculating sparse matrices, including:
[0006] Based on the first sparse matrix and the second sparse matrix, estimate the number of non-zero elements in each row of the resulting matrix after multiplying the first sparse matrix and the second sparse matrix.
[0007] Multiple threads are allocated to compute the result matrix. Based on the computation task of each thread and the number of non-zero elements in each row, intermediate memory space is allocated to each thread. The computation task of each thread includes computing multiple consecutive rows of the result matrix. The row result matrices computed by each thread are different.
[0008] The multiple threads execute their respective computation tasks, and the row result matrix calculated by each thread is cached in the intermediate memory space corresponding to each thread.
[0009] Allocate the target memory space based on the total number of non-zero elements cached in each intermediate memory space; copy the row result matrix cached in each intermediate memory space to the target memory space.
[0010] In some embodiments of this application, estimating the number of non-zero elements in each row of the resulting matrix after multiplying the first sparse matrix and the second sparse matrix based on the first sparse matrix and the second sparse matrix includes:
[0011] Based on the first sparse matrix and the second sparse matrix, determine the number of multiplication operations in each row of the result matrix, as well as the estimated compression ratio of the result matrix;
[0012] Based on the estimated compression ratio and the number of multiplication operations in each row of the result matrix, the number of non-zero elements in each row of the result matrix is calculated.
[0013] In some embodiments of this application, determining the estimated compression ratio of the resulting matrix based on the first sparse matrix and the second sparse matrix includes:
[0014] The first sparse matrix is sampled to obtain the sample matrix;
[0015] Multiply the sample matrix by the second sparse matrix to obtain the sample result matrix;
[0016] The estimated compression ratio of the result matrix is determined based on the total number of multiplication operations and the total number of non-zero elements in the sample result matrix.
[0017] In some embodiments of this application, the allocation of corresponding intermediate memory space for each thread based on the computational tasks of each thread and the number of non-zero elements in each row includes:
[0018] Based on the number of non-zero elements in each row and the multiple row result matrices that the first thread needs to calculate, calculate the total number of non-zero elements in the multiple row result matrices that the first thread needs to calculate; the first thread is any one of the threads.
[0019] Based on the total number of non-zero elements corresponding to the first thread, allocate intermediate memory space for the first thread.
[0020] In some embodiments of this application, caching the row result matrix calculated by each thread to the corresponding intermediate memory space includes:
[0021] Based on the fact that the data volume of the row result matrix currently calculated by the first thread is less than or equal to the capacity of the intermediate memory space corresponding to the first thread, the currently calculated row result matrix is stored in the intermediate memory space corresponding to the first thread; the first thread is any one of the threads.
[0022] Since the amount of data in the currently calculated row result matrix is greater than the capacity of the intermediate memory space corresponding to the first thread, a new intermediate memory space is allocated to the first thread to cache the row result matrix calculated by the first thread.
[0023] In some embodiments of this application, allocating new intermediate memory space for the first thread to cache the row result matrix calculated by the first thread includes:
[0024] Allocate a new intermediate memory space for the first thread, the capacity of which is greater than the capacity of the intermediate memory space previously allocated to the first thread;
[0025] The multi-row result matrix calculated by the first thread is cached in the new intermediate memory space, and the previously allocated intermediate memory space is released.
[0026] In some embodiments of this application, allocating new intermediate memory space for the first thread to cache the row result matrix calculated by the first thread includes:
[0027] Allocate new intermediate memory space for the first thread;
[0028] The row result matrix calculated by the first thread after allocating the new intermediate memory space is cached in the new intermediate memory space.
[0029] In some embodiments of this application, copying the row result matrix cached in each intermediate memory space to the target memory space includes:
[0030] Determine the start and end row numbers of the multi-row result matrices cached in each of the intermediate memory spaces;
[0031] Based on the start and end row numbers corresponding to each intermediate memory space, the multi-row result matrix cached in each intermediate memory space is copied sequentially to the target memory space.
[0032] A second aspect of this application provides a sparse matrix computing apparatus, comprising:
[0033] The estimation module is used to estimate the number of non-zero elements in each row of the result matrix of the multiplication of the first sparse matrix and the second sparse matrix based on the first sparse matrix and the second sparse matrix.
[0034] The allocation module is used to allocate multiple threads for calculating the result matrix. Based on the calculation task of each thread and the number of non-zero elements in each row, the module allocates intermediate memory space for each thread. The calculation task of each thread includes calculating multiple consecutive rows of the result matrix. The row result matrices calculated by each thread's calculation task are different.
[0035] The calculation module is used to execute the respective calculation tasks through the multiple threads, and cache the row result matrix calculated by each thread into the intermediate memory space corresponding to each thread.
[0036] The copy module is used to allocate a target memory space for the result matrix based on the total number of non-zero elements cached in each intermediate memory space; and to copy the row result matrix cached in each intermediate memory space to the target memory space.
[0037] An embodiment of the third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.
[0038] An embodiment of the fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.
[0039] An embodiment of the fifth aspect of this application provides a computer program product including a computer program that is executed by a processor to implement the method described in the first aspect.
[0040] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:
[0041] In this embodiment, the number of non-zero elements in each row of the result matrix obtained by multiplying the first and second sparse matrices is first estimated. Then, multiple threads are allocated to calculate the result matrix, and intermediate memory space is allocated to each thread based on its calculation task and the number of non-zero elements in each row. Each thread then executes its own calculation task, which includes calculating multiple consecutive rows of the result matrix, and caches the calculated row result matrices in their respective intermediate memory spaces. Next, a target memory space is allocated to the result matrix based on the total number of non-zero elements cached in each intermediate memory space, and the cached row result matrices are copied to the target memory space. Thus, by estimating the number of non-zero elements in each row of the result matrix, a faster calculation speed can be achieved compared to precisely calculating the number of non-zero elements in each row. By having multiple threads calculate multiple consecutive row result matrices and allocating intermediate memory space to each thread based on the estimated number of non-zero elements in each row, the calculated row result matrices can be cached contiguously and compactly in the intermediate memory space, thereby saving the intermediate memory space occupied by the row result matrices. Furthermore, the row result matrices stored in each intermediate memory space are continuous and compact, which can utilize the processor's maximum transmission bandwidth to copy the row result matrices cached in each intermediate memory space to a more precise target memory space, thereby further improving the calculation speed of sparse matrix multiplication and saving memory space.
[0042] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0043] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.
[0044] In the attached diagram:
[0045] Figure 1 A flowchart illustrating a sparse matrix calculation method provided in an embodiment of this application is shown.
[0046] Figure 2 A flowchart illustrating step S1 in one embodiment of this application is shown;
[0047] Figure 3 A flowchart illustrating step S11 in one embodiment of this application is shown;
[0048] Figure 4aThe diagram illustrates the storage state of the intermediate memory space using the sparse matrix calculation method provided in this embodiment.
[0049] Figure 4b This diagram illustrates the storage state of the intermediate memory space when multiple threads calculate a matrix of consecutive rows of results, based on an upper limit of memory allocation.
[0050] Figure 4c This diagram illustrates the storage state of the intermediate memory space when multiple threads calculate the row result matrix row by row, and intermediate memory space is allocated for each row result matrix.
[0051] Figure 5 A flowchart illustrating another method for calculating a sparse matrix provided in an embodiment of this application is shown.
[0052] Figure 6 A schematic diagram of the architecture of a sparse matrix computing device provided in one embodiment of this application is shown;
[0053] Figure 7 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;
[0054] Figure 8 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation
[0055] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.
[0056] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.
[0057] Multiplying two sparse matrices results in a sparse matrix. Since sparse matrices contain many zero elements, matrix multiplication involves numerous invalid operations, leading to high time complexity. Therefore, sparse matrix multiplication can be optimized to reduce time complexity and improve efficiency. Currently, most algorithms employ row-matrix multiplication dataflow to optimize the multiplication of two sparse matrices. Specifically, the calculation process of the row-matrix multiplication dataflow algorithm is as follows: First, determine the non-zero elements in the first row of the first sparse matrix. Then, determine the corresponding rows in the second sparse matrix based on the column indices of these non-zero elements; these are called target rows. Next, multiply the non-zero elements in each target row of the second sparse matrix by the non-zero elements in the first row of the first sparse matrix, obtaining multiple intermediate lists. Add the results from the non-zero elements with the same column indices in each intermediate list to obtain multiple non-zero result elements. These result elements form the non-zero elements in the first row of the result matrix. Then, repeat this process to determine the non-zero elements in the remaining rows of the result matrix.
[0058] Taking the first sparse matrix A and the second sparse matrix B as examples, when calculating the result matrix C of A×B, the calculation process of the first row of the result matrix C is as follows: First, determine the non-zero element 1 in the first row of the first sparse matrix A, and the row in the second sparse matrix B corresponding to the column index (first column) where 1 is located, that is, the first row of the second sparse matrix B. Multiply element 1 with the non-zero elements 2 and 1 in the first row of the second sparse matrix B respectively to obtain an intermediate list containing two elements, that is, 1×2=2 and 1×1=1. Since this list is based on the product of different column indices, the first row of the result matrix C includes two non-zero elements 2 and 1, which correspond to the column indices of the non-zero elements 2 and 1 in the first row of the second sparse matrix B respectively. Then the first row of the result matrix C is [20 0 1 0 0]. The calculation process for the second row of the resulting matrix C is as follows: First, determine the non-zero elements 3 and 5 in the second row of the first sparse matrix A, and the rows in the second sparse matrix B corresponding to the column indices of 3 and 5, namely the fourth and sixth rows of the second sparse matrix B. Multiply the elements 3 and 5 by the non-zero element 8 in the fourth row of the second sparse matrix B, and by the non-zero elements 1 and 5 in the sixth row, respectively, to obtain two intermediate lists, namely 3×8=24, 5×1=5, and 5×5=25. Then, merge the elements in the two lists based on the column indices of the elements 8, 1, and 5. Since 8 and 1 both belong to the second column, the second row of the resulting matrix C includes two non-zero elements 29(24+5) and 25, which correspond to the column indices of the elements 8 and 5 in the second sparse matrix B, respectively. The second row of the resulting matrix C is [0 0 290 25 0]. Then, calculate the remaining rows of the resulting matrix C according to the above calculation process. Finally, the resulting matrix C is shown below.
[0059]
[0060] When performing sparse matrix multiplication, memory space needs to be allocated in advance for storing the calculated result matrix. To save memory space, sparse matrices can be stored in compressed formats, such as COO (coordinate format), CSR (compressed sparse row format), and CSC (compressed sparse column format). The COO format stores the non-zero elements of the sparse matrix as coordinates, using ternary arrays: values (values, abbreviated val), rows (rows), and columns (columns, abbreviated col). Values represents real or complex numbers, including the non-zero elements of the sparse matrix, and can be arranged in any order. Rows represents the row of each data point in the values array, and columns represents the column of each data point in the values array. For example, the first sparse matrix A mentioned above, stored in COO format, has the following data structure: val: 1 3 5 2 4 7 6 8 9; rows (0 represents the first row): 0 1 1 2 3 3 4 5 5; col (0 represents the first column): 0 3 5 1 0 4 3 2 5.
[0061] The CSR format is an improvement on the COO format. It requires elements to be stored in row-major order, but elements within each row can be stored out of order. It does not require recording row indices for all elements; it only needs to represent the starting position of each row's elements. The CSR format further compresses the `row` array of the three arrays in the COO format, while keeping the other two arrays unchanged. The three arrays used are `row_ptr`, `columns`, and `values`. The `columns` and `values` arrays are the same as in the COO format. `row_ptr` stores the offset of the first non-zero element in each row from the first element in the `values` array. For example, the first sparse matrix A mentioned above, stored in CSR format, has the following data structure: `val: 1 3 5 2 4 7 6 8 9; col (0 represents the first column): 0 3 5 10 4 3 2 5; row_ptr (0 represents the first row): 0 1 3 4 6 7 9.` Therefore, the total number of non-zero elements in a sparse matrix stored in CSR format, i.e., the last value of the row_ptr array, is equal to the lengths of the val and col arrays. CSC format is similar to CSR format, except that elements are further compressed and stored column-wise.
[0062] This embodiment improves upon the aforementioned row multiplication dataflow algorithm and can use the CSR format for storage. Specifically, memory space can be allocated for the result matrix based on the non-zero elements of the result matrix. When multiple threads jointly perform the multiplication of two sparse matrices, each thread is used to calculate multiple row result matrices. Therefore, corresponding memory space needs to be allocated for the row result matrices calculated by each thread, i.e., memory space is allocated based on the non-zero elements in the row result matrices calculated by each thread.
[0063] However, without matrix multiplication, accurately calculating the number of non-zero elements in each row of the result matrix requires traversing the elements of both sparse matrices, which consumes a significant amount of execution time. Related techniques can also estimate the upper limit of memory for each output row of the result matrix based on the non-zero elements in the first and second sparse matrices, and allocate memory space for the result matrix accordingly. However, using this upper limit to determine the memory space for the result matrix may waste a considerable amount of memory.
[0064] Based on this, this application proposes a sparse matrix calculation method for performing multiplication of a first sparse matrix and a second sparse matrix, which is an improvement on the aforementioned row multiplication dataflow algorithm. This method first estimates the number of non-zero elements in each row of the result matrix from the multiplication of the first and second sparse matrices. Then, multiple threads are allocated to calculate the result matrix, and intermediate memory spaces are allocated to each thread based on its calculation task and the number of non-zero elements in each row. Each thread then executes its own calculation task, which includes calculating multiple consecutive rows of the result matrix, and caches the row result matrices calculated by each thread in its corresponding intermediate memory space. Finally, based on the total number of non-zero elements cached in each intermediate memory space, a target memory space is allocated for the result matrix, and the row result matrices cached in each intermediate memory space are copied to the target memory space.
[0065] Thus, by estimating the number of non-zero elements in each row of the result matrix, a faster computation speed can be achieved compared to precisely calculating the number of non-zero elements in each row. By using multiple threads to compute consecutive rows of result matrices and allocating intermediate memory space to each thread based on the estimated number of non-zero elements in each row, the multiple row result matrices computed by each thread can be cached contiguously and compactly in the intermediate memory space, thereby saving the intermediate memory space occupied by the row result matrices. Furthermore, the contiguous and compact storage of row result matrices in each intermediate memory space allows the processor's maximum bandwidth to be utilized to copy the cached row result matrices from each intermediate memory space to a more precise target memory space, further improving the computation speed of sparse matrix multiplication and saving memory space.
[0066] In this calculation, each thread produces a different row result matrix; that is, there are no duplicate row result matrices in the calculation tasks of different threads. This can be understood as each row result matrix being calculated by only one thread. The target memory space can be precisely calculated based on the number of non-zero elements in the row result matrix cached in the corresponding intermediate memory space. Therefore, the target memory space is often less than or equal to the corresponding intermediate memory space.
[0067] The sparse matrix in this embodiment can be any sparse matrix, including but not limited to sparse matrices in algorithm libraries, or sparse matrices constructed during actual applications. For example, when calculating interpersonal relationships in neural networks, the relationship between the research subject and others can be represented by numbers 0-9 according to the degree of intimacy, with higher numbers indicating greater intimacy. If the research subject is family, it can be recorded as 9; a good friend as 8; a colleague as 6, and so on; and if they do not know each other, it is recorded as 0. Since an individual's acquaintance is limited, when the research scope is large, there will be many people in the research scope who do not know the research subject. Therefore, the relationship between the research subject and others can be regarded as a sparse matrix.
[0068] Understandably, given that the number of cores in a Central Processing Unit (CPU) or other multi-core processor is often far less than the number of rows in a sparse matrix, it is not feasible to use many threads to calculate each row of the result matrix one-to-one when performing multiplication of two sparse matrices. Therefore, executing the sparse matrix calculation method described above on the CPU, with each thread calculating multiple consecutive rows of the result matrix, obviously achieves the goal of smaller memory footprint and faster calculation speed. However, this method can also be executed on a graphics processing unit (GPU) or other many-core processors, and this embodiment does not specifically limit it to this.
[0069] Please see Figure 1 This is a flowchart illustrating the sparse matrix calculation method provided in this application embodiment, as shown below. Figure 1 As shown, the method may include the following steps:
[0070] Step S1: Based on the first sparse matrix and the second sparse matrix, estimate the number of non-zero elements in each row of the resulting matrix after multiplying the first sparse matrix and the second sparse matrix.
[0071] In practical applications, the first and second sparse matrices can contain tens of thousands, or even more, of elements. Accurately calculating the number of non-zero elements in each row of the resulting matrix requires traversing all elements in both matrices, which is extremely labor-intensive and computationally inefficient. Using estimation methods can effectively reduce the workload and improve computational speed.
[0072] In some embodiments, such as Figure 2 As shown, step S1 above may include the following processing: step S11, based on the first sparse matrix and the second sparse matrix, determine the number of multiplication operations in each row of the result matrix and the estimated compression ratio of the result matrix; step S12, based on the estimated compression ratio and the number of multiplication operations in each row of the result matrix, calculate the number of non-zero elements in each row of the result matrix.
[0073] The number of multiplication operations, also known as the number of floating-point operations (flop), represents the number of floating-point multiplication operations performed when calculating a result matrix or a row of a result matrix. Specifically, the flop of a row in the result matrix is equal to the number of multiplication operations performed between the non-zero element of a row in the left matrix and the non-zero element of the row corresponding to that column index in the right matrix. The total flop of the result matrix is equal to the sum of the flop of each row. For example, when multiplying the first sparse matrix A and the second sparse matrix B, the flop of the first row of the result matrix C is equal to 2, and the flop of the second row is equal to 3.
[0074] Compression ratio (CR) includes the overall compression ratio of a sparse matrix and the compression ratio of a single row. The compression ratio of a single row is the ratio of the number of flops in a row of the sparse matrix to the number of non-zero elements in that row; the overall compression ratio is the ratio of the total number of flops in the sparse matrix to the total number of non-zero elements. The number of non-zero elements can be represented by Nnz (the number of non-zero elements of an amatrix or a row of a matrix), then the compression ratio CR = flop / Nnz.
[0075] In this embodiment, without actually calculating the result matrix, the exact number of non-zero elements in each row of the result matrix and the exact compression ratio are unknown. To allocate more accurate memory space, an estimated compression ratio can be used. Then, based on the aforementioned compression ratio formula, the number of non-zero elements in each row of the result matrix is determined according to the calculated flop and the estimated compression ratio, thereby improving computational efficiency. Furthermore, when calculating the estimated number of non-zero elements in each row of the obtained result matrix, the calculation is based on the estimated compression ratio of the sample result matrix. Compared to directly dividing the number of non-zero elements in the sample result matrix by the sampling coefficient, this method yields a more accurate number of non-zero elements, thus improving the accuracy of memory space allocation.
[0076] It is understood that the above method of calculating the number of non-zero elements in each row of the result matrix by estimating the compression ratio and the number of multiplication operations in each row of the result matrix is only one implementation method of this embodiment. This embodiment is not limited to this. For example, the number of non-zero elements in each row of the result matrix can also be the larger of the number of non-zero elements in one row of the first sparse matrix and the number of non-zero elements in the corresponding row of the second sparse matrix, or it can be based on multiplying the larger one by a coefficient greater than 1, etc.
[0077] Furthermore, such as Figure 3 As shown, step S11 above may include the following processing: step S111, sampling the first sparse matrix to obtain a sample matrix; step S112, multiplying the sample matrix with the second sparse matrix to obtain a sample result matrix; step S113, determining the estimated compression ratio of the result matrix based on the total number of multiplication operations and the total number of non-zero elements in the sample result matrix.
[0078] In practical applications, the first sparse matrix can be sampled. For example, if the first sparse matrix has 10,000 rows, 300 rows can be sampled to obtain a sample matrix. Then, the sample matrix is multiplied by the second sparse matrix to obtain the sample result matrix. Since the compression ratio of the sparse matrix is often relatively stable, the overall compression ratio and the compression ratio of each row of the result matrix are often close to the compression ratio of the sample result matrix. Therefore, the overall compression ratio of the sample result matrix can be regarded as the estimated compression ratio of the result matrix. That is, the ratio of the total number of multiplication operations to the total number of non-zero elements in the sample result matrix is determined as the estimated compression ratio of the result matrix. Then, according to the compression ratio calculation formula mentioned above, based on the estimated compression ratio and the calculated number of multiplication operations in each row of the result matrix, the number of non-zero elements in each row of the result matrix can be obtained.
[0079] It is understood that the above method of determining the estimated compression ratio of the result matrix based on the total number of multiplication operations and the total number of non-zero elements in the sample result matrix is only one implementation method of this embodiment. This embodiment is not limited to this. For example, it is also possible to further consider the compression ratio of each row of the sample result matrix as the estimated compression ratio of the corresponding row of the result matrix.
[0080] Step S2: Allocate multiple threads for calculating the result matrix. Based on the calculation task of each thread and the number of non-zero elements in each row, allocate corresponding intermediate memory space for each thread.
[0081] The computation task of each thread includes the computation of multiple consecutive rows in the result matrix. Each thread computes a different row result matrix, meaning that there are no duplicate rows in the row result matrix computed by each thread.
[0082] In this embodiment, multiple consecutive rows of the result matrix are assigned to a single thread, allowing these multiple rows to form a continuous and compact overall data structure. This eliminates the need to record the row numbers and offsets of intermediate rows, thereby improving overall compression performance and saving intermediate memory space when storing the result matrix.
[0083] Specifically, based on the number of non-zero elements in each row of the result matrix, the computation tasks of each thread can be configured according to the load balancing principle, so that the computation tasks of each thread are comparable, thereby further shortening the overall computation time.
[0084] In some embodiments, step S2 may include the following processing: calculating the total number of non-zero elements in the multiple row result matrices to be calculated by the first thread based on the number of non-zero elements in each row and the multiple row result matrices to be calculated by the first thread's computation task; the first thread is any thread among the threads used to perform the calculation of the result matrices. Based on the total number of non-zero elements corresponding to the first thread, intermediate memory space is allocated for the first thread.
[0085] In practical applications, a global array with global scope can be used to record the start and end line numbers corresponding to the computation tasks of each thread. When allocating intermediate memory space for each thread, the target row result matrix to be computed by each thread can be determined based on the start and end line numbers. The number of non-zero elements in each row result matrix of the target row result matrix is added together to obtain the total number of non-zero elements in the target row result matrix. Then, intermediate memory space is allocated to the corresponding thread based on the total number of non-zero elements.
[0086] Specifically, when allocating intermediate memory space for the corresponding thread based on the total number of non-zero elements, a magnification factor greater than 1 can be preset. The total number of non-zero elements is multiplied by this magnification factor to appropriately magnify the total number of non-zero elements. Then, intermediate memory space is allocated according to the magnified total number of non-zero elements, so that the allocated intermediate memory space is as large as possible or equal to the actual memory space required by the row result matrix calculated by each thread.
[0087] Step S3: Execute each computation task by multiple threads and cache the row result matrix calculated by each thread in the intermediate memory space corresponding to each thread.
[0088] After allocating corresponding computation tasks and intermediate memory spaces to each thread, multiple threads execute their respective computation tasks. The target row of the first sparse matrix can be determined according to the row number in each computation task. Then, the non-zero elements in the target row of the first sparse matrix are multiplied by the non-zero elements in the target row of the second sparse matrix corresponding to their column indices. The row result matrices calculated by each thread are cached in the intermediate memory space corresponding to each thread.
[0089] likeFigure 4a The diagram shows the storage state of the intermediate memory space after the row result matrix calculated by each thread is cached in the intermediate memory space corresponding to each thread, using the sparse matrix calculation method provided in this embodiment; Figure 4b The diagram illustrates the storage state of the intermediate memory space when multiple threads compute consecutive row result matrices based on the memory limit, and the row result matrices computed by each thread are cached in their respective intermediate memory spaces. Figure 4c The diagram illustrates the storage state of the intermediate memory space after multiple threads calculate the row result matrix row by row, allocating intermediate memory space for each row result matrix, and caching the row result matrix calculated by each thread into its corresponding intermediate memory space. Figures 4a-4c As can be seen, when applying the sparse matrix calculation method provided in this embodiment, the row result matrix stored in the intermediate memory space is continuous and compact, and the remaining space is very small. When allocating intermediate memory space based on the memory limit, and multiple threads calculate multiple consecutive row result matrices, the row result matrix stored in the intermediate memory space is also continuous and compact, but the remaining space is large, resulting in some memory waste. When multiple threads calculate the row result matrix row by row and allocate intermediate memory space for each row result matrix, not only is the row result matrix stored in the intermediate memory space sparse and discontinuous, but the remaining space is also large, resulting in significant memory waste.
[0090] In some embodiments, step S3 may include the following processing: storing the currently calculated row result matrix in the intermediate memory space corresponding to the first thread based on the fact that the data volume of the currently calculated row result matrix is less than or equal to the capacity of the intermediate memory space corresponding to the first thread; allocating a new intermediate memory space for the first thread based on the fact that the data volume of the currently calculated row result matrix is greater than the capacity of the intermediate memory space corresponding to the first thread, so as to cache the row result matrix calculated by the first thread.
[0091] Here, "thread one" refers to any one of the threads, which can be understood as a general term for a particular thread. That is, the process can be applied to any thread; this explanation uses only thread one as an example. The data volume of the currently calculated row result matrix can be understood as the total data volume of all row result matrices that the thread is currently calculating and has already calculated.
[0092] In this embodiment, each thread can perform calculations and store data simultaneously during the execution of its computational tasks, that is, cache the output results of the calculations to an intermediate memory space in real time or periodically. For example... Figure 5As shown, before caching, it can be determined whether the data size of the currently calculated row result matrix is less than or equal to the capacity of the intermediate memory space corresponding to the first thread. If so, the currently calculated row result matrix can be stored in the intermediate memory space corresponding to the first thread; otherwise, a new intermediate memory space can be allocated for the first thread to cache the row result matrix calculated by the first thread.
[0093] Furthermore, taking the first thread as an example, when allocating new intermediate memory space for the first thread, the capacity of the new intermediate memory space can be greater than the capacity of the intermediate memory space previously allocated for the first thread. Then, the multi-row result matrix calculated by the first thread is cached in the new intermediate memory space, and the previously allocated intermediate memory space is released.
[0094] In this embodiment, the newly allocated intermediate memory space for the first thread is larger than the previously allocated intermediate memory space to accommodate the multi-row result matrix calculated by the first thread. The specific capacity of the new intermediate memory space can be estimated based on the current computational workload. For example, if the first thread's computation task includes 1000 rows, and it is found that the previously allocated intermediate memory space can no longer accommodate the current amount of data when 500 rows are computed, then a larger new intermediate memory space can be allocated for the first thread. The new intermediate memory space can be 2.2-2.5 (or even larger) times the previously allocated intermediate memory space, so that the new intermediate memory space can accommodate all the multi-row result matrices corresponding to the first thread's computation task as much as possible.
[0095] In other embodiments, taking the first thread as an example, when allocating a new intermediate memory space for the first thread, the row result matrix calculated by the first thread after allocating the new intermediate memory space can be cached in the new intermediate memory space.
[0096] In this embodiment, the capacity of the newly allocated intermediate memory space can be estimated based on the current computational load to accommodate the remaining row result matrix to be computed. The row result matrix computed before the allocation of the new intermediate memory space remains cached in the previously allocated intermediate memory space. If the capacity of the new intermediate memory space is insufficient to accommodate the remaining row result matrix to be computed, another new intermediate memory space can be allocated until it can accommodate the remaining row result matrix to be computed. For example, if the computation task of the first thread includes 1000 rows, and after computed 700 rows, it is found that the previously allocated intermediate memory space can no longer accommodate the current amount of data being computed, then a new intermediate memory space can be allocated for the first thread. The new intermediate memory space can be 0.5-0.7 times the size of the previously allocated intermediate memory space to accommodate the remaining 300 row result matrix. If the capacity of the new intermediate memory space is insufficient to accommodate the remaining 300 row result matrix, another new intermediate memory space can be allocated until it can accommodate the remaining 300 row result matrix.
[0097] It is understood that the specific size of the new intermediate memory space can be larger or smaller than the size of the previously allocated intermediate memory space. This embodiment does not make a specific limitation on this.
[0098] Step S4: Allocate target memory space based on the total number of non-zero elements cached in each intermediate memory space; copy the row result matrix cached in each intermediate memory space to the target memory space.
[0099] The target memory space can be understood as the minimum memory space that can accommodate the currently cached result matrix. The capacity of the target memory space can be determined based on the total number of non-zero elements in the result matrix and the storage format of the result matrix. For example, in the CSR format, three arrays are required: row_ptr, columns, and values. The lengths of the columns array and the values array are both equal to the total number of non-zero elements in the result matrix. The length of the row_ptr array is equal to the number of rows in the result matrix plus 1. The value of the last element in the row_ptr array is the total number of non-zero elements in the result matrix.
[0100] In this embodiment, after caching the row result matrix calculated by each thread to the corresponding intermediate memory space, the required target memory space can be allocated to each thread according to the total number of non-zero elements in the row result matrix calculated by each thread. That is, each thread corresponds to an intermediate memory space and also to a target memory space. The row result matrix cached in each intermediate memory space can be stored in the corresponding target memory space, thereby releasing the remaining space in the intermediate memory space and further saving memory space.
[0101] In some embodiments, step S4 may include the following processing: determining the start and end row numbers of the multi-row result matrix cached in each intermediate memory space; and copying the multi-row result matrix cached in each intermediate memory space to the target memory space in sequence based on the start and end row numbers corresponding to each intermediate memory space.
[0102] In this embodiment, when copying the multi-row result matrix to the target memory space, the amount of data is large, and the multi-row result matrix is compact and continuous to form a data block. This allows the processor's maximum transmission bandwidth to be utilized to quickly copy the row result matrix cached in each intermediate memory space to a more accurate target memory space, which can further improve the calculation speed of sparse matrices and further save memory space.
[0103] Based on the same inventive concept as the above-described sparse matrix calculation method, this embodiment also provides a sparse matrix calculation apparatus for executing the sparse matrix calculation method, such as... Figure 6 As shown, the device includes:
[0104] The estimation module is used to estimate the number of non-zero elements in each row of the result matrix of the multiplication of the first sparse matrix and the second sparse matrix, based on the first sparse matrix and the second sparse matrix.
[0105] The allocation module is used to allocate multiple threads for calculating the result matrix. Based on the calculation task of each thread and the number of non-zero elements in each row, it allocates the corresponding intermediate memory space for each thread. The calculation task of a thread includes calculating multiple consecutive rows of the result matrix. The row result matrices calculated by the calculation task of each thread are different.
[0106] The calculation module is used to execute the respective calculation tasks through multiple threads, and cache the row result matrix calculated by each thread to the intermediate memory space corresponding to each thread.
[0107] The copy module is used to allocate a target memory space for the result matrix based on the total number of non-zero elements cached in each intermediate memory space; and to copy the row result matrix cached in each intermediate memory space to the target memory space.
[0108] This application also provides an electronic device for performing the above-described sparse matrix calculation method. Please refer to... Figure 7 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 7As shown, the electronic device 4 includes: a processor 400, a memory 401, a bus 402, and a communication interface 403. The processor 400, the communication interface 403, and the memory 401 are connected via the bus 402. The memory 401 stores a computer program that can run on the processor 400. When the processor 400 runs the computer program, it executes the sparse matrix calculation method provided in any of the foregoing embodiments of this application.
[0109] The memory 401 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 403 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.
[0110] Bus 402 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 401 is used to store programs. After receiving an execution instruction, processor 400 executes the program. The sparse matrix calculation method disclosed in any of the foregoing embodiments of this application can be applied to processor 400, or implemented by processor 400.
[0111] The processor 400 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 400 or by instructions in software form. The processor 400 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 401. The processor 400 reads the information in memory 401 and, in conjunction with its hardware, completes the steps of the above method.
[0112] The electronic device provided in this application embodiment and the sparse matrix calculation method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.
[0113] This application also provides a computer-readable storage medium corresponding to the sparse matrix calculation method provided in the foregoing embodiments. Please refer to [link / reference]. Figure 8 The computer-readable storage medium shown is an optical disc 50, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the sparse matrix calculation method provided in any of the foregoing embodiments.
[0114] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.
[0115] This application also provides a computer program product, including a computer program that is executed by a processor to implement the sparse matrix calculation method described in any of the above embodiments.
[0116] The computer-readable storage medium and computer program product provided in the above embodiments of this application are based on the same inventive concept as the sparse matrix calculation method provided in the embodiments of this application, and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.
[0117] It should be noted that:
[0118] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0119] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.
[0120] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
[0121] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for calculating sparse matrices, characterized in that, include: Based on the first sparse matrix and the second sparse matrix, estimate the number of non-zero elements in each row of the resulting matrix after multiplying the first sparse matrix and the second sparse matrix. Multiple threads are allocated to compute the result matrix. Based on the computation task of each thread and the number of non-zero elements in each row, intermediate memory space is allocated to each thread. The computational task of each thread includes calculating multiple consecutive rows of the result matrix, and the row result matrices calculated by each thread are different; The multiple threads execute their respective computation tasks, and the row result matrix calculated by each thread is cached in the intermediate memory space corresponding to each thread. Allocate the target memory space based on the total number of non-zero elements cached in each intermediate memory space; copy the row result matrix cached in each intermediate memory space to the target memory space.
2. The method according to claim 1, characterized in that, The step of estimating the number of non-zero elements in each row of the matrix resulting from the multiplication of the first and second sparse matrices, based on the first and second sparse matrices, includes: Based on the first sparse matrix and the second sparse matrix, determine the number of multiplication operations in each row of the result matrix, as well as the estimated compression ratio of the result matrix; Based on the estimated compression ratio and the number of multiplication operations in each row of the result matrix, the number of non-zero elements in each row of the result matrix is calculated.
3. The method according to claim 2, characterized in that, The step of determining the estimated compression ratio of the result matrix based on the first sparse matrix and the second sparse matrix includes: The first sparse matrix is sampled to obtain the sample matrix; Multiply the sample matrix by the second sparse matrix to obtain the sample result matrix; The estimated compression ratio of the result matrix is determined based on the total number of multiplication operations and the total number of non-zero elements in the sample result matrix.
4. The method according to claim 1, characterized in that, The intermediate memory space allocated to each thread based on the computational tasks of each thread and the number of non-zero elements in each row includes: Based on the number of non-zero elements in each row and the multiple row result matrices that the first thread needs to calculate, calculate the total number of non-zero elements in the multiple row result matrices that the first thread needs to calculate; the first thread is any one of the threads. Based on the total number of non-zero elements corresponding to the first thread, allocate intermediate memory space for the first thread.
5. The method according to claim 1, characterized in that, The step of caching the row result matrix calculated by each thread to the corresponding intermediate memory space includes: Based on the fact that the data volume of the row result matrix currently calculated by the first thread is less than or equal to the capacity of the intermediate memory space corresponding to the first thread, the currently calculated row result matrix is stored in the intermediate memory space corresponding to the first thread; the first thread is any one of the threads. Since the amount of data in the currently calculated row result matrix is greater than the capacity of the intermediate memory space corresponding to the first thread, a new intermediate memory space is allocated to the first thread to cache the row result matrix calculated by the first thread.
6. The method according to claim 5, characterized in that, The step of allocating new intermediate memory space for the first thread to cache the row result matrix calculated by the first thread includes: Allocate a new intermediate memory space for the first thread, the capacity of which is greater than the capacity of the intermediate memory space previously allocated to the first thread; The multi-row result matrix calculated by the first thread is cached in the new intermediate memory space, and the previously allocated intermediate memory space is released.
7. The method according to claim 5, characterized in that, The step of allocating new intermediate memory space for the first thread to cache the row result matrix calculated by the first thread includes: Allocate new intermediate memory space for the first thread; The row result matrix calculated by the first thread after allocating the new intermediate memory space is cached in the new intermediate memory space.
8. The method according to claim 1, characterized in that, The step of copying the row result matrix cached in each intermediate memory space to the target memory space includes: Determine the start and end row numbers of the multi-row result matrices cached in each of the intermediate memory spaces; Based on the start and end row numbers corresponding to each intermediate memory space, the multi-row result matrix cached in each intermediate memory space is copied sequentially to the target memory space.
9. A sparse matrix computing device, characterized in that, include: The estimation module is used to estimate the number of non-zero elements in each row of the result matrix of the multiplication of the first sparse matrix and the second sparse matrix based on the first sparse matrix and the second sparse matrix. The allocation module is used to allocate multiple threads for calculating the result matrix, and allocates corresponding intermediate memory space for each thread based on the calculation task of each thread and the number of non-zero elements in each row. The computation task of a thread includes calculating multiple consecutive rows of the result matrix, and the row result matrices calculated by each thread's computation task are different; The calculation module is used to execute the respective calculation tasks through the multiple threads, and cache the row result matrix calculated by each thread into the intermediate memory space corresponding to each thread. The copying module is used to allocate target memory space for the result matrix based on the total number of non-zero elements cached in each intermediate memory space. Copy the row result matrix cached in each intermediate memory space to the target memory space.
10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method as described in any one of claims 1-8.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method as described in any one of claims 1-8.
12. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the method according to any one of claims 1-8.