Spgemm dynamic scheduling method, device, equipment and storage medium
Patent Information
- Application Number
- CN202611062701.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-16
- Publication Date
- 2026-10-27
AI Technical Summary
由于"零"元素不需要存储和计算，所以SpGEMM比普通稠密矩阵乘法复杂得多——不同的输入矩阵结构（有的行很稠密、有的行很稀疏、有的分布均匀、有的分布不均）和不同的硬件（CPU、NVIDIA GPU、AMD GPU）会导致同一种算法的性能天差地别
[0023]本申请提供一种SpGEMM动态调度方法、装置、设备、存储介质，该方法包括：提取待相乘的稀疏矩阵的关键特征和硬件平台信息；根据关键特征过滤累加器；根据硬件平台信息确定计算范式；基于过滤后的累加器和计算范式，进行稀疏通用矩阵乘法SpGEMM调度。本申请的方法，根据待相乘的稀疏矩阵的关键特征和硬件平台信息进行自动的累加器的过滤和计算范式的确定，提升了SpGEMM调度方案通用性和SpGEMM调度性能的稳定性，确保SpGEMM调度的高精度和强鲁棒性。
Smart Images
Figure CN122986235A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a SpGEMM dynamic scheduling method, apparatus, device, and storage medium. Background Technology
[0002] SpGEMM (Sparse General Matrix-Matrix Multiplication) calculates the multiplication of two matrices whose "most elements are zero," and the result also retains the "most elements are zero" format. This computation is very common in scientific simulations, graph computing, artificial intelligence, and other fields. Because the "zero" elements do not need to be stored or computed, SpGEMM is much more complex than ordinary dense matrix multiplication—different input matrix structures (some rows are dense, some are sparse, some are uniformly distributed, and some are unevenly distributed) and different hardware (CPU, NVIDIA GPU, AMD GPU) can lead to drastically different performance of the same algorithm.
[0003] Currently, when scheduling SpGEMM, users need to manually input the scheduling scheme, which makes the existing SpGEMM scheduling scheme have poor universality and unstable performance. Summary of the Invention
[0004] To address one of the aforementioned technical deficiencies, this application provides a SpGEMM dynamic scheduling method, apparatus, device, and storage medium.
[0005] The first aspect of this application provides a SpGEMM dynamic scheduling method, which includes: Extract key features and hardware platform information of the sparse matrices to be multiplied; wherein the sparse matrices to be multiplied are the first sparse matrix and the second sparse matrix; the first sparse matrix has the following number of rows. The number of columns in the first sparse matrix is The second sparse matrix has rows. The number of columns in the first sparse matrix is Key features include , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix ; Filter accumulators based on key features; Determine the computational paradigm based on hardware platform information; Based on the filtered accumulator and computation paradigm, sparse general matrix multiplication (SpGEMM) scheduling is performed.
[0006] Optionally, .
[0007] Optionally, , .
[0008] Optionally, the accumulator is filtered based on key features, including: like The remaining memory is then filtered out from the Dense accumulator and the Heap accumulator; among them, This is the memory ratio value; like First threshold, or, The first threshold filters out the Dense accumulator; like The second threshold, and The second threshold filters out all accumulators except for the Dense accumulator.
[0009] Optionally, the hardware platform information is the CPU platform; The computational paradigm is determined based on hardware platform information, including: like Third threshold and The fourth threshold determines the computation paradigm as a single-stage dense mode. like [Third threshold, fifth threshold], then the calculation paradigm is determined to be the two-stage method; like The fifth threshold determines the computational paradigm as a two-stage hybrid mode.
[0010] Optionally, the hardware platform information is a GPU platform; The computational paradigm is determined based on hardware platform information, including: like If the sixth threshold is met and the hardware supports it, then the computation paradigm is determined to be the single-stage GPU dense mode. like If the sixth threshold and the seventh threshold are used, the computation paradigm is determined to be the single-stage GPU dense mode, or the computation paradigm is determined to be the single-stage GPU dense mode and the two-stage unfold-sort-shrink mode. like (Seventh threshold, eighth threshold) then the computational paradigm is determined to be the two-stage expansion-sorting-contraction mode; like The eighth threshold, or, according to If the obtained coefficient of variation is greater than the ninth threshold, then the calculation paradigm is determined to be a two-stage hybrid mode.
[0011] Optionally, after determining the computational paradigm as a two-stage method, the method further includes: like The tenth threshold determines that the filtered accumulator is only a Hybrid accumulator; like The eleventh threshold determines the filtered accumulator, which is implemented based on a hash table. like The twelfth threshold determines whether the filtered accumulator is based on a GPU-based expand-sort-shrink implementation or a CPU-based merge sort implementation.
[0012] Optionally, if the SpGEMM scheduling process is interrupted, the filtered accumulator and / or computation paradigm are updated, and SpGEMM scheduling is performed again based on the filtered accumulator and computation paradigm.
[0013] A second aspect of this application provides a SpGEMM dynamic scheduling apparatus, the apparatus comprising: The extraction module is used to extract key features and hardware platform information of the sparse matrices to be multiplied; wherein the sparse matrices to be multiplied are a first sparse matrix and a second sparse matrix; the first sparse matrix has 10 rows. The number of columns in the first sparse matrix is The second sparse matrix has rows. The number of columns in the first sparse matrix is Key features include , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix ; The filtering module is used to filter the accumulator based on key features; The determination module is used to determine the computation paradigm based on hardware platform information; The scheduling module is used for scheduling sparse general matrix multiplication (SpGEMM) based on the filtered accumulator and computation paradigm.
[0014] Optionally, .
[0015] Optionally, , .
[0016] Optionally, the filtering module is used for... The remaining memory is then filtered out from the Dense accumulator and the Heap accumulator; among them, This is the memory ratio value; if First threshold, or, The first threshold filters out the Dense accumulator; if The second threshold, and The second threshold filters out all accumulators except for the Dense accumulator.
[0017] Optionally, the hardware platform information is the CPU platform; Determine the module, used for if Third threshold and The fourth threshold determines the computation paradigm as a single-stage dense mode; if [Third threshold, fifth threshold], then the computational paradigm is determined to be the two-stage method; if The fifth threshold determines the computational paradigm as a two-stage hybrid mode.
[0018] Optionally, the hardware platform information is a GPU platform; Determine the module, used for if If the sixth threshold is met and the hardware supports it, then the computation paradigm is determined to be the single-stage GPU dense mode; if If the sixth threshold and the seventh threshold are both met, then the computation paradigm is determined to be either a single-stage GPU dense mode or a two-stage unfold-sort-shrink mode. (Seventh threshold, eighth threshold) then the computational paradigm is determined to be a two-stage expansion-sorting-contraction model; if The eighth threshold, or, according to If the obtained coefficient of variation is greater than the ninth threshold, then the calculation paradigm is determined to be a two-stage hybrid mode.
[0019] Optionally, the device further includes: Processing module, used for if The tenth threshold determines that the filtered accumulator is only a Hybrid accumulator; if The eleventh threshold determines that the filtered accumulator is implemented based on a hash table; if... The twelfth threshold determines whether the filtered accumulator is based on a GPU-based expand-sort-shrink implementation or a CPU-based merge sort implementation.
[0020] Optionally, if the SpGEMM scheduling process is interrupted, the filtered accumulator and / or computation paradigm are updated, and the scheduling module is used to re-perform SpGEMM scheduling based on the filtered accumulator and computation paradigm.
[0021] A third aspect of this application provides an electronic device, comprising: Memory; Processor; and Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method described in the first aspect above.
[0022] In a fourth aspect, this application provides a computer-readable storage medium having a computer program stored thereon; the computer program is executed by a processor to implement the method described in the first aspect above.
[0023] This application provides a SpGEMM dynamic scheduling method, apparatus, device, and storage medium. The method includes: extracting key features and hardware platform information of the sparse matrix to be multiplied; filtering accumulators based on the key features; determining the computation paradigm based on the hardware platform information; and performing sparse general matrix multiplication (SpGEMM) scheduling based on the filtered accumulators and computation paradigm. The method of this application automatically filters accumulators and determines the computation paradigm based on the key features and hardware platform information of the sparse matrix to be multiplied, improving the versatility of the SpGEMM scheduling scheme and the stability of SpGEMM scheduling performance, ensuring high accuracy and strong robustness of SpGEMM scheduling. Attached Figure Description
[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a SpGEMM dynamic scheduling method provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of a SpGEMM dynamic scheduling device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0025] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.
[0026] In developing this application, the inventors discovered that SpGEMM (Sparse General Matrix-Matrix Multiplication) calculates the multiplication of two matrices whose "most elements are zero," and the result also maintains the "most elements are zero" format. This computation is very common in scientific simulation, graph computing, artificial intelligence, and other fields. Because "zero" elements do not require storage or computation, SpGEMM is far more complex than ordinary dense matrix multiplication—different input matrix structures (some rows are dense, some are sparse, some are evenly distributed, some are unevenly distributed) and different hardware (CPU, NVIDIA GPU, AMD GPU) can lead to drastically different performance for the same algorithm. Currently, scheduling SpGEMM requires users to manually input the scheduling scheme, which makes existing SpGEMM scheduling schemes incompatible and unstable in performance.
[0027] To address the aforementioned issues, this application provides a SpGEMM dynamic scheduling method, apparatus, device, and storage medium. The method includes: extracting key features and hardware platform information of the sparse matrix to be multiplied; filtering accumulators based on the key features; determining the computation paradigm based on the hardware platform information; and performing SpGEMM scheduling based on the filtered accumulators and computation paradigm. This method automatically filters accumulators and determines the computation paradigm based on the key features and hardware platform information of the sparse matrix to be multiplied, improving the versatility of the SpGEMM scheduling scheme and the stability of its performance, ensuring high accuracy and robustness of SpGEMM scheduling.
[0028] See Figure 1 This embodiment provides a SpGEMM dynamic scheduling method, the implementation process of which is as follows: 101. Extract key features and hardware platform information of the sparse matrix to be multiplied.
[0029] Among them, the sparse matrices to be multiplied are the first sparse matrix and the second sparse matrix.
[0030] The number of rows in the first sparse matrix is The number of columns in the first sparse matrix is The second sparse matrix has rows. The number of columns in the first sparse matrix is In other words, the first sparse matrix is The matrix, the second sparse matrix is The first sparse matrix has the same number of columns as the second sparse matrix.
[0031] In step 101, key features and hardware platform information of the first and second sparse matrices can be fully extracted.
[0032] 1. Key features of the first and second sparse matrices Key features of the first and second sparse matrices include dimension, total number of non-zero elements, average number of non-zero elements per row, and distribution variance.
[0033] Key features include , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix The standard deviation of the non-zero element distribution in the second sparse matrix rows .
[0034] Using the first sparse matrix as The matrix, the second sparse matrix is The matrix, the first row of the first sparse matrix has The second row of the first sparse matrix contains non-zero elements. The n non-zero elements, ..., the nth non-zero element of the first sparse matrix. Xingyou The first row of the second sparse matrix contains non-zero elements. The second row of the second sparse matrix contains non-zero elements. The nth non-zero element, ..., the nth non-zero element of the second sparse matrix. Xingyou Take a non-zero element as an example.
[0035] (1) Obtain the number of rows of the first sparse matrix. The first sparse matrix is the number of columns or the second sparse matrix is the number of rows. The number of columns in the second sparse matrix .
[0036] (2) Determine the total number of non-zero elements in the first sparse matrix. Determine the total number of non-zero elements in the second sparse matrix. .
[0037] (3) Determine the number of non-zero element predictions in the result matrix .
[0038] (4) Determine the average number of non-zero elements in the first sparse matrix. Determine the average number of non-zero elements in the rows of the second sparse matrix. .
[0039] (5) Determine the standard deviation of the non-zero element distribution of the first sparse matrix rows. Determine the standard deviation of the non-zero element distribution of the rows of the second sparse matrix. .
[0040] 2. Hardware platform information Hardware platform information, such as CPU (Central Processing Unit) platform, or GPU (Graphics Processing Unit) platform.
[0041] In addition, step 101 can also extract key resource information, such as memory capacity and storage capacity.
[0042] Hardware platform information and key resource information can reflect the current execution environment.
[0043] In practice, step 101 can be implemented using a lightweight scan with O(n) complexity (which can be achieved using sampling techniques).
[0044] Since the scheduling method includes a computational paradigm and an accumulator, after extracting the key features and hardware platform information of the sparse matrix to be multiplied in step 101, the accumulator is filtered in step 102, and the computational paradigm is determined in step 103. In this way, the filtered accumulator and the determined computational paradigm constitute the fastest and most memory-efficient scheduling scheme.
[0045] 102, Filter accumulator based on key features.
[0046] In step 102, hard rules are applied to filter out infeasible accumulators (such as those with memory limits).
[0047] The implementation process of step 102 is as follows: 102-1, if If there is remaining memory, disable accumulators that consume a lot of memory, i.e., filter out Dense accumulators and Heap accumulators.
[0048] in, This is the memory ratio value. It is experience value, such as Or 0.6. If the memory size occupied by the intermediate variable is 30%, then a 30% margin is reserved for the intermediate variable. .
[0049] The remaining memory is the available memory.
[0050] by For example, if Then the Dense accumulator and the Heap accumulator will be filtered out.
[0051] 102-2, if The first threshold, or, The first threshold filters out Dense accumulators.
[0052] The first threshold is an empirical value, such as 512 or 256. Based on the current hardware characteristics (mainly cache size), the computation time of multiple sample sparse matrices (with dimensions at least from 4 to 2048) under the first threshold of 512 and 256 is measured, and the threshold with the shortest computation time is selected. For example, if 512 has a shorter overall computation time, then the first threshold is 512.
[0053] Taking a first threshold of 512 as an example, if ,or, Then the Dense accumulator will be filtered out.
[0054] 102-3, if The second threshold, and The second threshold indicates that the sparse matrix is extremely small, which can filter out all accumulators except for the Dense accumulator. In other words, the Dense accumulator is selected directly.
[0055] The second threshold is an empirical value, such as 1000.
[0056] Taking a second threshold of 1000 as an example, if ,and If so, then directly select the Dense accumulator.
[0057] 103. Determine the computation paradigm based on the hardware platform information.
[0058] The computational paradigm can be based on And hardware platform information is determined. And based on the sparse matrix (especially...) This determines whether to adopt a single-stage or multi-stage computation paradigm. For example: (a) If the hardware platform information is a CPU platform, then the implementation process of step 103 is as follows: 201, if Third threshold and The fourth threshold determines the computation paradigm as a single-phase dense mode (i.e., 1-phase (Dense)).
[0059] The third threshold is an empirical value, such as 16. The fourth threshold is also an empirical value, such as... .
[0060] With the third threshold set at 16 and the fourth threshold set at... For example, if and Then the computational paradigm is determined to be 1-phase (Dense).
[0061] 202, if [Third threshold, fifth threshold], then the calculation paradigm is determined to be the two-phase method (i.e., 2-phase).
[0062] The fifth threshold is an empirical value, such as 100.
[0063] Taking a third threshold of 16 and a fifth threshold of 100 as an example, if Then the computational paradigm is determined to be 2-phase.
[0064] In the classic SpGEMM algorithm, the 2-phase two-stage method employs a "count first, then calculate" strategy. Specifically, in the first phase (symbolic analysis phase), the non-zero element structure of the first and second sparse matrices is traversed, and only the number of non-zero elements in each row of the resulting matrix C is calculated (symbolic prediction / counting). In the second phase (numerical calculation phase), memory is allocated based on the row offsets obtained in the first phase, and then the numerical multiplication and addition operations are actually performed to obtain the value of the resulting matrix C.
[0065] 203, if The fifth threshold determines the computational paradigm as a two-phase hybrid mode (i.e., 2-phase Hybrid).
[0066] Taking the fifth threshold of 100 as an example, if Therefore, the computational paradigm is determined to be 2-phase Hybrid.
[0067] In a two-phase framework, the 2-phase Hybrid uses a merge-based algorithm for dense rows in the result matrix, while switching to a hash table- or accumulator-based algorithm for sparse rows. Both phases dynamically select the optimal kernel based on the row sparsity, rather than using a single strategy throughout.
[0068] (ii) If the hardware platform information is a GPU platform, then the implementation process of step 103 is as follows: 301, if If the sixth threshold is met and the hardware supports it, then the computation paradigm is determined to be the single-phase GPU dense mode (i.e., 1-phase (GPU Dense)).
[0069] The sixth threshold is an empirical value, such as 32.
[0070] Taking the sixth threshold of 32 as an example, if If the hardware supports it, then the computational paradigm is determined to be 1-phase (GPU Dense).
[0071] In 1-phase (GPU Dense), when the GPU detects that some rows of the result matrix are very dense, the traditional symbol counting stage is skipped, and a sufficiently large fixed-size buffer is pre-allocated for these rows (or a dense accumulator is used). Non-zero element statistics and numerical calculations are completed simultaneously in a kernel call, avoiding the overhead of two global synchronizations.
[0072] 302, if If the sixth threshold and the seventh threshold are selected, then 1-phase (GPU Dense) is preferred. That is, the computation paradigm is determined to be the single-stage GPU Dense mode (i.e., 1-phase (GPU Dense)), or the computation paradigm is determined to be the single-stage GPU Dense mode (i.e., 1-phase (GPU Dense)) and the two-stage unfold-sort-shrink mode (i.e., 2-phase ESC).
[0073] The seventh threshold is an empirical value, such as 64.
[0074] Taking a sixth threshold of 32 and a seventh threshold of 64 as an example, if If so, the computation paradigm is determined to be 1-phase (GPU Dense), or the computation paradigm is determined to be 1-phase (GPU Dense) and 2-phase ESC.
[0075] ESC (Expansion-Sorting-Compression) is a SpGEMM strategy that does not require a hash table. The first phase of 2-phase ESC expands all intermediate product terms of the first sparse matrix × the second sparse matrix into a list of triples and records their target row numbers. The second phase of 2-phase ESC sorts this list by row number and then "shrinks" it by merging terms in the same row and column to obtain the final matrix C. The entire process consists of two explicit phases: expansion and shrinking.
[0076] 303, if (Seventh threshold, eighth threshold) then the calculation paradigm is determined to be the two-phase expansion-sorting-shrinking mode (i.e., 2-phase ESC).
[0077] The eighth threshold is an empirical value, such as 200.
[0078] Taking a seventh threshold of 64 and an eighth threshold of 200 as an example, if Therefore, the computational paradigm is determined to be 2-phase ESC.
[0079] 304, if The eighth threshold, or, according to If the obtained coefficient of variation is greater than the ninth threshold, then it indicates that... If the value is very high or unevenly distributed, the computational paradigm can be determined to be a two-phase hybrid mode.
[0080] in, Eighth Threshold Explanation Very high, according to The obtained coefficient of variation being greater than the ninth threshold indicates that Uneven distribution.
[0081] The ninth threshold is an empirical value, such as 0.5.
[0082] Taking an eighth threshold of 200 and a ninth threshold of 0.5 as an example, if , or, according to The obtained coefficient of variation Therefore, the computational paradigm is determined to be 2-phase Hybrid.
[0083] The calculation paradigm has now been determined.
[0084] If the computation paradigm is determined to be a two-phase method (including the 2-phase method determined in step 202, the 2-phase Hybrid method determined in step 203 or 304, the 2-phase ESC method determined in step 302 or 303, etc.) through steps 201-203 or 304, then a specific accumulator will be selected based on fine-grained characteristics. This allows for the selection of the optimal accumulator type from various types such as Dense, Heap, Hashmap, ESC, and MergeSort within the selected paradigm, based on finer-grained heuristic rules (such as distribution variance and overlap). For example: 401, if The tenth threshold indicates high variance, which means that the filtered accumulator is only a Hybrid accumulator, i.e., only Hybrid accumulators are selected.
[0085] The tenth threshold is an empirical value, such as 0.5.
[0086] Taking the tenth threshold as 0.5 as an example, if If so, then a Hybrid accumulator should be selected.
[0087] 402, if The eleventh threshold indicates extreme sparsity, and it is determined that the filtered accumulator is based on a hash table, i.e., only HashMap is used.
[0088] The eleventh threshold is an empirical value, such as 5.
[0089] Taking the eleventh threshold as 5 as an example, if If so, then HashMap should be used.
[0090] In the numerical computation phase of SpGEMM, a HashMap is used as the row accumulator: column indices are used as keys and partial sums as values. When processing the L_i-th row of the first sparse matrix and multiplying it by the corresponding column (e.g., col_j) of the second sparse matrix, the result val is obtained, which generates (col_j, val) and inserts it into the hash table; if col_j already exists, val is incremented. After processing this row, all non-zero items are extracted from the hash table and written into the (L_i)-th row of the result matrix C, and then the hash table is cleared for reuse in the next row.
[0091] 403, if The twelfth threshold indicates high overlap. The filtered accumulator is determined to be either based on the expand-sort-shrink implementation on the GPU (i.e., using ESC (GPU)) or based on the merge sort implementation on the CPU (i.e., using MergeSort (CPU)). Ultimately, the one that performs better in actual tests is selected.
[0092] The twelfth threshold is an empirical value, such as 50.
[0093] Taking the twelfth threshold of 50 as an example, if If so, select ESC (GPU) or MergeSort (CPU).
[0094] ESC (GPU) leverages the massive parallelism of the GPU when implementing Expansion-Sorting-Compression: multiple thread blocks "expand" in parallel to generate a massive number of intermediate triples, then the high-performance radix sort on the GPU is called to sort the triples by (row, col), and finally a parallel reduction kernel is used to complete the "shrinking" and merging. The entire process fully utilizes the high bandwidth and sorting primitives of the GPU. Here, row represents the row and col represents the column.
[0095] Merge Sort (CPU) is a multi-way merge sorting algorithm used in SpGEMM on the CPU when merging multiple ordered lists of partial products into a single row of the resulting matrix. It maintains a min-heap (priority queue), popping the element with the smallest column index each time. If the column index is the same as the previously popped element, the value is incremented; otherwise, a new non-zero element is output. This method leverages CPU cache friendliness and branch prediction advantages, avoiding random memory accesses to hash tables.
[0096] 104. Based on the filtered accumulator and computation paradigm, SpGEMM (Sparse General Matrix-Matrix Multiplication) scheduling is performed.
[0097] SpGEMM scheduling can be performed by calling the corresponding API, based on the filtered accumulator and the determined computation paradigm.
[0098] In practical implementation, if all complete algorithms are available to be invoked by the method described in this embodiment, then in step 104, the algorithm corresponding to the filtered accumulator and the computation paradigm can be called. If not all complete algorithms are available to be invoked by the method described in this embodiment, then it is necessary to set up an interface for calling other libraries. For example, kokkos-kernel contains many algorithms, and it is possible to specify from the outside which algorithm to use to compute SpGEMM.
[0099] like: enum class SPGEMMAlgorithm { SPGEMM_DEFAULT, / / Automatic selection SPGEMM_KK, / / Classic KK Algorithm SPGEMM_KK_MEMORY, / / Memory-optimized version of KK SPGEMM_KK_DENSE, / / Dense rollback SPGEMM_MERGE_PATH, / / Merge path algorithm SPGEMM_DEBUG / / Debug mode }; For example, if the filtered accumulator is a dense accumulator, the SPGEMM_KK_DENSE option can be set to implement the corresponding call.
[0100] Furthermore, if the SpGEMM scheduling process is interrupted, the filtered accumulator and / or computation paradigm are updated, and SpGEMM scheduling is re-performed based on the filtered accumulator and computation paradigm, thus implementing a rollback mechanism. In other words, if execution is interrupted due to memory allocation failure or other reasons, it automatically rolls back to an alternative algorithm with lower memory requirements (e.g., if the memory usage of 2-phase ESC is less than that of 1-phase (Dense), and the current computation paradigm is 1-phase (Dense), it will automatically roll back from 1-phase (Dense) to 2-phase ESC) and re-execute. This addresses extreme cases of inaccurate predictions, thereby ensuring the high accuracy and robustness of the solution.
[0101] When automatically falling back to an alternative algorithm with lower memory requirements, generally, the dense accumulator requires the most memory, followed by ESC, and then Hashmap requires the least. You can directly fall back to Hashmap, or you can first fall back to ESC to see if it can be computed before considering falling back to Hashmap. Here, we don't consider the speed of computation, but only whether it can be computed correctly; this is the fallback mechanism provided in this embodiment to ensure no crashes.
[0102] In addition, for critical or unknown scenarios, lightweight benchmark tests are performed on 3-5 candidate algorithms during the first call, the optimal solution is selected based on time and memory weighted metrics, and the results are cached for subsequent reuse.
[0103] When selecting from 3-5 candidate algorithms, you can choose 3-5 solutions (which may not be accurate) based on your understanding of the problem and the simple judgment criteria mentioned above (for example, in the rapid elimination phase, remove infeasible solutions; if there are too many, continue to filter out some high-probability infeasible solutions; if there is no way to filter, just randomly pick 3-5). Then, determine the optimal solution by testing some data.
[0104] Since the scheduling scheme includes the number of computation stages (computation paradigm) and the accumulator to be used, the final scheduling scheme is determined after the computation paradigm and the accumulator are determined.
[0105] It should be noted that this embodiment and subsequent embodiments do not limit the relationship between the thresholds. That is, all the thresholds from the first threshold to the twelfth threshold can be different, or any two or more of the thresholds can be the same.
[0106] The SpGEMM dynamic scheduling method provided in this embodiment is an adaptive algorithm selection strategy for SpGEMM. Before calling SpGEMM, step 101 automatically analyzes the structure of the first and second sparse matrices (i.e., what the first and second sparse matrices look like) and the hardware platform information (i.e., the current hardware configuration). Then, steps 102 and 103 automatically select the most suitable accumulator and computation paradigm. In step 104, the scheduling scheme composed of the selected most suitable accumulator and computation paradigm is executed for SpGEMM scheduling. In this way, the user does not need to manually input the scheduling scheme; the SpGEMM dynamic scheduling method provided in this embodiment can automatically determine the fastest and most memory-efficient scheduling scheme.
[0107] The SpGEMM dynamic scheduling method provided in this embodiment enables refined decision-making driven by multi-dimensional features. Unlike existing schemes that rely solely on coarse-grained metrics such as matrix size, the SpGEMM dynamic scheduling method in this embodiment... , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix The standard deviation of the non-zero element distribution in the second sparse matrix rows By employing fine-grained structural features, a dynamic scheduling mechanism with multi-dimensional feature awareness is constructed, thereby significantly improving the accuracy and adaptability of the SpGEMM scheduling scheme.
[0108] The SpGEMM dynamic scheduling method provided in this embodiment can perform progressive SpGEMM scheduling scheme determination through accumulator filtering in step 102 and calculation paradigm determination in step 103. It has clear hierarchy, rigorous logic, significantly improves scheduling efficiency, and facilitates engineering implementation and later maintenance.
[0109] The SpGEMM dynamic scheduling method provided in this embodiment has a built-in robustness guarantee mechanism. Through a fallback mechanism, the SpGEMM dynamic scheduling method can automatically and seamlessly switch to a conservative but reliable alternative algorithm when the heuristic prediction fails due to abnormal input characteristics, fundamentally ensuring operational stability and result accuracy.
[0110] The SpGEMM dynamic scheduling method provided in this embodiment supports lightweight online performance feedback and parameter fine-tuning. Even when facing unknown or dynamically changing sparse patterns, it can still continuously approach the theoretical optimal performance, breaking through the adaptability bottleneck of most static scheduling schemes.
[0111] The SpGEMM dynamic scheduling method provided in this embodiment decouples the decision logic from the underlying hardware and adapts to different computing architectures through an abstract and unified interface, enabling seamless deployment on multiple platforms such as CPU, NVIDIA GPU, and AMD GPU.
[0112] This embodiment provides a SpGEMM dynamic scheduling method, which extracts key features and hardware platform information of the sparse matrix to be multiplied; filters accumulators based on key features; determines the computation paradigm based on hardware platform information; and performs SpGEMM scheduling based on the filtered accumulators and computation paradigm. This method automatically filters accumulators and determines the computation paradigm based on the key features and hardware platform information of the sparse matrix to be multiplied, improving the versatility and stability of the SpGEMM scheduling scheme and ensuring high accuracy and robustness of SpGEMM scheduling.
[0113] Based on the same inventive concept as the above method, this embodiment provides a SpGEMM dynamic scheduling device, see [link to relevant documentation]. Figure 2 The device includes: Extraction module 201 is used to extract key features and hardware platform information of the sparse matrices to be multiplied. The sparse matrices to be multiplied are a first sparse matrix and a second sparse matrix. The first sparse matrix has the following number of rows: The number of columns in the first sparse matrix is The second sparse matrix has rows. The number of columns in the first sparse matrix is Key features include , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix .
[0114] Filtering module 202 is used to filter the accumulator based on key features.
[0115] The determination module 203 is used to determine the computation paradigm based on the hardware platform information.
[0116] The scheduling module 204 is used for scheduling sparse general matrix multiplication (SpGEMM) based on the filtered accumulator and computation paradigm.
[0117] in, .
[0118] in, , .
[0119] Among them, the filter module 202 is used for if The remaining memory is then filtered out by discarding the Dense accumulator and the Heap accumulator. This is the memory ratio value. If... The first threshold, or, The first threshold filters out Dense accumulators. If The second threshold, and The second threshold filters out all accumulators except for the Dense accumulator.
[0120] The hardware platform information refers to the CPU platform.
[0121] Determine module 203, used for if Third threshold and The fourth threshold determines the computational paradigm as a single-stage dense mode. If... [Third threshold, fifth threshold], then the computational paradigm is determined to be the two-stage method. If The fifth threshold determines the computational paradigm as a two-stage hybrid mode.
[0122] The hardware platform information is for the GPU platform.
[0123] Determine module 203, used for if If the sixth threshold is met and the hardware supports it, then the computation paradigm is determined to be the single-stage GPU dense mode. If... If the sixth threshold and the seventh threshold are both set, then the computation paradigm is determined to be either a single-stage GPU dense mode or a two-stage unfold-sort-shrink mode. (Seventh threshold, eighth threshold) then the computational paradigm is determined to be a two-stage expansion-sorting-contraction model. If The eighth threshold, or, according to If the obtained coefficient of variation is greater than the ninth threshold, then the calculation paradigm is determined to be a two-stage hybrid mode.
[0124] The device also includes: Processing module, used for if The tenth threshold determines that the filtered accumulator is only a Hybrid accumulator. If The eleventh threshold determines that the filtered accumulator is implemented based on a hash table. If The twelfth threshold determines whether the filtered accumulator is based on a GPU-based expand-sort-shrink implementation or a CPU-based merge sort implementation.
[0125] If the SpGEMM scheduling process is interrupted, the filtered accumulator and / or computation paradigm are updated, and the scheduling module 204 is used to re-perform SpGEMM scheduling based on the filtered accumulator and computation paradigm.
[0126] The apparatus provided in this embodiment automatically filters the accumulator and determines the computation paradigm based on the key features of the sparse matrix to be multiplied and the hardware platform information, thereby improving the versatility of the SpGEMM scheduling scheme and the stability of its performance, and ensuring the high accuracy and robustness of SpGEMM scheduling. Based on the same inventive concept as the above method, this embodiment provides an electronic device, which is as follows: Figure 3 As shown, it includes: a memory 301, a processor 302, and a computer program.
[0127] The computer program is stored in memory 301 and configured to be executed by processor 302 to implement the above method.
[0128] Specifically, Extract key features and hardware platform information from the sparse matrices to be multiplied. The sparse matrices to be multiplied are the first sparse matrix and the second sparse matrix. The first sparse matrix has the following number of rows: The number of columns in the first sparse matrix is The second sparse matrix has rows. The number of columns in the first sparse matrix is Key features include , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix .
[0129] Filter accumulator based on key features.
[0130] The computational paradigm is determined based on the hardware platform information.
[0131] Based on the filtered accumulator and computation paradigm, sparse general matrix multiplication (SpGEMM) scheduling is performed.
[0132] in, .
[0133] in, , .
[0134] The filter accumulator based on key features includes: like The remaining memory is then filtered out by discarding the Dense accumulator and the Heap accumulator. This is the memory ratio value.
[0135] like The first threshold, or, The first threshold filters out Dense accumulators.
[0136] like The second threshold, and The second threshold filters out all accumulators except for the Dense accumulator.
[0137] The hardware platform information refers to the CPU platform.
[0138] The computational paradigm is determined based on hardware platform information, including: like Third threshold and The fourth threshold determines the computation paradigm as a single-stage dense mode.
[0139] like [Third threshold, fifth threshold], then the calculation paradigm is determined to be the two-stage method.
[0140] like The fifth threshold determines the computational paradigm as a two-stage hybrid mode.
[0141] The hardware platform information is for the GPU platform.
[0142] The computational paradigm is determined based on hardware platform information, including: like If the sixth threshold is met and the hardware supports it, then the computation paradigm is determined to be the single-stage GPU dense mode.
[0143] like If the sixth threshold and the seventh threshold are used, the computation paradigm is determined to be either a single-stage GPU dense mode or a two-stage unfold-sort-shrink mode.
[0144] like (Seventh threshold, eighth threshold) then the computation paradigm is determined to be the two-stage expansion-sorting-shrinking mode.
[0145] like The eighth threshold, or, according to If the obtained coefficient of variation is greater than the ninth threshold, then the calculation paradigm is determined to be a two-stage hybrid mode.
[0146] After determining the computational paradigm as the two-stage method, the method also includes: like The tenth threshold determines that the filtered accumulator is only a Hybrid accumulator.
[0147] like The eleventh threshold determines the filtered accumulator, which is implemented based on a hash table.
[0148] like The twelfth threshold determines whether the filtered accumulator is based on a GPU-based expand-sort-shrink implementation or a CPU-based merge sort implementation.
[0149] If the SpGEMM scheduling process is interrupted, the filtered accumulator and / or computation paradigm are updated, and SpGEMM scheduling is performed again based on the filtered accumulator and computation paradigm.
[0150] The electronic device provided in this embodiment has a computer program executed by a processor to automatically filter the accumulator and determine the calculation paradigm based on the key features of the sparse matrix to be multiplied and the hardware platform information. This improves the versatility of the SpGEMM scheduling scheme and the stability of SpGEMM scheduling performance, ensuring the high accuracy and strong robustness of SpGEMM scheduling.
[0151] Based on the same inventive concept as the above method, this embodiment provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the above method.
[0152] Specifically, Extract key features and hardware platform information from the sparse matrices to be multiplied. The sparse matrices to be multiplied are the first sparse matrix and the second sparse matrix. The first sparse matrix has the following number of rows: The number of columns in the first sparse matrix is The second sparse matrix has rows. The number of columns in the first sparse matrix is Key features include , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix .
[0153] Filter accumulator based on key features.
[0154] The computational paradigm is determined based on the hardware platform information.
[0155] Based on the filtered accumulator and computation paradigm, sparse general matrix multiplication (SpGEMM) scheduling is performed.
[0156] in, .
[0157] in, , .
[0158] The filter accumulator based on key features includes: like The remaining memory is then filtered out by discarding the Dense accumulator and the Heap accumulator. This is the memory ratio value.
[0159] like First threshold, or, The first threshold filters out Dense accumulators.
[0160] like The second threshold, and The second threshold filters out all accumulators except for the Dense accumulator.
[0161] The hardware platform information refers to the CPU platform.
[0162] The computational paradigm is determined based on hardware platform information, including: like Third threshold and The fourth threshold determines the computation paradigm as a single-stage dense mode.
[0163] like [Third threshold, fifth threshold], then the calculation paradigm is determined to be the two-stage method.
[0164] like The fifth threshold determines the computational paradigm as a two-stage hybrid mode.
[0165] The hardware platform information is for the GPU platform.
[0166] The computational paradigm is determined based on hardware platform information, including: like If the sixth threshold is met and the hardware supports it, then the computation paradigm is determined to be the single-stage GPU dense mode.
[0167] like If the sixth threshold and the seventh threshold are used, the computation paradigm is determined to be either a single-stage GPU dense mode or a two-stage unfold-sort-shrink mode.
[0168] like (Seventh threshold, eighth threshold) then the computation paradigm is determined to be the two-stage expansion-sorting-shrinking mode.
[0169] like The eighth threshold, or, according to If the obtained coefficient of variation is greater than the ninth threshold, then the calculation paradigm is determined to be a two-stage hybrid mode.
[0170] After determining the computational paradigm as the two-stage method, the method also includes: like The tenth threshold determines that the filtered accumulator is only a Hybrid accumulator.
[0171] like The eleventh threshold determines the filtered accumulator, which is implemented based on a hash table.
[0172] like The twelfth threshold determines whether the filtered accumulator is based on a GPU-based expand-sort-shrink implementation or a CPU-based merge sort implementation.
[0173] If the SpGEMM scheduling process is interrupted, the filtered accumulator and / or computation paradigm are updated, and SpGEMM scheduling is performed again based on the filtered accumulator and computation paradigm.
[0174] The computer-readable storage medium provided in this embodiment contains a computer program that is executed by a processor to automatically filter accumulators and determine the computation paradigm based on the key features of the sparse matrix to be multiplied and hardware platform information. This improves the versatility of the SpGEMM scheduling scheme and the stability of SpGEMM scheduling performance, ensuring the high accuracy and robustness of SpGEMM scheduling.
[0175] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.
[0176] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0177] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0178] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0179] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0180] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0181] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A SpGEMM dynamic scheduling method, characterized in that, The method includes: Extract key features and hardware platform information of the sparse matrices to be multiplied; wherein the sparse matrices to be multiplied are a first sparse matrix and a second sparse matrix; the first sparse matrix has the following number of rows. The number of columns in the first sparse matrix is The second sparse matrix has rows. The number of columns in the first sparse matrix is Key features include , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix ; Filter the accumulator based on the key features; Determine the computational paradigm based on hardware platform information; Based on the filtered accumulator and computation paradigm, sparse general matrix multiplication (SpGEMM) scheduling is performed.
2. The method according to claim 1, characterized in that, 。 3. The method according to claim 1, characterized in that, ， 。 4. The method according to claim 1, characterized in that, The filter accumulator based on the key features includes: like The remaining memory is then filtered out from the Dense accumulator and the Heap accumulator; among them, This is the memory ratio value; like First threshold, or, The first threshold filters out the Dense accumulator; like The second threshold, and The second threshold filters out all accumulators except for the Dense accumulator.
5. The method according to claim 1, characterized in that, The hardware platform information indicates a CPU platform. The step of determining the computation paradigm based on hardware platform information includes: like Third threshold and The fourth threshold determines the computation paradigm as a single-stage dense mode. like [Third threshold, fifth threshold], then the calculation paradigm is determined to be the two-stage method; like The fifth threshold determines the computational paradigm as a two-stage hybrid mode.
6. The method according to claim 1, characterized in that, The hardware platform information indicates a GPU platform. The step of determining the computation paradigm based on hardware platform information includes: like If the sixth threshold is met and the hardware supports it, then the computation paradigm is determined to be the single-stage GPU dense mode. like If the sixth threshold and the seventh threshold are used, the computation paradigm is determined to be the single-stage GPU dense mode, or the computation paradigm is determined to be the single-stage GPU dense mode and the two-stage unfold-sort-shrink mode. like (Seventh threshold, eighth threshold) then the computational paradigm is determined to be the two-stage expansion-sorting-contraction mode; like The eighth threshold, or, according to If the obtained coefficient of variation is greater than the ninth threshold, then the calculation paradigm is determined to be a two-stage hybrid mode.
7. The method according to claim 5 or 6, characterized in that, After determining the computational paradigm as the two-stage method, the method further includes: like The tenth threshold determines that the filtered accumulator is only a Hybrid accumulator; like The eleventh threshold determines the filtered accumulator, which is implemented based on a hash table. like The twelfth threshold determines whether the filtered accumulator is based on a GPU-based expand-sort-shrink implementation or a CPU-based merge sort implementation.
8. The method according to claim 1, characterized in that, If the SpGEMM scheduling process is interrupted, the filtered accumulator and / or computation paradigm are updated, and SpGEMM scheduling is performed again based on the filtered accumulator and computation paradigm.
9. A SpGEMM dynamic scheduling device, characterized in that, The device includes: An extraction module is used to extract key features and hardware platform information of the sparse matrices to be multiplied; wherein the sparse matrices to be multiplied are a first sparse matrix and a second sparse matrix; the first sparse matrix has the following number of rows: The number of columns in the first sparse matrix is The second sparse matrix has rows. The number of columns in the first sparse matrix is Key features include , , The total number of non-zero elements in the first sparse matrix The total number of non-zero elements in the second sparse matrix The number of non-zero predicted elements in the resulting matrix The average number of non-zero elements in the first sparse matrix The average number of non-zero elements in the rows of the second sparse matrix The standard deviation of the non-zero element distribution in the first row of the sparse matrix ; A filtering module is used to filter the accumulator based on the key features; The determination module is used to determine the computation paradigm based on hardware platform information; The scheduling module is used for scheduling sparse general matrix multiplication (SpGEMM) based on the filtered accumulator and computation paradigm.
10. The apparatus according to claim 9, characterized in that, 。 11. The apparatus according to claim 9, characterized in that, ， 。 12. The apparatus according to claim 9, characterized in that, The filtering module is used for if The remaining memory is then filtered out from the Dense accumulator and the Heap accumulator; among them, This is the memory ratio value; if First threshold, or, The first threshold filters out the Dense accumulator; if The second threshold, and The second threshold filters out all accumulators except for the Dense accumulator.
13. The apparatus according to claim 9, characterized in that, The hardware platform information indicates a CPU platform. The determining module is used if Third threshold and The fourth threshold determines the computation paradigm as a single-stage dense mode; if [Third threshold, fifth threshold], then the computational paradigm is determined to be the two-stage method; if The fifth threshold determines the computational paradigm as a two-stage hybrid mode.
14. The apparatus according to claim 9, characterized in that, The hardware platform information indicates a GPU platform. The determining module is used if If the sixth threshold is met and the hardware supports it, then the computation paradigm is determined to be the single-stage GPU dense mode; if If the sixth threshold and the seventh threshold are both met, then the computation paradigm is determined to be either a single-stage GPU dense mode or a two-stage unfold-sort-shrink mode. (Seventh threshold, eighth threshold) then the computational paradigm is determined to be a two-stage expansion-sorting-contraction model; if The eighth threshold, or, according to If the obtained coefficient of variation is greater than the ninth threshold, then the calculation paradigm is determined to be a two-stage hybrid mode.
15. The apparatus according to claim 13 or 14, characterized in that, The device further includes: Processing module, used for if The tenth threshold determines that the filtered accumulator is only a Hybrid accumulator; if The eleventh threshold determines that the filtered accumulator is implemented based on a hash table; if... The twelfth threshold determines whether the filtered accumulator is based on a GPU-based expand-sort-shrink implementation or a CPU-based merge sort implementation.
16. The apparatus according to claim 9, characterized in that, If the SpGEMM scheduling process is interrupted, the filtered accumulator and / or computation paradigm are updated, and the scheduling module is used to re-perform SpGEMM scheduling based on the filtered accumulator and computation paradigm.
17. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-8.
18. A computer-readable storage medium, characterized in that, It stores a computer program thereon; the computer program is executed by a processor to implement the method as described in any one of claims 1-8.