Calculation method and device for sparse matrix multiplication
By performing basic block division and dynamic programming solutions on matrices in large language model (LLM), the problem of sparse matrix multiplication adaptability is solved, and more efficient and flexible sparse matrix acceleration is achieved, improving computing efficiency and model accuracy.
Patent Information
- Application Number
- CN202510532188.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-08-08
AI Technical Summary
The prior art cannot effectively adapt to sparse matrix multiplication of different sizes and sparse levels in large language models (LLM), resulting in reduced model accuracy and inefficient hardware acceleration.
By performing basic block division of the first and second matrices, dynamic programming solves the optimal calculation strategy set and calculates based on the optimal strategy set to adapt to matrix multiplication of different sizes and sparseness levels.
A more efficient and flexible sparse matrix acceleration solution is achieved, improving the computing efficiency and flexibility of large language models (LLMs) and adapting to different matrix sizes and sparseness levels.
Smart Images
Figure CN120448688A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a calculation method and device for sparse matrix multiplication. Background Art
[0002] Matrix is a fundamental concept in mathematics. An M×N matrix is a rectangular array of elements arranged in M rows and N columns. Currently, matrix multiplication is a crucial operation in data computation within large language models (LLMs). As LLMs continue to grow in size and complexity, the scale of sparse matrix multiplication required also increases. Consequently, sparse matrix multiplication is necessary within LLMs.
[0003] Existing techniques combine structural pruning with hardware design to achieve a good balance between performance and accuracy. For example, this involves removing entire channels or neurons from convolutional kernels or fully connected layers. However, for large language models (LLMs), structural pruning can significantly reduce model accuracy. Existing sparse matrix acceleration libraries are unable to effectively accommodate the wider range of matrix sizes and varying sparsity levels found in LLMs. Consequently, the challenge of performing sparse matrix multiplication in LLMs remains urgent. Summary of the Invention
[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.
[0005] To this end, one object of the present invention is to propose a calculation method for sparse matrix multiplication, which performs dynamic programming solution on the first target data block and the second target data block based on basic block division to obtain an optimal calculation strategy set, so that the optimal calculation strategy set can automatically adapt to matrix multiplication of different sizes and sparsity levels, thereby providing a more efficient and flexible sparse matrix acceleration solution for LLM.
[0006] Another object of the present invention is to provide a computing device for sparse matrix multiplication.
[0007] To achieve the above objectives, an embodiment of the present invention provides a calculation method for sparse matrix multiplication, including:
[0008] Obtain the first matrix and the second matrix that need to be multiplied;
[0009] Determining a target dimension of a basic block, and dividing the first matrix and the second matrix based on the target dimension to obtain corresponding first target data blocks and second target data blocks;
[0010] Performing dynamic programming on the first target data block and the second target data block to obtain an optimal computing strategy set;
[0011] The first target data block and the second target data block are calculated based on the optimal calculation strategy set to obtain a target calculation result.
[0012] The calculation method of sparse matrix multiplication in the embodiment of the present invention may also have the following additional technical features:
[0013] Furthermore, determining the target dimension of the basic block includes:
[0014] Determine the minimum computing unit of the current device;
[0015] Based on the minimum computing unit, a target dimension of the basic block is determined.
[0016] Furthermore, performing dynamic programming on the first target data block and the second target data block to obtain an optimal computing strategy set includes:
[0017] determining a candidate set of target number of rows and columns for multiplication based on the basic block in the first target data block;
[0018] determining a target number of columns in the second target data block to be multiplied based on the basic block;
[0019] Determining an optimal number of columns for multiplying the first target data block based on the basic block based on the target number of rows, the candidate set of column numbers, and the target number of columns;
[0020] Determine the target number of rows, the optimal number of columns, and the target number of columns as a calculation strategy for current target numbers of rows of the first target data block and the second target data block;
[0021] Repeat the above steps to obtain all computing strategies corresponding to the first target data block and the second target data block, and determine all computing strategies as an optimal computing strategy set.
[0022] Furthermore, determining the optimal number of columns for multiplying the first target data block based on the basic block based on the target number of rows, the candidate set of column numbers, and the target number of columns includes:
[0023] Determining the number of multiplications corresponding to each candidate column number in the column number candidate set based on the target number of rows and the target number of columns;
[0024] The candidate column number corresponding to the least number of multiplications in the candidate column number set is determined as the optimal column number for multiplication of the first target data block based on the basic block.
[0025] Furthermore, the calculating the first target data block and the second target data block based on the optimal calculation strategy set to obtain a target calculation result includes:
[0026] Allocating each calculation strategy in the optimal calculation strategy set to a single thread to obtain a calculation result corresponding to each calculation strategy;
[0027] Based on the calculation results corresponding to each calculation strategy, a target calculation result is obtained.
[0028] Furthermore, the calculating the first target data block and the second target data block based on the optimal calculation strategy set to obtain a target calculation result includes:
[0029] Based on the optimal computing strategy set, the first target data block and the second target data block are calculated using a processor optimization method to obtain a target computing result.
[0030] To achieve the above object, another embodiment of the present invention provides a computing device for sparse matrix multiplication, the device comprising:
[0031] An acquisition module, configured to acquire a first matrix and a second matrix on which matrix multiplication is to be performed;
[0032] a partitioning module, configured to determine a target dimension of a basic block, and partition the first matrix and the second matrix based on the target dimension to obtain corresponding first target data blocks and second target data blocks;
[0033] a determination module, configured to perform dynamic programming on the first target data block and the second target data block to obtain an optimal calculation strategy set;
[0034] A calculation module is used to calculate the first target data block and the second target data block based on the optimal calculation strategy set to obtain a target calculation result.
[0035] The calculation method and device for sparse matrix multiplication proposed in the present invention perform dynamic programming solution on the first target data block and the second target data block based on basic block division to obtain the optimal calculation strategy set, so that the optimal calculation strategy set can automatically adapt to matrix multiplication of different sizes and sparsity levels, thereby providing a more efficient and flexible sparse matrix acceleration solution for LLM.
[0036] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:
[0038] Figure 1 is a flowchart of a calculation method for sparse matrix multiplication according to one embodiment of the present invention;
[0039] Figure 2 A schematic diagram of a calculation strategy according to an embodiment of the present invention;
[0040] Figure 3 A schematic diagram of a computing-storage overlap strategy according to one embodiment of the present invention;
[0041] Figure 4 A schematic diagram illustrating delay variations of different LLM models under sparse matrix workload according to an embodiment of the present invention;
[0042] Figure 5 Schematic diagram of the structure of a computing device for sparse matrix multiplication according to an embodiment of the present invention. DETAILED DESCRIPTION
[0043] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.
[0044] Among them, software-hardware co-optimization based on structured pruning has achieved a relatively good balance when faced with traditional deep learning model pruning tasks. However, pruning large language models (LLMs) presents different challenges than traditional model pruning. Specifically, it has been observed that enforcing strict structured sparsity in LLMs can lead to a significant decrease in model accuracy. For example, applying traditional pruning techniques to the LLaMA-7B model at a sparsity ratio of 2:8 results in a 1000-fold increase in perplexity. Furthermore, different layers of the LLM contribute significantly to the overall model performance, necessitating the optimal strategy to employ different sparsity levels at different layers. Furthermore, as LLMs scale, especially beyond the practical threshold of approximately 6 billion parameters, anomalous features that disproportionately affect perplexity begin to emerge, necessitating pruning techniques with greater sophistication. Simultaneously, these evolving requirements for pruning methods pose significant challenges for efficiently accelerating sparse matrix multiplication on existing hardware architectures.
[0045] Furthermore, NVIDIA's existing sparse matrix-matrix (SpMM) library, CuSPARSE, is primarily designed for scientific applications and features extremely high sparsity (over 99%), exceeding the sparsity levels required for deep learning matrix multiplication. However, LLMs operate on larger matrix sizes, requiring more sophisticated pruning techniques. Therefore, efficiently accelerating sparse matrix multiplication requires adapting to a wider range of matrix sizes and varying sparsity levels.
[0046] Based on the above description, the present invention proposes a calculation method for sparse matrix multiplication, which automatically adapts to matrix multiplication of different sizes and sparsity levels, thereby providing a more efficient and flexible sparse matrix acceleration solution for LLM.
[0047] First, a calculation method for sparse matrix multiplication proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0048] Figure 1 The figure is a flowchart of a calculation method of sparse matrix multiplication according to one embodiment of the present invention.
[0049] like Figure 1 As shown, the calculation method of the sparse matrix multiplication includes the following steps:
[0050] Step S1, obtaining a first matrix and a second matrix to be multiplied;
[0051] In one embodiment of the present invention, the first matrix and the second matrix for matrix multiplication need to satisfy a matrix multiplication condition, that is, the number of columns of the first matrix is the same as the number of rows of the second matrix.
[0052] Step S2, determining a target dimension of the basic block, and dividing the first matrix and the second matrix based on the target dimension to obtain corresponding first target data blocks and second target data blocks;
[0053] In one embodiment of the present invention, the method for determining the target dimension of a basic block may include determining a minimum computing unit of a current device and determining the target dimension of the basic block based on the minimum computing unit. In one embodiment of the present invention, the dimension of the minimum computing unit may be determined as the target dimension of the basic block.
[0054] For example, assuming that the current device is a GPU (Graphics Processing Unit), and the minimum computing unit of the GPU is 2×2, the target dimension of the basic block is 2×2.
[0055] In one embodiment of the present invention, the basic blocks can be adjusted as needed. If smaller basic blocks are used, more all-zero blocks can be extracted. If larger basic blocks are used, the overhead of fine-grained scheduling on the hardware platform can be reduced.
[0056] Furthermore, in one embodiment of the present invention, after determining the target dimension of the basic block through the above steps, the first matrix and the second matrix can be divided according to the target dimension to obtain the corresponding first target data block and second target data block.
[0057] Step S3, performing dynamic programming on the first target data block and the second target data block to obtain an optimal calculation strategy set;
[0058] In one embodiment of the present invention, after obtaining the first target data block and the second target data block through the above steps, dynamic programming can be performed on the first target data block and the second target data block to obtain an optimal calculation strategy set.
[0059] In one embodiment of the present invention, the method of performing dynamic programming on the first target data block and the second target data block to obtain an optimal computing strategy set may include the following steps:
[0060] Step S31, determining a candidate set of target row and column numbers for basic block-based multiplication in a first target data block;
[0061] Step S32, determining a target number of columns in the second target data block for multiplication based on the basic block;
[0062] Step S33, determining the optimal number of columns for multiplication of the first target data block based on the basic block based on the target number of rows, the candidate set of column numbers, and the target number of columns;
[0063] Step S34, determining the target number of rows, the optimal number of columns, and the target number of columns as a calculation strategy for the current target number of rows of the first target data block and the second target data block;
[0064] Step S35 , repeat the above steps to obtain all calculation strategies corresponding to the first target data block and the second target data block, and determine all calculation strategies as an optimal calculation strategy set.
[0065] Among them, in one embodiment of the present invention, the target number of rows for multiplication based on basic blocks in the above-mentioned first target data block and the target number of columns for multiplication based on basic blocks in the second target data block can be preset as needed. It should be noted that, in one embodiment of the present invention, the above-mentioned target number of rows and target number of columns are the number of rows and columns corresponding to the basic blocks. For example, in one embodiment of the present invention, assuming that the target number of rows is 3, this is the basic block of 3 rows in the first target data block. And, in one embodiment of the present invention, the above-mentioned target number of rows is less than the number of rows in the first target data block; the above-mentioned target number of columns is less than the number of columns in the second target data block.
[0066] Furthermore, in one embodiment of the present invention, the candidate set of column numbers for basic block-based multiplication in the first target data block may be pre-existing column numbers for which multiplication counting has been performed. Specifically, in one embodiment of the present invention, the pre-existing column numbers for which multiplication counting has been performed may be determined as the candidate set of column numbers. For example, the candidate set of column numbers includes {2, 3, 4, 5}.
[0067] Furthermore, in one embodiment of the present invention, the above-mentioned method for determining the optimal number of columns for multiplication of the first target data block based on the basic block based on the target number of rows, the candidate set of column numbers and the target number of columns may include: determining the number of multiplications corresponding to each candidate number of columns in the candidate set of column numbers based on the target number of rows and the target number of columns; and determining the candidate number of columns with the least corresponding number of multiplications in the candidate set of column numbers as the optimal number of columns for multiplication of the first target data block based on the basic block.
[0068] In one embodiment of the present invention, the average of the number of multiplications corresponding to the target number of rows, the target number of columns, and the candidate column numbers in the candidate column set can be used to determine the number of multiplications corresponding to each candidate column number. In another embodiment of the present invention, a multiplication calculation can be performed based on the target number of rows, the target number of columns, and the candidate column numbers in the candidate column set to determine the number of multiplications required.
[0069] In one embodiment of the present invention, after determining the optimal number of columns through the above steps, the target number of rows, the optimal number of columns, and the target number of columns can be determined as the calculation strategy for the current target number of rows of the first target data block and the second target data block. And the calculation strategy can be expressed as M abc , where a is the target number of rows, b is the optimal number of columns, c is the target number of columns, M abc Indicates the multiplication operation of a×b in the first target data block and b×c in the second target data block.
[0070] Furthermore, in one embodiment of the present invention, the calculation strategy for the current target number of rows can be determined through the above steps S31 to S34. If the optimal number of columns is less than the number of columns in the first target data block, the above steps need to be repeated until all the calculation strategies for the current target number of rows are obtained; the above steps S31 to S34 are repeated to obtain all the calculation strategies corresponding to the first target data block and the second target data block, and all the calculation strategies are determined as the optimal calculation strategy set. It should be noted that, in one embodiment of the present invention, when the above steps are repeated, the candidate set of target number of rows and number of columns for multiplication based on basic blocks in the first target data block and the target number of columns for multiplication based on basic blocks in the second target data block can be re-determined, that is, vertical column segmentation can be performed through dynamic programming technology, thereby providing flexibility in the calculation strategy.
[0071] For example, Figure 2 This is a schematic diagram of a calculation strategy proposed in an embodiment of the present invention. Figure 2 As shown in the figure, the left side is the first target data block and the right side is the second target data block. When the target number of rows in the first target data block is 4 and the target number of columns in the second target data block is 5, the calculation strategy for the current target number of rows obtained by the above steps is M 445 , at this time the optimal number of columns is 4, which is less than the total number of columns 9. Repeat the above steps to obtain the calculation strategy of the current objective function M 455 Based on this, we get the calculation strategy M of the current target number of rows 445 and M 455 Repeat the above steps to get the calculation strategy of the remaining target number of rows: 344 .
[0072] In one embodiment of the present invention, the above-mentioned method for determining the optimal set of computing strategies can adapt to different load sizes and sparsity levels, has a low tolerance for the sparsity of the matrix, and can reduce the number of multiplications even in the presence of all-zero blocks. That is, both high-sparsity and low-sparsity regions within the matrix can be optimized using the computing strategies in the optimal set of computing strategies.
[0073] Step S4: Calculate the first target data block and the second target data block based on the optimal calculation strategy set to obtain a target calculation result.
[0074] In one embodiment of the present invention, after obtaining the optimal calculation strategy set through the above steps, the first target data block and the second target data block can be calculated based on the optimal strategy set to obtain the target calculation result.
[0075] Specifically, in one embodiment of the present invention, the method for calculating the first target data block and the second target data block based on the optimal calculation strategy set to obtain the target calculation result may include the following steps:
[0076] Step S41: assign each computing strategy in the optimal computing strategy set to a single thread to obtain a computing result corresponding to each computing strategy;
[0077] Step S42: Obtain a target calculation result based on the calculation result corresponding to each calculation strategy.
[0078] In one embodiment of the present invention, each computing strategy in the optimal computing strategy set is assigned to a single thread, and the single thread is used to perform calculations to obtain the calculation results corresponding to each computing strategy.
[0079] Furthermore, in one embodiment of the present invention, the above-mentioned calculation is performed through a single thread to obtain the calculation results corresponding to each calculation strategy and the calculation results based on each calculation strategy. In the process of obtaining the target calculation results, the target calculation results can be obtained through a processor optimization method to achieve the overlap of calculation and data loading and improve calculation efficiency.
[0080] In one embodiment of the present invention, the matrix multiplication paradigm established above differs from the traditional matrix calculation method commonly used on existing processors. Therefore, optimization is required in the program implementation to improve the performance efficiency of the above method in the environment of existing processors (such as CPUs or GPU hardware platforms).
[0081] Furthermore, in one embodiment of the present invention, the processing optimization method may include a computation-storage overlap strategy, and the computation-storage overlap strategy may include a computation-load strategy and a computation-accumulation strategy.
[0082] Among them, in one embodiment of the present invention, the above-mentioned calculation loading strategy may include performing polynomial multiplication on the first target data block A and the second target data block B to obtain the intermediate calculation result m, and then accumulating the intermediate result m according to preset instructions to calculate the value of each element in the output matrix C corresponding to the output target calculation result.
[0083] For example, in one embodiment of the present invention, Figure 3 This is a schematic diagram of a computing-storage overlap strategy proposed in an embodiment of the present invention. Figure 3 As shown, the calculation of the intermediate result m8 is started without completely loading the first target data block A and the second target data block B, thereby allowing the multiplication of the intermediate calculation result to overlap with the data loading time.
[0084] Furthermore, in one embodiment of the present invention, the above-mentioned calculation accumulation strategy may include accumulating each intermediate result m of the calculation to the corresponding position in the output matrix C, and allocating the output matrix C in the shared memory, thereby effectively reducing the use of registers and alleviating the overhead of memory access.
[0085] Figure 4 Schematic diagram of the delay variation of different LLM models under sparse matrix workload provided by an embodiment of the present invention. Figure 4 As shown, CuBLAS demonstrates a relative advantage in executing large-scale matrix computation tasks, but the corresponding latency increases when models with 7 or 13 billion parameters are involved. PyTorch and Sputnik's performance degrades as the model size increases, and PyTorch's performance degrades significantly when processing a 65 billion parameter model and long sequence tasks. Furthermore, across 608 matrix workloads of three sizes, LLaMA-7B, LLaMA-13B, and LLaMA-65B, the present invention achieves improvements of 4.05x, 3.77x, 3.37x, and 2.39x over CuBLAS, Cusparse, PyTorch, and Sputnik, respectively.
[0086] According to the calculation method of sparse matrix multiplication proposed in an embodiment of the present invention, a first matrix and a second matrix that need to be matrix multiplied are obtained; the target dimension of the basic block is determined, and the first matrix and the second matrix are divided based on the target dimension to obtain the corresponding first target data block and the second target data block; the first target data block and the second target data block are dynamically programmed and solved to obtain an optimal calculation strategy set; the first target data block and the second target data block are calculated based on the optimal calculation strategy set to obtain a target calculation result. Therefore, the present invention performs dynamic programming and solves the first target data block and the second target data block divided based on the basic block to obtain an optimal calculation strategy set, so that the optimal calculation strategy set can automatically adapt to matrix multiplications of different sizes and sparsity levels, thereby providing a more efficient and flexible sparse matrix acceleration solution for LLM.
[0087] Next, a computing device for sparse matrix multiplication according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0088] Figure 5 Schematic diagram of the structure of a computing device for sparse matrix multiplication according to an embodiment of the present invention.
[0089] like Figure 5 As shown, the computing device 10 for sparse matrix multiplication includes: an acquisition module 501, a division module 502, a determination module 503 and a computing module 504, wherein:
[0090] An acquisition module 501 is used to acquire a first matrix and a second matrix that need to be multiplied;
[0091] A partitioning module 502 is configured to determine a target dimension of a basic block and partition the first matrix and the second matrix based on the target dimension to obtain corresponding first target data blocks and second target data blocks;
[0092] A determination module 503 is configured to perform dynamic programming on the first target data block and the second target data block to obtain an optimal calculation strategy set;
[0093] The calculation module 504 is configured to calculate the first target data block and the second target data block based on the optimal calculation strategy set to obtain a target calculation result.
[0094] Furthermore, the division module 502 is specifically configured to:
[0095] Determine the minimum computing unit of the current device;
[0096] Based on the minimum computing unit, the target dimension of the basic block is determined.
[0097] Furthermore, the determination module 503 is specifically configured to:
[0098] Determining a candidate set of target number of rows and columns for basic block-based multiplication in the first target data block;
[0099] determining a target number of columns in a second target data block to be multiplied based on the basic block;
[0100] Determining an optimal number of columns for multiplication of the first target data block based on the basic block based on the target number of rows, the candidate set of column numbers, and the target number of columns;
[0101] Determine the target number of rows, the optimal number of columns, and the target number of columns as a calculation strategy for the current target number of rows of the first target data block and the second target data block;
[0102] Repeat the above steps to obtain all calculation strategies corresponding to the first target data block and the second target data block, and determine all calculation strategies as the optimal calculation strategy set.
[0103] Furthermore, the determination module 503 is further configured to:
[0104] Based on the target number of rows and the target number of columns, determine the number of multiplications corresponding to each candidate column number in the column number candidate set;
[0105] The candidate column number corresponding to the least number of multiplications in the column number candidate set is determined as the optimal column number for multiplication of the first target data block based on the basic block.
[0106] Furthermore, the calculation module 504 is specifically configured to:
[0107] Assign each calculation strategy in the optimal calculation strategy set to a single thread, and obtain the calculation results corresponding to each calculation strategy;
[0108] Based on the calculation results corresponding to each calculation strategy, the target calculation result is obtained.
[0109] Furthermore, the calculation module 504 is further configured to:
[0110] Based on the optimal computing strategy set, the first target data block and the second target data block are calculated by a processor optimization method to obtain a target computing result.
[0111] According to the computing device for sparse matrix multiplication proposed in an embodiment of the present invention, a first matrix and a second matrix that need to be matrix multiplied are obtained; the target dimension of the basic block is determined, and the first matrix and the second matrix are divided based on the target dimension to obtain the corresponding first target data block and the second target data block; the first target data block and the second target data block are dynamically programmed and solved to obtain an optimal computing strategy set; the first target data block and the second target data block are calculated based on the optimal computing strategy set to obtain a target computing result. Therefore, the present invention performs dynamic programming and solves the first target data block and the second target data block divided based on the basic block to obtain an optimal computing strategy set, so that the optimal computing strategy set can automatically adapt to matrix multiplications of different sizes and sparsity levels, thereby providing a more efficient and flexible sparse matrix acceleration solution for LLM.
[0112] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.
[0113] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.
[0114] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A calculation method for sparse matrix multiplication, characterized in that, The method comprises: Obtain the first matrix and the second matrix that need to be multiplied; Determining a target dimension of a basic block, and dividing the first matrix and the second matrix based on the target dimension to obtain corresponding first target data blocks and second target data blocks; Performing dynamic programming on the first target data block and the second target data block to obtain an optimal computing strategy set; The first target data block and the second target data block are calculated based on the optimal calculation strategy set to obtain a target calculation result.
2. The method according to claim 1, characterized in that Determining the target dimension of the basic block includes: Determine the minimum computing unit of the current device; Based on the minimum computing unit, a target dimension of the basic block is determined.
3. The method according to claim 1, characterized in that The performing dynamic programming on the first target data block and the second target data block to obtain an optimal computing strategy set includes: determining a candidate set of target number of rows and columns for multiplication based on the basic block in the first target data block; determining a target number of columns in the second target data block to be multiplied based on the basic block; Determining an optimal number of columns for multiplying the first target data block based on the basic block based on the target number of rows, the candidate set of column numbers, and the target number of columns; Determine the target number of rows, the optimal number of columns, and the target number of columns as a calculation strategy for current target numbers of rows of the first target data block and the second target data block; Repeat the above steps to obtain all computing strategies corresponding to the first target data block and the second target data block, and determine all computing strategies as an optimal computing strategy set.
4. The method according to claim 3, characterized in that The determining, based on the target number of rows, the candidate set of column numbers, and the target number of columns, an optimal number of columns for multiplying the first target data block based on the basic block includes: Determining the number of multiplications corresponding to each candidate column number in the column number candidate set based on the target number of rows and the target number of columns; The candidate column number corresponding to the least number of multiplications in the candidate column number set is determined as the optimal column number for multiplication of the first target data block based on the basic block.
5. The method according to claim 3, characterized in that The calculating the first target data block and the second target data block based on the optimal calculation strategy set to obtain a target calculation result includes: Allocating each calculation strategy in the optimal calculation strategy set to a single thread to obtain a calculation result corresponding to each calculation strategy; Based on the calculation results corresponding to each calculation strategy, a target calculation result is obtained.
6. The method according to claim 1, characterized in that The calculating the first target data block and the second target data block based on the optimal calculation strategy set to obtain a target calculation result includes: Based on the optimal computing strategy set, the first target data block and the second target data block are calculated using a processor optimization method to obtain a target computing result.
7. A computing device for sparse matrix multiplication, characterized in that: The device comprises: An acquisition module, configured to acquire a first matrix and a second matrix on which matrix multiplication is to be performed; a partitioning module, configured to determine a target dimension of a basic block, and partition the first matrix and the second matrix based on the target dimension to obtain corresponding first target data blocks and second target data blocks; a determination module, configured to perform dynamic programming on the first target data block and the second target data block to obtain an optimal calculation strategy set; A calculation module is used to calculate the first target data block and the second target data block based on the optimal calculation strategy set to obtain a target calculation result.
8. The device according to claim 7, characterized in that The determining module is specifically configured to: determining a candidate set of target number of rows and columns for multiplication based on the basic block in the first target data block; determining a target number of columns in the second target data block to be multiplied based on the basic block; Determining an optimal number of columns for multiplying the first target data block based on the basic block based on the target number of rows, the candidate set of column numbers, and the target number of columns; Determine the target number of rows, the optimal number of columns, and the target number of columns as a calculation strategy for current target numbers of rows of the first target data block and the second target data block; Repeat the above steps to obtain all computing strategies corresponding to the first target data block and the second target data block, and determine all computing strategies as an optimal computing strategy set.
9. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.