Vector processor-oriented semi-precision sparse matrix multiplication multi-core parallel method and device
By building a variety of multi-core parallel modes and selecting the mode with the greatest theoretical computing efficiency for sparse matrix multiplication, the problem of low versatility in sparse matrix multiplication in the prior art is solved, and more efficient computing performance is achieved.
Patent Information
- Application Number
- CN202411049828.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-01
AI Technical Summary
Existing vector processors have low versatility in sparse matrix multiplication multiplication multicore computing and cannot be applied to multiple computing scenarios, resulting in insufficient computing performance.
According to the matrix dimension and the number of calculation cores, a variety of multi-core parallel modes are constructed, and the mode with the greatest theoretical calculation efficiency is selected for sparse matrix multiplication calculation, including parallel allocation of calculation cores on different dimensions of the weight matrix and dense input matrix to realize multi-core parallel.
The parallelism and calculation efficiency of sparse matrix multiplication calculation are improved, and the calculation performance of vector processors is improved.
Smart Images

Figure CN119045887A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of vector processors, and in particular to a method and device for multi-core parallel half-precision sparse matrix multiplication for vector processors. Background Art
[0002] FT-M7032 is a multi-core digital signal processor DSP (Digital Signal Processor) vector processor. Its overall architecture is as follows Figure 1a As shown in the figure, it consists of a 16-core central processing unit (CPU) and four general-purpose digital signal processor (GPDSP) clusters. Among them, four GPDSP clusters provide the main computing power support, and each GPDSP cluster consists of eight general-purpose DSP computing cores and a global shared memory (GSM) connected through CrossNet. GSM supports common access by all DSP computing cores in the same GPDSP cluster. The CPU and four GPDSP clusters share the global double data rate DDR (Double Data Rate) memory space, but each GPDSP cluster can only access the corresponding local DDR memory space. Direct memory access (DMA) is responsible for data transmission between the multi-level storage structure composed of DDR, GSM, scalar memory SM (scalar memory) / vector memory AM (vector memory). The theoretical bandwidth from GSM to DSP computing core in a single GPDSP cluster is much higher than the bandwidth from DDR to DSP computing core.
[0003] In the prior art, vector processors can realize multi-core computing of sparse matrix multiplication, but they cannot cover a variety of computing scenarios and have low versatility. Therefore, how to realize multi-core computing of sparse matrix multiplication, apply it to a variety of computing scenarios, and efficiently exert the computing performance of vector processors has become an urgent problem to be solved. Summary of the invention
[0004] Based on the above problems, the present application provides a method and device for multi-core parallel half-precision sparse matrix multiplication for vector processors, which is suitable for various computing scenarios and can efficiently exert the computing performance of vector processors while realizing multi-core computing of sparse matrix multiplication.
[0005] The present application discloses a multi-core parallel method for half-precision sparse matrix multiplication for vector processors. Sparse matrix multiplication is a process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix. The method includes:
[0006] According to the matrix dimension and the number of computing cores, multiple multi-core parallel modes are constructed; the matrix dimension is the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is [M][K], the matrix dimension of the dense input matrix is [K][N], and the matrix dimension of the dense output matrix is [M][N]; the multi-core parallel mode includes a first mode, a second mode and a third mode; the first mode is that each of the computing cores is parallel in the M dimension of the weight matrix; the second mode is that each of the computing cores is parallel in the N dimension of the dense input matrix; the third mode is that each of the computing cores is simultaneously parallel in the M dimension and the N dimension;
[0007] Based on the values of M and N, obtain the theoretical computational efficiency of performing sparse matrix multiplication calculation in each of the multi-core parallel modes;
[0008] Select the multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculations.
[0009] Optionally, the number of computing cores is 8, and multiple multi-core parallel modes are constructed according to the matrix dimension and the number of computing cores, including:
[0010] The computing cores are evenly distributed on the weight matrix to construct the first mode; wherein the number of rows of the weight matrix corresponding to each computing core is one-eighth M rounded up, and the number of columns of the weight matrix corresponding to each computing core is K.
[0011] Optionally, the obtaining of the theoretical computational efficiency of performing sparse matrix multiplication calculations in each of the multi-core parallel modes includes:
[0012] The theoretical calculation efficiency of the matrix dimension in the first mode is obtained as follows:
[0013]
[0014] Where, M_ratio is the theoretical calculation efficiency of the first mode, m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0015] Optionally, the number of computing cores is 8, and multiple multi-core parallel modes are constructed according to the matrix dimension and the number of computing cores, including:
[0016] The computing cores are evenly distributed on the dense input matrix to construct the second mode; wherein the number of rows of the dense input matrix corresponding to each computing core is K, and the number of columns of the dense input matrix corresponding to each computing core is one eighth of N rounded up.
[0017] Optionally, the obtaining of the theoretical computational efficiency of performing sparse matrix multiplication calculations in each of the multi-core parallel modes includes:
[0018] The theoretical calculation efficiency of the matrix dimension in the second mode is obtained as follows:
[0019]
[0020] Where N_ratio is the theoretical calculation efficiency of the second mode, m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0021] Optionally, the number of computing cores is 8, the third mode includes a first sub-mode and a second sub-mode, and the multiple multi-core parallel modes are constructed according to the matrix dimension and the number of computing cores, including:
[0022] The weight matrix is split in M dimensions to obtain two first sub-blocks with a row number of half M and a column number of K, and the dense input matrix is split in N dimensions to obtain four second sub-blocks with a row number of K and a column number of one quarter N, to construct the first sub-mode; one computing core corresponds to one first sub-block combination; the first sub-block combination is a combination of one first sub-block and one second sub-block;
[0023] By dividing the weight matrix into M dimensions, four third sub-blocks with a row number of one quarter M and a column number of K are obtained. Meanwhile, by dividing the dense input matrix into N dimensions, two fourth sub-blocks with a row number of K and a column number of one half N are obtained to construct the second sub-mode. One computing core corresponds to one second sub-block combination. The second sub-block combination is a combination of one third sub-block and one fourth sub-block.
[0024] Optionally, the obtaining of the theoretical computational efficiency of performing sparse matrix multiplication calculations in each of the multi-core parallel modes includes:
[0025] The theoretical calculation efficiency of the matrix dimension in the first sub-mode is obtained as follows:
[0026]
[0027] The theoretical calculation efficiency of the matrix dimension in the second sub-mode is obtained as follows:
[0028]
[0029] Wherein, MN_ratio1 is the theoretical calculation efficiency of the first sub-mode, MN_ratio2 is the theoretical calculation efficiency of the second sub-mode, and m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0030] Optionally, the selecting of a multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculations includes:
[0031] According to the selected multi-core parallel mode, obtaining the weight matrix block and the dense input matrix block corresponding to each computing core in parallel;
[0032] Sending the weight matrix block to a scalar storage space and sending the dense input matrix block to a vector storage space;
[0033] Micro-kernel calculation is performed on the matrix blocks in the scalar storage space and the vector storage space to obtain the dense output matrix.
[0034] Optionally, the microkernel computing includes:
[0035] Perform sparse matrix multiplication on the weight matrix block and the dense input matrix block to obtain a matrix of size m b ×n b The calculation result of m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0036] Based on the above-mentioned method for multi-core parallel half-precision sparse matrix multiplication for vector processors, the present application also discloses a multi-core parallel half-precision sparse matrix multiplication device for vector processors, wherein sparse matrix multiplication is a process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix, and the device includes: a pattern construction unit, an efficiency calculation unit and a calculation unit;
[0037] The mode construction unit is used to construct multiple multi-core parallel modes according to the matrix dimension and the number of computing cores; the matrix dimension is the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is [M][K], the matrix dimension of the dense input matrix is [K][N], and the matrix dimension of the dense output matrix is [M][N]; the multi-core parallel mode includes a first mode, a second mode and a third mode; the first mode is that each of the computing cores is parallel in the M dimension of the weight matrix; the second mode is that each of the computing cores is parallel in the N dimension of the dense input matrix; the third mode is that each of the computing cores is simultaneously parallel in the M dimension and the N dimension;
[0038] The efficiency calculation unit is used to obtain the theoretical calculation efficiency of the sparse matrix multiplication calculation performed in each multi-core parallel mode based on the values of M and N;
[0039] The computing unit is used to select a multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculations.
[0040] Optionally, the number of the computing cores is 8, and the pattern building unit includes:
[0041] The first allocation subunit is used to evenly distribute each of the computing cores on the weight matrix to construct the first mode; wherein the number of rows of the weight matrix corresponding to each of the computing cores is one-eighth M rounded up, and the number of columns of the weight matrix corresponding to each of the computing cores is K.
[0042] Optionally, the efficiency calculation unit includes:
[0043] The first efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the first mode as follows:
[0044]
[0045] Where, M_ratio is the theoretical calculation efficiency of the first mode, m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0046] Optionally, the number of the computing cores is 8, and the pattern building unit includes:
[0047] The second allocation subunit is used to evenly distribute each of the computing cores on the dense input matrix to construct the second mode; wherein the number of rows of the dense input matrix corresponding to each of the computing cores is K, and the number of columns of the dense input matrix corresponding to each of the computing cores is one-eighth N rounded up.
[0048] Optionally, the efficiency calculation unit includes:
[0049] The second efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the second mode as follows:
[0050]
[0051] Where N_ratio is the theoretical calculation efficiency of the second mode, m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0052] Optionally, the number of the computing cores is 8, the third mode includes a first sub-mode and a second sub-mode, and the mode construction unit includes:
[0053] The third allocation subunit is used for splitting the weight matrix in M dimension to obtain two first sub-blocks with the number of rows being half M and the number of columns being K, and splitting the dense input matrix in N dimension to obtain four second sub-blocks with the number of rows being K and the number of columns being one quarter N, so as to construct the first sub-mode; one computing core corresponds to one first sub-block combination; the first sub-block combination is a combination of one first sub-block and one second sub-block;
[0054] The fourth allocation subunit is used to divide the weight matrix into M dimensions to obtain 4 third sub-blocks with a row number of one quarter M and a column number of K, and to divide the dense input matrix into N dimensions to obtain 2 fourth sub-blocks with a row number of K and a column number of one half N, so as to construct the second sub-mode; one computing core corresponds to one second sub-block combination; the second sub-block combination is a combination of one third sub-block and one fourth sub-block.
[0055] Optionally, the efficiency calculation unit includes:
[0056] The third efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the first sub-mode as follows:
[0057]
[0058] The fourth efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the second sub-mode as follows:
[0059]
[0060] Wherein, MN_ratio1 is the theoretical calculation efficiency of the first sub-mode, MN_ratio2 is the theoretical calculation efficiency of the second sub-mode, and m b is the preset M-dimensional block size, nb is the preset N-dimensional block size.
[0061] Optionally, the computing unit includes:
[0062] A block subunit is used to obtain the weight matrix block and dense input matrix block corresponding to each computing core in parallel according to the selected multi-core parallel mode;
[0063] A storage subunit, used for sending the weight matrix block to a scalar storage space and sending the dense input matrix block to a vector storage space;
[0064] The output subunit is used to perform micro-kernel calculation on the matrix blocks in the scalar storage space and the vector storage space to obtain the dense output matrix.
[0065] Optionally, the output subunit is used to:
[0066] Perform sparse matrix multiplication on the weight matrix block and the dense input matrix block to obtain a matrix of size m b ×n b The calculation result of m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0067] The present application discloses a method and device for multi-core parallelization of half-precision sparse matrix multiplication for vector processors. According to the dimension of the matrix and the number of computing cores, three multi-core parallel modes are constructed, which are suitable for various computing scenarios and make full use of the multi-core architecture of the vector processor. At the same time, the computational redundancy under the dimension specifications of some matrices is reduced, the parallelism of sparse matrix multiplication calculation is improved, and the computing performance of the vector processor is brought into play. Among them, each multi-core parallel mode is to parallelize multiple computing cores on each dimension of the weight matrix and the dense input matrix, and implement sparse matrix multiplication in parallel with cores in different dimensions. Based on the dimension specifications of the two matrices, the theoretical computational efficiency of sparse matrix multiplication calculation in each multi-core parallel mode is obtained. Then the multi-core parallel mode with the highest theoretical computational efficiency is selected to perform sparse matrix multiplication calculation. In this way, the mode with the best performance can be automatically adapted to perform calculations, which has high versatility and improves computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0069] Figure 1a This is a schematic diagram of the structure of the FT-M7032 disclosed in the embodiment of this application;
[0070] Figure 1b This is a schematic diagram of the structure of the DSP computing core in the FT-M7032 disclosed in the embodiment of the present application;
[0071] Figure 2 A flowchart of a multi-core parallel method for half-precision sparse matrix multiplication for a vector processor disclosed in an embodiment of the present application;
[0072] Figure 3a A schematic diagram of multi-core parallel design and calculation in the first mode disclosed in the embodiment of the present application;
[0073] Figure 3b A schematic diagram of the intra-core data block design disclosed in the embodiment of the present application;
[0074] Figure 3c A schematic diagram of multi-core parallel design and calculation in the second mode disclosed in the embodiment of the present application;
[0075] Figure 3d A schematic diagram of multi-core parallel design and calculation in the first sub-mode disclosed in an embodiment of the present application;
[0076] Figure 4 A schematic diagram of the structure of a multi-core parallel device for half-precision sparse matrix multiplication for a vector processor disclosed in an embodiment of the present application. DETAILED DESCRIPTION
[0077] The structure and function of FT-M7032 are introduced: each DSP computing core is implemented based on VLIM architecture, and its internal architecture is as follows: Figure 1bAs shown, it is mainly composed of a scalar processing unit SPU (scalar processing unit), a vector processing unit VPU (vector processing unit), an instruction fetch unit IFU (instruction fetch unit) and a DMA component. Among them, the SPU is responsible for scalar calculation and process control, and mainly includes a scalar processing unit SPE (scalar processing elements) and SM. The VPU is responsible for vector calculation, and is composed of P vector processing units VPE (vector processing elements) and AM. The VPE operates collaboratively in a single instruction multiple data SIMD (single instruction multiple data) manner. Each VPE contains three floating point multiply accumulate FMAC (floating point multiply accumulator) components, and the FMAC is responsible for vector calculation of floating point data. The vector processor can transfer data from scalar registers to vector registers through broadcast / shared registers.
[0078] Floating point numbers are a type of binary floating point data used by computers. The DSP computing core of FT-M7032 supports double-precision, single-precision and half-precision floating point data calculations, which use 8 bytes, 4 bytes and 2 bytes for storage respectively. This application is developed and designed based on the vector processor for half-precision floating point numbers.
[0079] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0080] Embodiment 1: This application discloses a multi-core parallel method for half-precision sparse matrix multiplication for vector processors.
[0081] For details, please refer to Figure 2 The present embodiment discloses a multi-core parallel method for half-precision sparse matrix multiplication for a vector processor, comprising the following steps:
[0082] Step 101: Construct multiple multi-core parallel modes according to the matrix dimension and the number of computing cores.
[0083] In the method described in this embodiment, sparse matrix multiplication is a process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix, and the calculation formula is expressed as W×X=Y. Wherein, the weight matrix is W, and its matrix dimension is [M][K], that is, [number of rows] and [number of columns], which can be written as W:[M][K]. Similarly, the dense input matrix is X, and its matrix dimension is [K][N], written as X:[K][N], and the dense output matrix is Y, and its matrix dimension is [M][N], written as Y:[M][N].
[0084] In the method described in this embodiment, the multi-core parallel mode includes a first mode, a second mode and a third mode. Among them, the first mode is that each computing core is parallel in the M dimension of W, and this method is suitable for sparse matrix multiplication calculations with a larger M dimension and a smaller N dimension. The second mode is that each computing core is parallel in the N dimension of X, and this method is suitable for sparse matrix multiplication calculations with a larger N dimension and a smaller M dimension. The third mode is that each computing core is parallel in both M and N dimensions, and this method is suitable for sparse matrix multiplication calculations with larger M and N dimensions.
[0085] In the method described in this embodiment, as an optional method, the number of computing cores is 8. Therefore, the computing cores are evenly distributed on W, and the parallel design between the cores is expanded on the M dimension to construct the first mode. Among them, the number of rows m of W corresponding to each computing core is one eighth of M rounded up, that is, The number of columns of W corresponding to each computing core is K. Therefore, the part of W sub-blocks allocated to each computing core can be recorded as W m :[m][K].
[0086] In the method described in this embodiment, each computing core is evenly distributed on X, and the parallel design between the cores is expanded on N dimensions to construct the second mode. The number of rows of X corresponding to each computing core is K, and the number of columns of X corresponding to each computing core is n, which is one eighth of N rounded up, that is, Therefore, the part of X sub-blocks allocated to each computing core can be recorded as X n :[K][n].
[0087] In the method described in this embodiment, each computing core is evenly distributed on W and X, and the parallel design between cores is simultaneously expanded in M and N dimensions to construct the third mode. The M and N dimensions perform split-core calculations simultaneously, and the output Y will be divided into 8 sub-blocks, with each computing core executing one sub-block. Therefore, there are two paradigms for sub-blocks: 2×4 and 4×2, corresponding to the first sub-mode and the second sub-mode of the third mode, respectively.
[0088] In the first sub-mode, W is split in the M dimension to obtain two first sub-blocks W with half M rows and K columns.m :[M / 2][K]. At the same time, the N-dimensional segmentation of X is performed to obtain the second sub-block X with four rows of K and one-quarter of N columns. n :[K][N / 4]. One computing core corresponds to one first sub-block combination, and the first sub-block combination is a combination of a first sub-block and a second sub-block. Finally, each computing core calculates a size of Y mn :[M / 2][N / 4] sub-block results.
[0089] In the second sub-mode, W is split in the M dimension to obtain four third sub-blocks W with a row number of one-quarter M and a column number of K. m :[M / 4][K]. At the same time, X is split in N dimensions to obtain a fourth sub-block X with two rows of K and two columns of half N. n :[K][N / 2]. One computing core corresponds to one second sub-block combination, and the second sub-block combination is a combination of a third sub-block and a fourth sub-block. Finally, each computing core calculates a size of Y mn :[M / 4][N / 2] sub-block results.
[0090] Step 102: Based on the values of M and N, obtain the theoretical computational efficiency of sparse matrix multiplication calculations performed in each of the multi-core parallel modes.
[0091] In the method described in this embodiment, as an optional method, the micro-kernel algorithm specification under the sparse matrix multiplication multi-core parallel computing is preset to be an M-dimensional block size m b ×K-dimensional block size k b ×N-dimensional block size n b Among them, m b and n b is the preset value, k is a variable, and k is determined by the input parameters. Based on this, the theoretical calculation efficiency in the first mode is expressed as follows:
[0092]
[0093] Where, M_ratio is the theoretical calculation efficiency of the first mode, m b is the preset M-dimensional block size, n b The preset N-dimensional block size.
[0094] The theoretical calculation efficiency in the second mode is expressed as follows:
[0095]
[0096] Where N_ratio is the theoretical calculation efficiency of the second mode.
[0097] The theoretical calculation efficiency under the first sub-mode is expressed as follows:
[0098]
[0099] Where MN_ratio1 is the theoretical calculation efficiency of the first sub-mode.
[0100] The theoretical calculation efficiency under the second sub-mode is expressed as follows:
[0101]
[0102] Where MN_ratio2 is the theoretical calculation efficiency of the second sub-mode.
[0103] Step 103: Select the multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculation.
[0104] In the method described in this embodiment, after obtaining M and N, formulas (1)-(4) are executed respectively, the four theoretical computing efficiency calculation results are compared, and the multi-core parallel mode corresponding to the highest value is selected, and sparse matrix multiplication calculation is performed accordingly.
[0105] As an optional method, when performing sparse matrix multiplication calculations, according to the selected multi-core parallel mode, obtain the corresponding W allocated to each computing core in parallel m and X n . Then W m Send to SM space, change X n Send to AM space. Perform microkernel calculation on the matrix sub-blocks in SM space and AM space to obtain Y. m In SM space, divide X n The AM space is divided into blocks, and the sparse matrix multiplication calculation of the microkernel is performed at the block granularity to obtain a size of m b ×n b The calculation result is denoted as Y b :[m b ][n b ]. Then, the Y of each block b Sent to the DDR outside the core until all blocks are calculated, and finally Y m :[m][N].
[0106] The method described in this embodiment adopts a multi-core parallel design scheme, supports the construction of multiple multi-core parallel modes according to the dimension of the matrix and the number of computing cores, is applicable to a variety of computing scenarios, and makes full use of the multi-core architecture of the vector processor. The multi-core implementation of sparse matrix multiplication on the vector processor is realized, the computational redundancy under the dimension specifications of some matrices in a single mode is reduced, the parallelism of the calculation is improved, and the computing performance of the vector processor is helped to be brought into play. And based on the dimension specifications of the matrix, the theoretical computational efficiency of sparse matrix multiplication calculation in each multi-core parallel mode is obtained. Then the multi-core parallel mode with the highest theoretical computational efficiency is selected to perform sparse matrix multiplication calculation. In this way, the mode with the best performance can be automatically adapted to perform the calculation, which has high versatility and improves the computational efficiency.
[0107] Embodiment 2: This application discloses a method for implementing sparse matrix multiplication in the first mode:
[0108] Step 201: According to the first mode, M and K, obtain the matrix block W allocated to each computing core m :[m][K]、W m The block W in SM space m s :[m s ][K], number of blocks
[0109] In the method described in this embodiment, due to the limited space of SM and AM in the kernel, data often needs to be transmitted and calculated in blocks. The specific block design in the kernel is related to the microkernel block specification. b , K and SM available space size, get SM W m The M-dimensional value of the block Get SM on W m The block is denoted as W m s :[m s ][K], the number is According to m b 、n b , K, AM available space size and DMA double buffer mechanism, calculate the N-dimensional value of the submatrix of X on AM Get the block X on AM, denoted as X a :[K][N a ], the number is
[0110] In addition, each computing core is based on the core number and W m The matrix specification gets mapped to the corresponding data address in W.
[0111] Step 202: Obtain the block X of X in the AM space according to the first mode, N and K a :[K][N a ]、Number of blocks And divide the AM space into three intervals: AM_X1, AM_X2 and AM_Y.
[0112] Step 203: Initialize W m s In W m For index i on , let i=0.
[0113] Step 204: Use DMA to transfer the corresponding W m s Sent to SM space in a point-to-point manner.
[0114] In the method described in this embodiment, W of each computing core m The matrix will be W m s Form points n_W m s times to SM. X will be transmitted as X a The form of n_X a The next transmission is to AM.
[0115] Step 205: Use DMA to transfer the first X to X a Sent to AM_X1 via broadcast.
[0116] Step 206: Initialize X a At index j on X, let j=1.
[0117] Step 207: Use DMA to send the corresponding X a Sent to AM_X2 via broadcast.
[0118] Step 208: Set W with index i m s and X with index j-1 a Perform sparse matrix multiplication, store the result in AM_Y, and transfer it back to DDR via DMA in a point-to-point manner. m s and X a All data calculation and transmission are completed.
[0119] In the method described in this embodiment, the block size of the microkernel computing block for performing sparse matrix multiplication is m b ×K×n b , so we can get the size of m each time b ×n b The calculation result of .
[0120] Step 209: Let j=j+1, and determine whether j is less than n_X a If yes, then return to step 207. If no, then proceed to step 210.
[0121] Step 210: Set W with index i m s and X with index j-1 a Perform sparse matrix multiplication, store the result in AM_Y, and transfer it back to DDR via DMA in a point-to-point manner. m s and X a All data calculation and transmission are completed.
[0122] Step 211: Let i=i+1, and determine whether i is less than n_W m s If yes, then return to step 204. If no, then proceed to step 212.
[0123] Step 212: Fence synchronization, wait for all computing cores to complete the calculation, and transmit m×N Y m to DDR.
[0124] In the method described in this embodiment, in the first mode, each computing core performs multi-core parallel operation in M dimensions, and its design and calculation are as follows: Figure 3a As shown in Figure 1, the computing cores are evenly distributed on W, and the parallel design between the cores is expanded on the M dimension. Specifically, the number of computing cores is 8, and each computing core is allocated to obtain W m , each computing core completes its own W m Multiply the sparse matrix with X to get Y m Among them, in the data storage structure design, W m Send to SM space, X is sent to AM space, and the calculated Y m Also stored in AM.
[0125] In the method described in this embodiment, the idea of intra-core segmentation is as follows: Figure 3b As shown, W m s Where Dxx is the data of the compressed matrix, Ixx is the index, and W m s and X a With size m b ×K×n b The microkernel performs sparse matrix multiplication calculations at the granularity of m. b ×n b The calculation result is denoted as Y b :[m b ][n b ], the Y bStored in AM and transported to the DDR space outside the core. Then traverse loop W m s and X a Get m s ×N a The calculation result of Y is obtained. At this point, the sparse matrix multiplication calculation of the single data transferred by DMA is completed. Finally, the M and N dimensions in DDR are traversed until the calculation is completed, and each calculation core finally obtains Y m .
[0126] Embodiment 3: This application discloses a method for implementing sparse matrix multiplication in the second mode:
[0127] Step 301: Obtain the block W of W in the SM space according to the second mode, M and K s :[M s ][K], number of blocks
[0128] In the method described in this embodiment, according to m b , K and the size of the available space in SM, and obtain the M-dimensional value of the block of W on SM Get the W block on SM, denoted as W s :[M s ][K], the number is According to m b 、n b , K, AM available space size and DMA double buffer mechanism, calculate AM on X n The N-dimensional value of the submatrix Get AM on X n The block is denoted as X n a :[K][n a ], the number is
[0129] Step 302: According to the second mode, N and K, obtain the matrix block X allocated to each computing core n :[K][n], X n Block X in AM space n a :[K][N a ]、Number of blocks And divide the AM space into three intervals: AM_X1, AM_X2 and AM_Y.
[0130] In the method described in this embodiment, each computing core is assigned a number based on the core number and X. n a The matrix specification gets mapped to the corresponding data addresses in X.
[0131] Step 303: Initialize W s At index i on W, let i=0.
[0132] Step 304: Set the corresponding W s Send to SM space.
[0133] In the method described in this embodiment, the W matrix of each computing core will be W s Form points n_W s Transfer to SM. X n X n a The form of n_X n a The next transmission is to AM.
[0134] Step 305: X n First X n a Sent to AM_X1.
[0135] Step 306: Initialize X n a At index j on X, let j=1.
[0136] Step 307: Corresponding to X n a Sent to AM_X2.
[0137] Step 308: Set W with index i s and X with index j-1 n a Perform sparse matrix multiplication, store the result in AM_Y, and transfer it back to DDR. s and X n a All data calculation and transmission are completed.
[0138] Step 309: Set j=j+1, and determine whether j is less than n_X n a If yes, then return to step 307. If no, then proceed to step 310.
[0139] Step 310: Set W with index i s and X with index j-1 n a Perform sparse matrix multiplication, store the result in AM_Y, and transfer it back to DDR. s and X n a All data calculation and transmission are completed.
[0140] Step 311: Let i=i+1, and determine whether i is less than n_Ws If yes, then return to step 304. If no, then proceed to step 312.
[0141] Step 312: Fence synchronization, wait for all computing cores to complete the calculation, and transmit M×n Y n to DDR.
[0142] In the method described in this embodiment, in the second mode, each computing core performs multi-core parallel operation in N dimensions, and its design and calculation are as follows: Figure 3c As shown in the figure, each computing core is evenly distributed on X, and the parallel design between cores is expanded on N dimensions. Specifically, the number of computing cores is 8, and each computing core is allocated to obtain X n , each computing core completes its own W and X n The sparse matrix multiplication calculation of Y n Among them, in the data storage structure design, W is sent to the SM space, X n Sent to AM space, the calculated Y n Also stored in AM.
[0143] In the method described in this embodiment, W s and X n a With size m b ×K×n b The sparse matrix multiplication of the microkernel is calculated to obtain Y b . Then traverse loop W s and X n a Get M s ×n a The calculation result is finally traversed through the M and N dimensions until the calculation is completed. Each calculation core finally obtains Y n .
[0144] Embodiment 4: This application discloses a method for implementing sparse matrix multiplication in the first sub-mode:
[0145] Step 401: According to the first sub-mode, M and K, obtain the matrix block W allocated to each computing core of W m :[M / 2][K], W m The block W in SM space m s :[M s ][K], number of blocks
[0146] In the method described in this embodiment, according to m b , K and SM available space size, get SM W m The M-dimensional value of the block Get the W block on SM, denoted as W m s :[M s ][K], the number is According to m b 、n b , K, N / 4, AM available space size and DMA double buffer mechanism, calculate the X on AM n The N-dimensional value of the submatrix Get AM on X n The block is denoted as X n a :[K][n a ], the number is
[0147] In the method described in this embodiment, each computing core is based on the core sequence number and W m The matrix specification gets mapped to the corresponding data address in W.
[0148] Step 402: According to the second mode, N and K, obtain the matrix block X allocated to each computing core n :[K][N / 4], X n Block X in AM space n a :[K][N a ]、Number of blocks And divide the AM space into three intervals: AM_X1, AM_X2 and AM_Y.
[0149] Step 403: Initialize W m s In W m For index i on , let i=0.
[0150] Step 404: Set the corresponding W m s Send to SM space.
[0151] In the method described in this embodiment, W of each computing core m The matrix will be W m s Form points n_W m s Transfer to SM. X n X n a The form of n_X n a The next transmission is to AM.
[0152] Step 405: X n First X n aSent to AM_X1.
[0153] Step 406: Initialize X n a At index j on X, let j=1.
[0154] Step 407: Corresponding to X n a Sent to AM_X2.
[0155] Step 408: Set W with index i m s and X with index j-1 n a Perform sparse matrix multiplication, store the result in AM_Y, and transfer it back to DDR. m s and X n a All data calculation and transmission are completed.
[0156] Step 409: Set j=j+1, and determine whether j is less than n_X n a If yes, then return to step 407. If no, then proceed to step 410.
[0157] Step 410: Set W with index i m s and X with index j-1 n a Perform sparse matrix multiplication, store the result in AM_Y, and transfer it back to DDR. m s and X n a All data calculation and transmission are completed.
[0158] Step 411: Let i=i+1, and determine whether i is less than n_W m s If yes, then return to step 404. If no, then proceed to step 412.
[0159] Step 412: Fence synchronization, wait for all computing cores to complete the calculation, and transmit M / 2×N / 4 Y mn to DDR.
[0160] In the method described in this embodiment, in the third mode, each computing core is evenly distributed on the W matrix and the X matrix, and the inter-core parallel design is simultaneously carried out in the M and N dimensions. Specifically, the number of computing cores is 8. As an optional method, the M-dimensional segmentation of the W matrix is used to obtain two sub-blocks W. m :[M / 2][K], and at the same time, the X matrix is split in N dimensions to obtain 4 sub-blocks Xn :[K][N / 4], 8 cores are assigned a sub-block combination in turn, and each core calculates a size of Y mn :[M / 2][N / 4] is the sub-block result, which is the first sub-pattern. As another optional method, the M-dimensional segmentation of the W matrix is used to obtain 4 sub-blocks W m :[M / 4][K], and at the same time, the N-dimensional segmentation of the X matrix is performed to obtain two sub-blocks X n :[K][N / 2], 8 cores are assigned a sub-block combination in turn, and each core calculates a size of Y mn :[M / 4][N / 2] sub-block result, that is, the second sub-mode.
[0161] The first sub-mode is taken as an example to introduce in detail. Its design and calculation are as follows: Figure 3d As shown: In the first sub-mode, each computing core is allocated W m :[M / 2][K] and X n :[K][N / 4]. In the data storage structure design, W m Send to SM space, X n Sent to AM space, the calculated Y mn Also stored in AM.
[0162] In the method described in this embodiment, W m s and X n a With size m b ×K×n b The sparse matrix multiplication of the microkernel is calculated to obtain Y b . Then traverse loop W m s and X n a Get M s ×N a The calculation result is finally traversed through the M and N dimensions until the calculation is completed. Each calculation core finally obtains Y mn .
[0163] Based on the method for half-precision sparse matrix multiplication multi-core parallelization for vector processors disclosed in the above embodiment, this embodiment discloses a device for half-precision sparse matrix multiplication multi-core parallelization for vector processors. Sparse matrix multiplication is the process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix. Please refer to Figure 4 , the device comprises: a mode construction unit 501, an efficiency calculation unit 502 and a calculation unit 503;
[0164] The mode construction unit 501 is used to construct multiple multi-core parallel modes according to the matrix dimension and the number of computing cores; the matrix dimension is the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is [M][K], the matrix dimension of the dense input matrix is [K][N], and the matrix dimension of the dense output matrix is [M][N]; the multi-core parallel mode includes a first mode, a second mode and a third mode; the first mode is that each of the computing cores is parallel in the M dimension of the weight matrix; the second mode is that each of the computing cores is parallel in the N dimension of the dense input matrix; the third mode is that each of the computing cores is parallel in both the M dimension and the N dimension;
[0165] The efficiency calculation unit 502 is used to obtain the theoretical calculation efficiency of the sparse matrix multiplication calculation performed in each multi-core parallel mode based on the values of M and N;
[0166] The computing unit 503 is used to select a multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculations.
[0167] Optionally, the number of the computing cores is 8, and the pattern building unit 501 includes:
[0168] The first allocation subunit is used to evenly distribute each of the computing cores on the weight matrix to construct the first mode; wherein the number of rows of the weight matrix corresponding to each of the computing cores is one-eighth M rounded up, and the number of columns of the weight matrix corresponding to each of the computing cores is K.
[0169] Optionally, the efficiency calculation unit 502 includes:
[0170] The first efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the first mode as follows:
[0171]
[0172] Where, M_ratio is the theoretical calculation efficiency of the first mode, m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0173] Optionally, the number of the computing cores is 8, and the pattern building unit 501 includes:
[0174] The second allocation subunit is used to evenly distribute each of the computing cores on the dense input matrix to construct the second mode; wherein the number of rows of the dense input matrix corresponding to each of the computing cores is K, and the number of columns of the dense input matrix corresponding to each of the computing cores is one-eighth N rounded up.
[0175] Optionally, the efficiency calculation unit 502 includes:
[0176] The second efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the second mode as follows:
[0177]
[0178] Where N_ratio is the theoretical calculation efficiency of the second mode, m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0179] Optionally, the number of the computing cores is 8, the third mode includes a first sub-mode and a second sub-mode, and the mode construction unit 501 includes:
[0180] The third allocation subunit is used for splitting the weight matrix in M dimension to obtain two first sub-blocks with the number of rows being half M and the number of columns being K, and splitting the dense input matrix in N dimension to obtain four second sub-blocks with the number of rows being K and the number of columns being one quarter N, so as to construct the first sub-mode; one computing core corresponds to one first sub-block combination; the first sub-block combination is a combination of one first sub-block and one second sub-block;
[0181] The fourth allocation subunit is used to divide the weight matrix into M dimensions to obtain 4 third sub-blocks with a row number of one quarter M and a column number of K, and to divide the dense input matrix into N dimensions to obtain 2 fourth sub-blocks with a row number of K and a column number of one half N, so as to construct the second sub-mode; one computing core corresponds to one second sub-block combination; the second sub-block combination is a combination of one third sub-block and one fourth sub-block.
[0182] Optionally, the efficiency calculation unit 502 includes:
[0183] The third efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the first sub-mode as follows:
[0184]
[0185] The fourth efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the second sub-mode as follows:
[0186]
[0187] Wherein, MN_ratio1 is the theoretical calculation efficiency of the first sub-mode, MN_ratio2 is the theoretical calculation efficiency of the second sub-mode, and mb is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0188] Optionally, the calculation unit 503 includes:
[0189] A block subunit is used to obtain the weight matrix block and dense input matrix block corresponding to each computing core in parallel according to the selected multi-core parallel mode;
[0190] A storage subunit, used for sending the weight matrix block to a scalar storage space and sending the dense input matrix block to a vector storage space;
[0191] The output subunit is used to perform micro-kernel calculation on the matrix blocks in the scalar storage space and the vector storage space to obtain the dense output matrix.
[0192] Optionally, the output subunit is used to:
[0193] Perform sparse matrix multiplication on the weight matrix block and the dense input matrix block to obtain a matrix of size m b ×n b The calculation result of m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
[0194] The embodiments in this specification are described in a progressive manner. As for the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.
[0195] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0196] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0197] The features described in the embodiments of this specification can be replaced or combined with each other, so that professional and technical personnel in this field can implement or use the present application.
[0198] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A multi-core parallel method for half-precision sparse matrix multiplication for vector processors, characterized in that: Sparse matrix multiplication is a process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix. The method includes: According to the matrix dimension and the number of computing cores, multiple multi-core parallel modes are constructed; the matrix dimension is the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is [M][K], the matrix dimension of the dense input matrix is [K][N], and the matrix dimension of the dense output matrix is [M][N]; the multi-core parallel mode includes a first mode, a second mode and a third mode; the first mode is that each of the computing cores is parallel in the M dimension of the weight matrix; the second mode is that each of the computing cores is parallel in the N dimension of the dense input matrix; the third mode is that each of the computing cores is simultaneously parallel in the M dimension and the N dimension; Based on the values of M and N, obtain the theoretical computational efficiency of performing sparse matrix multiplication calculation in each of the multi-core parallel modes; Select the multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculations.
2. The method according to claim 1, characterized in that The number of computing cores is 8. According to the matrix dimension and the number of computing cores, multiple multi-core parallel modes are constructed, including: The computing cores are evenly distributed on the weight matrix to construct the first mode; wherein the number of rows of the weight matrix corresponding to each computing core is one-eighth M rounded up, and the number of columns of the weight matrix corresponding to each computing core is K.
3. The method according to claim 2, characterized in that The obtaining of the theoretical computational efficiency of performing sparse matrix multiplication calculations in each of the multi-core parallel modes includes: The theoretical calculation efficiency of the matrix dimension in the first mode is obtained as follows: Where, M_ratio is the theoretical calculation efficiency of the first mode, m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
4. The method according to claim 1, characterized in that: The number of computing cores is 8. According to the matrix dimension and the number of computing cores, multiple multi-core parallel modes are constructed, including: The computing cores are evenly distributed on the dense input matrix to construct the second mode; wherein the number of rows of the dense input matrix corresponding to each computing core is K, and the number of columns of the dense input matrix corresponding to each computing core is one eighth of N rounded up.
5. The method according to claim 4, characterized in that The obtaining of the theoretical computational efficiency of performing sparse matrix multiplication calculations in each of the multi-core parallel modes includes: The theoretical calculation efficiency of the matrix dimension in the second mode is obtained as follows: Where N_ratio is the theoretical calculation efficiency of the second mode, m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
6. The method according to claim 1, characterized in that The number of computing cores is 8, the third mode includes a first sub-mode and a second sub-mode, and multiple multi-core parallel modes are constructed according to the matrix dimension and the number of computing cores, including: The weight matrix is split in M dimensions to obtain two first sub-blocks with a row number of half M and a column number of K, and the dense input matrix is split in N dimensions to obtain four second sub-blocks with a row number of K and a column number of one quarter N, to construct the first sub-mode; one computing core corresponds to one first sub-block combination; the first sub-block combination is a combination of one first sub-block and one second sub-block; By dividing the weight matrix into M dimensions, four third sub-blocks with a row number of one quarter M and a column number of K are obtained. Meanwhile, by dividing the dense input matrix into N dimensions, two fourth sub-blocks with a row number of K and a column number of one half N are obtained to construct the second sub-mode. One computing core corresponds to one second sub-block combination. The second sub-block combination is a combination of one third sub-block and one fourth sub-block.
7. The method according to claim 6, characterized in that The obtaining of the theoretical computational efficiency of performing sparse matrix multiplication calculations in each of the multi-core parallel modes includes: The theoretical calculation efficiency of the matrix dimension in the first sub-mode is obtained as follows: The theoretical calculation efficiency of the matrix dimension in the second sub-mode is obtained as follows: Wherein, MN_ratio1 is the theoretical calculation efficiency of the first sub-mode, MN_ratio2 is the theoretical calculation efficiency of the second sub-mode, and m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
8. The method according to claim 1, characterized in that: The multi-core parallel mode with the highest theoretical computing efficiency is selected to perform sparse matrix multiplication calculation, including: According to the selected multi-core parallel mode, obtaining the weight matrix block and the dense input matrix block corresponding to each computing core in parallel; Sending the weight matrix block to a scalar storage space and sending the dense input matrix block to a vector storage space; Micro-kernel calculation is performed on the matrix blocks in the scalar storage space and the vector storage space to obtain the dense output matrix.
9. The method according to claim 8, characterized in that The microkernel computing includes: Perform sparse matrix multiplication on the weight matrix block and the dense input matrix block to obtain a matrix of size m b ×n b The calculation result of m b is the preset M-dimensional block size, n b is the preset N-dimensional block size.
10. A multi-core parallel device for half-precision sparse matrix multiplication for vector processors, characterized in that: The sparse matrix multiplication is a process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix. The device includes: a pattern construction unit, an efficiency calculation unit and a calculation unit; The mode construction unit is used to construct multiple multi-core parallel modes according to the matrix dimension and the number of computing cores; the matrix dimension is the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is [M][K], the matrix dimension of the dense input matrix is [K][N], and the matrix dimension of the dense output matrix is [M][N]; the multi-core parallel mode includes a first mode, a second mode and a third mode; the first mode is that each of the computing cores is parallel in the M dimension of the weight matrix; the second mode is that each of the computing cores is parallel in the N dimension of the dense input matrix; the third mode is that each of the computing cores is simultaneously parallel in the M dimension and the N dimension; The efficiency calculation unit is used to obtain the theoretical calculation efficiency of the sparse matrix multiplication calculation performed in each multi-core parallel mode based on the values of M and N; The computing unit is used to select a multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculations.
Citation Information
Patent Citations
Sparse matrix vector multiplication vectorization implementation method
CN111859277A
Matrix inversion method and device, storage medium and electronic equipment
CN114741651A
Matrix multiplication computing unit, acceleration unit, computing system and related method
CN115310037A
Compression storage of sparse matrix and parallel processing method of vector multiplication of sparse matrix
CN116257209A
Method and device for implementing vector multiplication of low-bit-width data matrix of vector processor
CN117493748A
Cited By
Input data sharing and cache optimization method and system in matrix multiplication calculation and application
CN121072733A
