A method and device for semi-precision sparse matrix multiplication multi-core parallel of a vector processor

Abstract

The application discloses a kind of semi-precision sparse matrix multiplication multicore parallel method and device for vector processor.There are three kinds of multicore parallel modes according to the dimension of matrix and the number of computing core, suitable for a variety of computing scenarios, make full use of the multicore architecture of vector processor.At the same time, it reduces the calculation redundancy under part of matrix dimension specification, improves the parallelism of sparse matrix multiplication calculation, helps to play the computing performance of vector processor.Each multicore parallel mode is to parallel multiple computing cores in the dimension of weight matrix and dense input matrix, and to realize sparse matrix multiplication in different dimensions.The theoretical calculation efficiency of sparse matrix multiplication calculation in each multicore parallel mode is obtained based on the dimension specification of two matrices.Then the multicore parallel mode with the maximum theoretical calculation efficiency is selected for sparse matrix multiplication calculation.This can automatically adapt the optimal mode to perform calculation, with high versatility and improved calculation efficiency.

Description

Technical Field

[0001] This application relates to the field of vector processor technology, and in particular to a method and apparatus for multi-core parallel half-precision sparse matrix multiplication for vector processors. Background Technology

[0002] The FT-M7032 is a multi-core digital signal processor (DSP) vector processor, and its overall architecture is as follows: Figure 1a As shown, it consists of a 16-core CPU (Central Processing Unit) and four general-purpose digital signal processor (GPDSP) clusters. The four GPDSP clusters provide the main computing power, each consisting of eight general-purpose DSP cores and a global shared memory (GSM) connected via CrossNet. The GSM allows all DSP cores within the same GPDSP cluster to access it. The CPU and the four GPDSP clusters share a global double data rate (DDR) memory space, but each GPDSP cluster can only access its corresponding local DDR memory space. Direct memory access (DMA) handles data transfer between the multi-level memory structure consisting of DDR, GSM, scalar memory (SM) / vector memory (AM). The theoretical bandwidth from GSM to DSP cores within a single GPDSP cluster is significantly higher than the bandwidth from DDR to DSP cores.

[0003] In existing technologies, vector processors can perform multi-core computations of sparse matrix multiplication, but they cannot cover a wide range of computational scenarios and have low versatility. Therefore, how to achieve multi-core computations of sparse matrix multiplication while being applicable to various computational scenarios and efficiently leveraging the computational performance of vector processors has become an urgent problem to be solved. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method and apparatus for multi-core parallel half-precision sparse matrix multiplication oriented towards vector processors. This method is applicable to various computing scenarios and efficiently leverages the computing performance of vector processors while achieving multi-core computation of sparse matrix multiplication.

[0005] This application discloses a multi-core parallel method for half-precision sparse matrix multiplication oriented towards vector processors. The sparse matrix multiplication is a process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix. The method includes:

[0006] Multiple multi-core parallel modes are constructed based on the matrix dimension and the number of computational cores. The matrix dimension refers to the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is ​​[M][K], the matrix dimension of the dense input matrix is ​​[K][N], and the matrix dimension of the dense output matrix is ​​[M][N]. The multi-core parallel modes include a first mode, a second mode, and a third mode. In the first mode, each computational core operates in parallel on the M dimension of the weight matrix. In the second mode, each computational core operates in parallel on the N dimension of the dense input matrix. In the third mode, each computational core operates in parallel on both the M and N dimensions simultaneously.

[0007] Based on the values ​​of M and N, the theoretical computational efficiency of sparse matrix multiplication calculation in each of the multi-core parallel modes is obtained.

[0008] The sparse matrix multiplication calculation is performed using the multi-core parallel mode, which has the highest theoretical computational efficiency.

[0009] Optionally, the number of computational cores is 8, and multiple multi-core parallel modes are constructed based on the matrix dimension and the number of computational cores, including:

[0010] The first mode is constructed by evenly distributing the computational cores on the weight matrix; wherein the number of rows of the weight matrix corresponding to each computational core is the rounded-up of one-eighth M, and the number of columns of the weight matrix corresponding to each computational core is K.

[0011] Optionally, obtaining the theoretical computational efficiency of sparse matrix multiplication in each of the multi-core parallel modes includes:

[0012] The theoretical computational efficiency of the matrix dimension in the first mode is obtained as follows:

[0013]

[0014] In the formula, M_ratio represents the theoretical computational efficiency of the first mode, and m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0015] Optionally, the number of computational cores is 8, and multiple multi-core parallel modes are constructed based on the matrix dimension and the number of computational cores, including:

[0016] The second mode is constructed by evenly distributing the computational cores on the dense input matrix; wherein the number of rows in the dense input matrix corresponding to each computational core is K, and the number of columns in the dense input matrix corresponding to each computational core is the rounded-up 1 / 8N.

[0017] Optionally, obtaining the theoretical computational efficiency of sparse matrix multiplication in each of the multi-core parallel modes includes:

[0018] The theoretical computational efficiency of the matrix dimension in the second mode is obtained as follows:

[0019]

[0020] In the formula, N_ratio represents the theoretical computational efficiency of the second mode, m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0021] Optionally, the number of computational cores is 8, the third mode includes a first sub-mode and a second sub-mode, and multiple multi-core parallel modes are constructed based on the matrix dimension and the number of computational cores, including:

[0022] The weight matrix is ​​divided into M dimensions to obtain two first sub-blocks with 1 / 2 M rows and K columns. Simultaneously, the dense input matrix is ​​divided into N dimensions to obtain four second sub-blocks with K rows and 1 / 4 N columns, thus constructing the first sub-pattern. One computing kernel corresponds to one combination of the first sub-blocks. The first sub-block combination is a combination of one first sub-block and one second sub-block.

[0023] The weight matrix is ​​divided into M dimensions to obtain four third sub-blocks with 1 / 4 M rows and K columns. At the same time, the dense input matrix is ​​divided into N dimensions to obtain two fourth sub-blocks with K rows and 1 / 2 N columns, thus constructing the second sub-mode. One computing kernel corresponds to one combination of the second sub-blocks. The combination of the second sub-blocks is a combination of one of the third sub-blocks and one of the fourth sub-blocks.

[0024] Optionally, obtaining the theoretical computational efficiency of sparse matrix multiplication in each of the multi-core parallel modes includes:

[0025] The theoretical computational efficiency of the matrix dimension in the first sub-mode is obtained as follows:

[0026]

[0027] The theoretical computational efficiency of the matrix dimension in the second sub-mode is obtained as follows:

[0028]

[0029] In the formula, MN_ratio1 is the theoretical computational efficiency of the first sub-mode, MN_ratio2 is the theoretical computational efficiency of the second sub-mode, and m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0030] Optionally, the step of selecting the multi-core parallel mode with the highest theoretical computational efficiency for sparse matrix multiplication includes:

[0031] Based on the selected multi-core parallel mode, obtain the weight matrix block and dense input matrix block corresponding to each computing core in parallel mode;

[0032] The weight matrix block is sent to the scalar storage space, and the dense input matrix block is sent to the vector storage space;

[0033] The dense output matrix is ​​obtained by performing microkernel calculations on the matrix blocks in the scalar storage space and the vector storage space.

[0034] Optionally, the microkernel computing includes:

[0035] Perform sparse matrix multiplication on the weight matrix block and the dense input matrix block to obtain a matrix of size m. b ×n b The calculation result; the m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0036] Based on the above-mentioned method for multi-core parallel half-precision sparse matrix multiplication for vector processors, this application also discloses a device for multi-core parallel half-precision sparse matrix multiplication for vector processors. Sparse matrix multiplication is a process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix. The device includes: a pattern construction unit, an efficiency calculation unit, and a calculation unit.

[0037] The mode construction unit is used to construct multiple multi-core parallel modes based on the matrix dimension and the number of computational cores; the matrix dimension is the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is ​​[M][K], the matrix dimension of the dense input matrix is ​​[K][N], and the matrix dimension of the dense output matrix is ​​[M][N]; the multi-core parallel modes include a first mode, a second mode, and a third mode; the first mode is that each of the computational cores operates in parallel on the M dimension of the weight matrix; the second mode is that each of the computational cores operates in parallel on the N dimension of the dense input matrix; the third mode is that each of the computational cores operates in parallel simultaneously on both the M and N dimensions;

[0038] The efficiency calculation unit is used to obtain the theoretical computational efficiency of sparse matrix multiplication calculation in each of the multi-core parallel modes based on the values ​​of M and N.

[0039] The computing unit is used to select the multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculations.

[0040] Optionally, the computing cores are 8, and the pattern construction unit includes:

[0041] The first allocation subunit is used to evenly distribute the computational cores on the weight matrix to construct the first mode; wherein the number of rows of the weight matrix corresponding to each computational core is one-eighth of M rounded up, and the number of columns of the weight matrix corresponding to each computational core is K.

[0042] Optionally, the efficiency calculation unit includes:

[0043] The first efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the first mode as follows:

[0044]

[0045] In the formula, M_ratio represents the theoretical computational efficiency of the first mode, and m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0046] Optionally, the computing cores are 8, and the pattern construction unit includes:

[0047] The second allocation subunit is used to evenly distribute each of the computing cores on the dense input matrix to construct the second mode; wherein, the number of rows of the dense input matrix corresponding to each computing core is K, and the number of columns of the dense input matrix corresponding to each computing core is the rounded-up of one-eighth N.

[0048] Optionally, the efficiency calculation unit includes:

[0049] The second efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the second mode as follows:

[0050]

[0051] In the formula, N_ratio represents the theoretical computational efficiency of the second mode, m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0052] Optionally, the number of computing cores is 8, the third mode includes a first sub-mode and a second sub-mode, and the mode construction unit includes:

[0053] The third allocation subunit is used to divide the weight matrix in M ​​dimensions to obtain two first sub-blocks with half M rows and K columns, and simultaneously divide the dense input matrix in N dimensions to obtain four second sub-blocks with K rows and one-quarter N columns, thus constructing the first sub-pattern; one computing kernel corresponds to one combination of the first sub-blocks; the first sub-block combination is a combination of one first sub-block and one second sub-block;

[0054] The fourth allocation subunit is used to divide the weight matrix in M ​​dimensions to obtain four third sub-blocks with one-quarter M rows and K columns, and simultaneously divide the dense input matrix in N dimensions to obtain two fourth sub-blocks with K rows and one-half N columns, thus constructing the second sub-mode; one computing kernel corresponds to one combination of second sub-blocks; the combination of second sub-blocks is a combination of one third sub-block and one fourth sub-block.

[0055] Optionally, the efficiency calculation unit includes:

[0056] The third efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the first sub-mode as follows:

[0057]

[0058] The fourth efficiency calculation subunit is used to obtain the theoretical computational efficiency of the matrix dimension in the second sub-mode as follows:

[0059]

[0060] In the formula, MN_ratio1 is the theoretical computational efficiency of the first sub-mode, MN_ratio2 is the theoretical computational efficiency of the second sub-mode, and m b n is the preset block size of the M-dimensional array.b The preset size of the N-dimensional blocks.

[0061] Optionally, the computing unit includes:

[0062] The block subunit is used to obtain the weight matrix block and dense input matrix block corresponding to each computing core in parallel mode, according to the selected multi-core parallel mode.

[0063] A storage subunit is used to send the weight matrix block to the scalar storage space and the dense input matrix block to the vector storage space;

[0064] The output subunit is used to perform microkernel calculations on the matrix blocks in the scalar storage space and the vector storage space to obtain the dense output matrix.

[0065] Optionally, the output subunit is used for:

[0066] Perform sparse matrix multiplication on the weight matrix block and the dense input matrix block to obtain a matrix of size m. b ×n b The calculation result; the m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0067] This application discloses a method and apparatus for multi-core parallel processing of half-precision sparse matrix multiplication for vector processors. Based on the matrix dimension and the number of computational cores, three multi-core parallel modes are constructed, applicable to various computing scenarios, and fully utilizing the multi-core architecture of vector processors. Simultaneously, computational redundancy is reduced under certain matrix dimension specifications, improving the parallelism of sparse matrix multiplication and contributing to the performance of vector processors. Specifically, each multi-core parallel mode parallelizes multiple computational cores along various dimensions of the weight matrix and dense input matrix, implementing sparse matrix multiplication in parallel across different dimensions using multiple cores. Based on the dimension specifications of the two matrices, the theoretical computational efficiency for sparse matrix multiplication in each multi-core parallel mode is obtained. Then, the multi-core parallel mode with the highest theoretical computational efficiency is selected for sparse matrix multiplication. This automatically adapts to the optimal performance mode for computation, offering high versatility and improved computational efficiency. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0069] Figure 1a This is a schematic diagram of the structure of the FT-M7032 disclosed in the embodiments of this application;

[0070] Figure 1b This is a schematic diagram of the DSP computing core structure in the FT-M7032 disclosed in the embodiments of this application;

[0071] Figure 2 This is a flowchart illustrating a multi-core parallel method for half-precision sparse matrix multiplication oriented towards vector processors, as disclosed in an embodiment of this application.

[0072] Figure 3a This is a schematic diagram of the design and computation of multi-core parallelism in the first mode disclosed in the embodiments of this application;

[0073] Figure 3b This is a schematic diagram of the kernel data block design disclosed in an embodiment of this application;

[0074] Figure 3c This is a schematic diagram of the design and computation of multi-core parallelism in the second mode disclosed in the embodiments of this application;

[0075] Figure 3d This is a schematic diagram of the design and computation of multi-core parallelism in the first sub-mode disclosed in the embodiments of this application;

[0076] Figure 4 This is a schematic diagram of the structure of a multi-core parallel device for half-precision sparse matrix multiplication oriented to a vector processor, as disclosed in an embodiment of this application. Detailed Implementation

[0077] This section introduces the structure and function of the FT-M7032: Each DSP computing core is implemented based on the VLIM architecture, and its internal architecture is as follows... Figure 1bAs shown, it mainly consists of a scalar processing unit (SPU), a vector processing unit (VPU), an instruction fetch unit (IFU), and a DMA unit. The SPU is responsible for scalar computation and flow control, and mainly includes scalar processing elements (SPEs) and vector processing units (SMs). The VPU is responsible for vector computation and consists of P vector processing elements (VPEs) and vector processing units (AMs). The VPEs operate collaboratively in a single instruction multiple data (SIMD) manner. Each VPE contains three floating-point multiply accumulator (FMAC) units, which are responsible for vector computation of floating-point data. This vector processor can transfer data from scalar registers to vector registers via broadcast / shared registers.

[0078] Floating-point numbers are a binary floating-point data type used by computers. The FT-M7032's DSP computing core supports double-precision, single-precision, and half-precision floating-point data calculations, which are stored using 8 bytes, 4 bytes, and 2 bytes, respectively. This application is based on this vector processor and is designed for half-precision floating-point numbers.

[0079] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0080] Example 1: This application discloses a multi-core parallel method for half-precision sparse matrix multiplication oriented to vector processors.

[0081] For details, please refer to Figure 2 The multi-core parallel method for half-precision sparse matrix multiplication oriented to vector processors disclosed in this embodiment includes the following steps:

[0082] Step 101: Construct multiple multi-core parallel modes based on the matrix dimension and the number of computation kernels.

[0083] In the method described in this embodiment, sparse matrix multiplication is the process of multiplying the weight matrix compressed in a sparse format with the dense input matrix to obtain a dense output matrix. The calculation formula is expressed as W×X=Y. Here, the weight matrix is ​​W, and its matrix dimension is [M][K], which represents the number of rows and columns, and can be denoted as W:[M][K]. Similarly, the dense input matrix is ​​X, and its matrix dimension is [K][N], denoted as X:[K][N], and the dense output matrix is ​​Y, and its matrix dimension is [M][N], denoted as Y:[M][N].

[0084] In the method described in this embodiment, the multi-core parallel mode includes a first mode, a second mode, and a third mode. The first mode involves each computational core operating in parallel on the M-dimensional dimension of W; this method is suitable for sparse matrix multiplication computations where M-dimensional dimensions are large and N-dimensional dimensions are small. The second mode involves each computational core operating in parallel on the N-dimensional dimension of X; this method is suitable for sparse matrix multiplication computations where N-dimensional dimensions are large and M-dimensional dimensions are small. The third mode involves each computational core operating in parallel simultaneously on both the M-dimensional and N-dimensional dimensions; this method is suitable for sparse matrix multiplication computations where both M-dimensional and N-dimensional dimensions are large.

[0085] In the method described in this embodiment, as an optional approach, there are 8 computational cores. Therefore, by evenly distributing the computational cores across W and expanding the parallel design between cores in M ​​dimensions, the first mode can be constructed. Here, the number of rows m in W corresponding to each computational core is the rounded-up one-eighth of M, i.e. The number of columns in W corresponding to each computational core is K. Therefore, the portion of W allocated to each computational core can be denoted as W0. m :[m][K].

[0086] In the method described in this embodiment, the computational cores are evenly distributed across X, and the parallel design between the cores is expanded in N dimensions to construct the second mode. Here, the number of rows in X corresponding to each computational core is K, and the number of columns in X corresponding to each computational core is n, which is the rounded-up 1 / 8 of N. Therefore, the portion of X sub-block allocated to each computing core can be denoted as X. n :[K][n].

[0087] In the method described in this embodiment, the computational cores are evenly distributed across W and X, and the parallel design between the cores is simultaneously unfolded in the M and N dimensions to construct the third mode. Since core-based computation is performed simultaneously in the M and N dimensions, the output Y is divided into 8 sub-blocks, with each computational core executing one sub-block. Therefore, there are two paradigms for the sub-blocks: 2×4 and 4×2, corresponding to the first and second sub-modes of the third mode, respectively.

[0088] In the first sub-pattern, W is divided into M dimensions to obtain two first sub-blocks W with half the number of rows M and the number of columns K.m [M / 2][K]. Simultaneously, by dividing X in N dimensions, four second sub-blocks X are obtained, each with K rows and a quarter of N columns. n [K][N / 4]. One computational core corresponds to one type of first sub-block combination, which is a combination of one first sub-block and one second sub-block. Ultimately, each computational core calculates a value of Y. mn The result of the sub-block [M / 2][N / 4].

[0089] In the second sub-pattern, W is divided in M ​​dimensions to obtain four third sub-blocks W, each with 1 / 4 M rows and K columns. m [M / 4][K]. Simultaneously, by dividing X in N dimensions, two fourth sub-blocks X with K rows and half N columns are obtained. n [K][N / 2]. One computational core corresponds to one combination of second sub-blocks, which is a combination of a third sub-block and a fourth sub-block. Ultimately, each computational core calculates a value of Y. mn The result of the sub-block [M / 4][N / 2].

[0090] Step 102: Based on the values ​​of M and N, obtain the theoretical computational efficiency of sparse matrix multiplication calculation in each of the multi-core parallel modes.

[0091] In the method described in this embodiment, as an optional approach, the microkernel algorithm for multi-core parallel computation of sparse matrix multiplication is preset to have a block size of M dimensions m. b ×K-dimensional block size k b ×N-dimensional block size n b Among them, m b and n b Here, k is a preset value, and k is a variable determined by the input parameters. Based on this, the theoretical computational efficiency in the first mode is expressed by the following formula:

[0092] (1)

[0093] In the formula, M_ratio represents the theoretical computational efficiency of the first mode, and m b Given a preset M-dimensional block size, n b This is the preset N-dimensional block size.

[0094] The theoretical computational efficiency under the second mode is expressed by the following formula:

[0095] (2)

[0096] In the formula, N_ratio represents the theoretical computational efficiency of the second mode.

[0097] The theoretical computational efficiency under the first sub-mode is expressed by the following formula:

[0098] (3)

[0099] In the formula, MN_ratio1 is the theoretical computational efficiency of the first sub-mode.

[0100] The theoretical computational efficiency under the second sub-mode is expressed by the following formula:

[0101] (4)

[0102] In the formula, MN_ratio2 represents the theoretical computational efficiency of the second sub-mode.

[0103] Step 103: Select the multi-core parallel mode with the highest theoretical computational efficiency to perform sparse matrix multiplication calculation.

[0104] In the method described in this embodiment, after obtaining M and N, formulas (1)-(4) are executed respectively to compare the four theoretical computational efficiency calculation results, and the multi-core parallel mode corresponding to the highest value is selected to perform sparse matrix multiplication calculation accordingly.

[0105] As an optional method, when performing sparse matrix multiplication, the W allocated to each computing core during parallel processing is obtained according to the selected multi-core parallel mode. m and X n Then W m Send to SM space, X n Send to AM space. Perform microkernel calculations on the matrix sub-blocks in SM and AM spaces to obtain Y. Among them, W... m Divide the SM space into blocks, and X n By dividing the AM space into blocks and performing sparse matrix multiplication of the microkernel at the granularity of the blocks, a sparse matrix multiplication of size m can be obtained. b ×n b The calculation result is denoted as Y. b :[m b ][n b ]. Then take the Y of each block b The data is sent to the DDR outside the core until all blocks have been calculated, ultimately yielding Y. m :[m][N].

[0106] The method described in this embodiment employs a multi-core parallel design, supporting the construction of various multi-core parallel modes based on the matrix dimension and the number of computational cores. It is applicable to various computing scenarios and fully utilizes the multi-core architecture of vector processors. This achieves multi-core implementation of sparse matrix multiplication on vector processors, reducing computational redundancy in certain matrix dimensions under single-mode conditions, improving computational parallelism, and contributing to the performance of vector processors. Furthermore, based on the matrix dimension, the theoretical computational efficiency for sparse matrix multiplication in each multi-core parallel mode is obtained. The multi-core parallel mode with the highest theoretical computational efficiency is then selected for sparse matrix multiplication. This automatically adapts to the optimal performance mode for computation, offering high versatility and improved computational efficiency.

[0107] Example 2: This application discloses a method for implementing sparse matrix multiplication in the first mode:

[0108] Step 201: Based on the first mode, M, and K, obtain the matrix blocks W allocated to each computing core. m :[m][K]、W m In the SM space, the W block m s :[m s [K], Number of blocks n_W m s : .

[0109] In the method described in this embodiment, due to the limited SM and AM spaces within the kernel, data often needs to be transmitted and computed in blocks. The specific block design within the kernel is related to the microkernel block specification. This can be determined according to m... b The available space sizes of K and SM are used to obtain W on SM. m The M-dimensional values ​​of the block m s = (SM available space size) / FP16 bytes / K / m b ×m b , obtained SM on W m The block is denoted as W. m s :[m s [K], with a number of n_W m s : According to m b n b K and AM can be used with space size and DMA double buffering mechanism to calculate the N-dimensional values ​​N of the submatrix of X on AM. a = (AM available space size / FP16 bytes number - (m b ×n b)) / 2 / K / n b ×n b We obtain the block X on AM, denoted as X. a :[K][N a The number of [items] is n_X a : .

[0110] In addition, each computational core is based on its core number and W. m The matrix specification is used to obtain the corresponding data address mapped to W.

[0111] Step 202: Based on the first mode, N, and K, obtain the block X of X in the AM space. a :[K][N a Number of blocks n_X a : The AM space is divided into three intervals: AM_X1, AM_X2, and AM_Y.

[0112] Step 203: Initialize W m s In W m Let the index i be 0.

[0113] Step 204: Transfer the corresponding W via DMA m s Send to SM space in a point-to-point manner.

[0114] In the method described in this embodiment, the W of each computing core m The matrix will be in W m s Formal division n_W m s This transmission will be sent to SM. X will be transmitted in X format. a Forms of n_X a The next transmission is to AM.

[0115] Step 205: Transfer the first X on X via DMA a Send it to AM_X1 via broadcast.

[0116] Step 206: Initialize X a Let j = 1, which is the index j on X.

[0117] Step 207: Transfer the corresponding X via DMA a Send to AM_X2 via broadcast.

[0118] Step 208: Set index i of W... m s With index j-1 X aPerform sparse matrix multiplication calculations, store the results in AM_Y, and then transfer them back to DDR via DMA in a point-to-point manner. Until W... m s and X a All data calculations and transmissions are complete.

[0119] In the method described in this embodiment, the block size of the microkernel computing block for performing sparse matrix multiplication is m. b ×K×n b Therefore, we can obtain a size of m each time. b ×n b The calculation results.

[0120] Step 209: Let j = j + 1, and determine whether j is less than n_X. a If yes, return to step 207. If no, proceed to step 210.

[0121] Step 210: Set the W index to i m s With index j-1 X a Perform sparse matrix multiplication calculations, store the results in AM_Y, and then transfer them back to DDR via DMA in a point-to-point manner. Until W... m s and X a All data calculations and transmissions are complete.

[0122] Step 211: Let i = i + 1, and determine whether i is less than n_W. m s If yes, return to step 204. If no, proceed to step 212.

[0123] Step 212: Fence synchronization, wait for all computation cores to complete the calculation and transmit the m×N Y. m To DDR.

[0124] In the method described in this embodiment, in the first mode, each computing core performs multi-core parallelism in M ​​dimensions, and its design and calculation are as follows: Figure 3a As shown, the computational cores are evenly distributed across W, and the parallel design among the cores unfolds in M ​​dimensions. Specifically, there are 8 computational cores, and each core is allocated W... m Each computing core completes its own W m The sparse matrix multiplication with X yields Y. m In the data storage structure design, W will be used. m X is sent to SM space, X is sent to AM space, and Y is calculated. m It is also stored in AM.

[0125] In the method described in this embodiment, the kernel block partitioning approach is as follows: Figure 3b As shown, W m s In the diagram, Dxx represents the data in the compressed matrix, Ixx represents the index, and W represents the index. m s and X a With size m b ×K×n b The sparse matrix multiplication of the microkernel is performed at the granularity level, and the microkernel obtains m values ​​each time. b ×n b The calculation result is denoted as Y. b :[m b ][n b ], the Y b It is stored in AM and then transported to the extracore DDR space. Then the loop iterates through W. m s and X a Get m s ×N a The calculation results are now complete, thus concluding the sparse matrix multiplication calculation of the single data transfer from DMA. Finally, the M and N dimensions in the DDR are traversed until the calculation is complete, and each calculation core ultimately obtains Y. m .

[0126] Example 3: This application discloses a method for implementing sparse matrix multiplication in the second mode:

[0127] Step 301: Based on the second mode, M, and K, obtain the block W of W in the SM space. s :[M s [K], Number of blocks n_W s : .

[0128] In the method described in this embodiment, it can be based on m b Given the available space sizes of K and SM, we can obtain the M-dimensional values ​​M of the W blocks on SM. s = (SM available space size) / FP16 bytes / K / m b ×m b We obtain the W block on SM, denoted as W. s :[M s [K], with a number of n_W s : According to m b n b K, AM available space size and DMA double buffering mechanism, calculate X on AM n The submatrix has N dimensions and takes n values a = (AM available space size / FP16 bytes number - (mb ×n b )) / 2 / K / n b ×n b , obtain AM on X n The block is denoted as X. n a :[K][n a The number of [items] is n_X n a : .

[0129] Step 302: Based on the second mode, N, and K, obtain the matrix blocks X allocated to each computing core. n :[K][n]、X n In AM space, the block X n a :[K][N a Number of blocks n_X n a : The AM space is then divided into three intervals: AM_X1, AM_X2, and AM_Y.

[0130] In the method described in this embodiment, each computing core is based on the core sequence number and X. n a The matrix specification is used to obtain the corresponding data address mapped to X.

[0131] Step 303: Initialize W s Let i be the index of W, and let i = 0.

[0132] Step 304: Place the corresponding W s Send to SM space.

[0133] In the method described in this embodiment, the W matrix of each computational core will be in W... s Formal division n_W s This transmission is sent to SM. X n Will be X n a Forms of n_X n a The next transmission is to AM.

[0134] Step 305: Place X n The first X n a Send to AM_X1.

[0135] Step 306: Initialize X n a Let j = 1, which is the index j on X.

[0136] Step 307: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] n a Send to AM_X2.

[0137] Step 308: Set index i of W... s With index j-1 X n a Perform sparse matrix multiplication, store the results in AM_Y, and then transfer them back to DDR. Until W... s and X n a All data calculations and transmissions are complete.

[0138] Step 309: Let j = j + 1, and determine whether j is less than n_X. n a If yes, return to step 307. If no, proceed to step 310.

[0139] Step 310: Set the index of W to i s With index j-1 X n a Perform sparse matrix multiplication, store the results in AM_Y, and then transfer them back to DDR. Until W... s and X n a All data calculations and transmissions are complete.

[0140] Step 311: Let i = i + 1, and determine whether i is less than n_W. s If yes, return to step 304. If no, proceed to step 312.

[0141] Step 312: Fence synchronization, wait for all computation cores to complete the calculation and transmit M×n Y. n To DDR.

[0142] In the method described in this embodiment, in the second mode, each computing core performs multi-core parallelism in N dimensions, and its design and calculation are as follows: Figure 3c As shown, the computational cores are evenly distributed across X, and the parallel design among the cores unfolds in N dimensions. Specifically, there are 8 computational cores, and each core is allocated X... n Each computational core completes its respective W and X calculations. n The sparse matrix multiplication calculation yields Y. n In the data storage structure design, W is sent to the SM space, and X... n Send to AM space, calculate Y n It is also stored in AM.

[0143] In the method described in this embodiment, W s and X n aWith size m b ×K×n b Y is obtained by performing sparse matrix multiplication on a microkernel at the granularity level. b Then iterate through loop W. s and X n a Get M s ×n a The calculation results are then iterated through dimensions M and N until the calculation is complete, and each calculation kernel finally obtains Y. n .

[0144] Example 4: This application discloses a method for implementing sparse matrix multiplication under the first sub-pattern:

[0145] Step 401: Based on the first sub-pattern, M, and K, obtain the matrix blocks W allocated to each computational core of W. m [M / 2][K]、W m In the SM space, the W block m s :[M s [K], Number of blocks n_W m s : .

[0146] In the method described in this embodiment, it can be based on m b The available space sizes of K and SM are used to obtain W on SM. m The M-dimensional values ​​of the block s = (SM available space size) / FP16 bytes / K / m b ×m b We obtain the W block on SM, denoted as W. m s :[M s [K], with a number of n_W m s : According to m b n b Given K, N / 4, AM available space size and DMA double buffering mechanism, calculate X on AM. n The submatrix has N dimensions and takes N values. a = (AM available space size / FP16 bytes number - (m b ×n b )) / 2 / K / n b ×n b , obtain AM on X n The block is denoted as X. n a:[K][n a The number of [items] is n_X n a : .

[0147] In the method described in this embodiment, each computing core is based on its core number and W. m The matrix specification is used to obtain the corresponding data address mapped to W.

[0148] Step 402: Based on the second mode, N, and K, obtain the matrix blocks X allocated to each computing core. n :[K][N / 4]、X n In AM space, the block X n a :[K][N a Number of blocks n_X n a : The AM space is then divided into three intervals: AM_X1, AM_X2, and AM_Y.

[0149] Step 403: Initialize W m s In W m Let the index i be 0.

[0150] Step 404: Place the corresponding W m s Send to SM space.

[0151] In the method described in this embodiment, the W of each computing core m The matrix will be in W m s Formal division n_W m s This transmission is sent to SM. X n Will be X n a Forms of n_X n a The next transmission is to AM.

[0152] Step 405: Place X n The first X n a Send to AM_X1.

[0153] Step 406: Initialize X n a Let j = 1, which is the index j on X.

[0154] Step 407: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] n a Send to AM_X2.

[0155] Step 408: Set index i of W... m s With index j-1 X n a Perform sparse matrix multiplication, store the results in AM_Y, and then transfer them back to DDR. Until W... m s and X n a All data calculations and transmissions are complete.

[0156] Step 409: Let j = j + 1, and determine whether j is less than n_X. n a If yes, return to step 407. If no, proceed to step 410.

[0157] Step 410: Set the W index to i m s With index j-1 X n a Perform sparse matrix multiplication, store the results in AM_Y, and then transfer them back to DDR. Until W... m s and X n a All data calculations and transmissions are complete.

[0158] Step 411: Let i = i + 1, and determine whether i is less than n_W. m s If yes, return to step 404. If no, proceed to step 412.

[0159] Step 412: Fence synchronization, wait for all computation cores to complete the calculation and transmit the M / 2×N / 4 Y. mn To DDR.

[0160] In the method described in this embodiment, in the third mode, the computational cores are evenly distributed across the W and X matrices, and the parallel design between cores is simultaneously deployed in the M and N dimensions. Specifically, the number of computational cores is 8. As an optional method, the W matrix is ​​divided in the M dimension to obtain two sub-blocks W. m [M / 2][K], and simultaneously divide the X matrix in N dimensions to obtain 4 sub-blocks X. n [K][N / 4], 8 cores are sequentially assigned a combination of sub-blocks, and each core calculates a size of Y. mn The result of the sub-blocks is [M / 2][N / 4], which is the first sub-pattern. Alternatively, the W matrix can be divided into M dimensions to obtain 4 sub-blocks W. m [M / 4][K], and simultaneously divide the X matrix in N dimensions to obtain two sub-blocks X. n[K][N / 2], 8 cores are sequentially assigned a combination of sub-blocks, and each core calculates a size of Y. mn The result of the sub-block [M / 4][N / 2] is the second sub-pattern.

[0161] The first sub-pattern is described in detail, including its design and calculation. Figure 3d As shown: In the first sub-mode, each computing core is allocated W. m [M / 2][K] and X n [K][N / 4]. In data storage structure design, W... m Send to SM space, X n Send to AM space, calculate Y mn It is also stored in AM.

[0162] In the method described in this embodiment, W m s and X n a With size m b ×K×n b Y is obtained by performing sparse matrix multiplication on a microkernel at the granularity level. b Then iterate through loop W. m s and X n a Get M s ×N a The calculation results are then iterated through dimensions M and N until the calculation is complete, and each calculation kernel finally obtains Y. mn .

[0163] Based on the multi-core parallel method for half-precision sparse matrix multiplication for vector processors disclosed in the above embodiments, this embodiment correspondingly discloses a multi-core parallel apparatus for half-precision sparse matrix multiplication for vector processors. Sparse matrix multiplication is the process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix. Please refer to [link to relevant documentation]. Figure 4 The device includes: a pattern construction unit 501, an efficiency calculation unit 502, and a calculation unit 503;

[0164] The mode construction unit 501 is used to construct multiple multi-core parallel modes based on the matrix dimension and the number of computational cores; the matrix dimension is the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is ​​[M][K], the matrix dimension of the dense input matrix is ​​[K][N], and the matrix dimension of the dense output matrix is ​​[M][N]; the multi-core parallel modes include a first mode, a second mode, and a third mode; the first mode is that each of the computational cores operates in parallel on the M dimension of the weight matrix; the second mode is that each of the computational cores operates in parallel on the N dimension of the dense input matrix; the third mode is that each of the computational cores operates in parallel on both the M dimension and the N dimension simultaneously;

[0165] The efficiency calculation unit 502 is used to obtain the theoretical computational efficiency of sparse matrix multiplication calculation in each of the multi-core parallel modes based on the values ​​of M and N.

[0166] The computing unit 503 is used to select the multi-core parallel mode with the highest theoretical computing efficiency to perform sparse matrix multiplication calculation.

[0167] Optionally, the computing cores are 8, and the pattern construction unit 501 includes:

[0168] The first allocation subunit is used to evenly distribute the computational cores on the weight matrix to construct the first mode; wherein the number of rows of the weight matrix corresponding to each computational core is one-eighth of M rounded up, and the number of columns of the weight matrix corresponding to each computational core is K.

[0169] Optionally, the efficiency calculation unit 502 includes:

[0170] The first efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the first mode as follows:

[0171]

[0172] In the formula, M_ratio represents the theoretical computational efficiency of the first mode, and m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0173] Optionally, the computing cores are 8, and the pattern construction unit 501 includes:

[0174] The second allocation subunit is used to evenly distribute each of the computing cores on the dense input matrix to construct the second mode; wherein, the number of rows of the dense input matrix corresponding to each computing core is K, and the number of columns of the dense input matrix corresponding to each computing core is the rounded-up of one-eighth N.

[0175] Optionally, the efficiency calculation unit 502 includes:

[0176] The second efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the second mode as follows:

[0177]

[0178] In the formula, N_ratio represents the theoretical computational efficiency of the second mode, m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0179] Optionally, the number of computing cores is 8, the third mode includes a first sub-mode and a second sub-mode, and the mode construction unit 501 includes:

[0180] The third allocation subunit is used to divide the weight matrix in M ​​dimensions to obtain two first sub-blocks with half M rows and K columns, and simultaneously divide the dense input matrix in N dimensions to obtain four second sub-blocks with K rows and one-quarter N columns, thus constructing the first sub-pattern; one computing kernel corresponds to one combination of the first sub-blocks; the first sub-block combination is a combination of one first sub-block and one second sub-block;

[0181] The fourth allocation subunit is used to divide the weight matrix in M ​​dimensions to obtain four third sub-blocks with one-quarter M rows and K columns, and simultaneously divide the dense input matrix in N dimensions to obtain two fourth sub-blocks with K rows and one-half N columns, thus constructing the second sub-mode; one computing kernel corresponds to one combination of second sub-blocks; the combination of second sub-blocks is a combination of one third sub-block and one fourth sub-block.

[0182] Optionally, the efficiency calculation unit 502 includes:

[0183] The third efficiency calculation subunit is used to obtain the theoretical calculation efficiency of the matrix dimension in the first sub-mode as follows:

[0184]

[0185] The fourth efficiency calculation subunit is used to obtain the theoretical computational efficiency of the matrix dimension in the second sub-mode as follows:

[0186]

[0187] In the formula, MN_ratio1 is the theoretical computational efficiency of the first sub-mode, MN_ratio2 is the theoretical computational efficiency of the second sub-mode, and mb n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0188] Optionally, the computing unit 503 includes:

[0189] The block subunit is used to obtain the weight matrix block and dense input matrix block corresponding to each computing core in parallel mode, according to the selected multi-core parallel mode.

[0190] A storage subunit is used to send the weight matrix block to the scalar storage space and the dense input matrix block to the vector storage space;

[0191] The output subunit is used to perform microkernel calculations on the matrix blocks in the scalar storage space and the vector storage space to obtain the dense output matrix.

[0192] Optionally, the output subunit is used for:

[0193] Perform sparse matrix multiplication on the weight matrix block and the dense input matrix block to obtain a matrix of size m. b ×n b The calculation result; the m b n is the preset block size of the M-dimensional array. b The preset size of the N-dimensional blocks.

[0194] The embodiments in this specification are described in a progressive manner. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant details can be found in the method section.

[0195] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0196] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0197] The features described in the embodiments of this specification can be substituted for or combined with each other, so that those skilled in the art can implement or use this application.

[0198] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for semi-precision sparse matrix multiplication multi-core parallelism for a vector-oriented processor, characterized in that, The sparse matrix multiplication is a process of multiplying a weight matrix compressed in a sparse format with a dense input matrix to obtain a dense output matrix, and the method comprises: According to the matrix dimension and the number of calculation cores, a plurality of multi-core parallel modes are constructed; the matrix dimension is the number of rows and columns in the matrix, wherein the matrix dimension of the weight matrix is [M][K], the matrix dimension of the dense input matrix is [K][N], and the matrix dimension of the dense output matrix is [M][N]; the multi-core parallel mode comprises a first mode, a second mode and a third mode; the first mode is that each calculation core is parallel in the M dimension of the weight matrix; the second mode is that each calculation core is parallel in the N dimension of the dense input matrix; and the third mode is that each calculation core is parallel in the M dimension and the N dimension simultaneously; Based on the values of M and N, the theoretical calculation efficiency of sparse matrix multiplication calculation in each multi-core parallel mode is obtained; The multi-core parallel mode with the maximum theoretical calculation efficiency is selected for sparse matrix multiplication calculation; When the multi-core parallel mode is the first mode, the number of calculation cores is 8, and the plurality of multi-core parallel modes are constructed according to the matrix dimension and the number of calculation cores, comprising: The calculation cores are evenly distributed on the weight matrix to construct the first mode; wherein the number of rows of the weight matrix corresponding to each calculation core is the integer part of one eighth of M, and the number of columns of the weight matrix corresponding to each calculation core is K; The theoretical calculation efficiency of sparse matrix multiplication calculation in each multi-core parallel mode is obtained, comprising: The theoretical calculation efficiency of the matrix dimension in the first mode is obtained as follows: ； where M_ratio is the theoretically calculated efficiency of the first mode, m b is a preset block size in the M dimension, n b is a preset block size in the N dimension.

2. The method of claim 1, wherein, When the number of calculation cores is 8, the plurality of multi-core parallel modes are constructed according to the matrix dimension and the number of calculation cores, comprising: The calculation cores are evenly distributed on the dense input matrix to construct the second mode; wherein the number of rows of the dense input matrix corresponding to each calculation core is K, and the number of columns of the dense input matrix corresponding to each calculation core is the integer part of one eighth of N.

3. The method of claim 2, wherein, The theoretical calculation efficiency of sparse matrix multiplication calculation in each multi-core parallel mode is obtained, comprising: The theoretical calculation efficiency of the matrix dimension in the second mode is obtained as follows: ； where N_ratio is the theoretically calculated efficiency of the second mode, m b is a preset block size in the M dimension, n b is a preset block size in the N dimension.

4. The method of claim 1, wherein, When the number of calculation cores is 8, the third mode comprises a first sub-mode and a second sub-mode, and the plurality of multi-core parallel modes are constructed according to the matrix dimension and the number of calculation cores, comprising: The M dimension of the weight matrix is divided to obtain two first sub-blocks with a row number of one half of M and a column number of K, and the N dimension of the dense input matrix is divided to obtain four second sub-blocks with a row number of K and a column number of one fourth of N, to construct the first sub-mode; one calculation core corresponds to one first sub-block combination; the first sub-block combination is a combination of one first sub-block and one second sub-block; In the M-dimensional partition of the weight matrix, four third sub-blocks with row number of one fourth of M and column number of K are obtained, and in the N-dimensional partition of the dense input matrix, two fourth sub-blocks with row number of K and column number of one half of N are obtained, to construct the second sub-mode; one of the calculation cores corresponds to one second sub-block combination; the second sub-block combination is a combination of one third sub-block and one fourth sub-block.

5. The method of claim 4, wherein, The theoretical calculation efficiency of the sparse matrix multiplication calculation in each of the multi-core parallel modes is obtained, including: The theoretical calculation efficiency of the matrix dimension in the first sub-mode is obtained as follows: ； The theoretical calculation efficiency of the matrix dimension in the second sub-mode is obtained as follows: ； where MN_ratio1 is the theoretically calculated efficiency of the first sub-mode, MN_ratio2 is the theoretically calculated efficiency of the second sub-mode, m b is the preset block size in the M dimension, n b is the preset block size in the N dimension.

6. The method of claim 1, wherein, The multi-core parallel mode with the maximum theoretical calculation efficiency is selected for the sparse matrix multiplication calculation, including: According to the selected multi-core parallel mode, the weight matrix block and the dense input matrix block corresponding to each calculation core in parallel are obtained; The weight matrix block is sent to the scalar storage space, and the dense input matrix block is sent to the vector storage space; The matrix blocks in the scalar storage space and the vector storage space are subjected to micro-kernel calculation to obtain the dense output matrix.

7. The method of claim 6, wherein, The micro-kernel calculation includes: performing sparse matrix multiplication calculation on the weight matrix block and the dense input matrix block to obtain a calculation result with a size of m b ×n b ; m b is a preset block size in the M dimension, and n b is a preset block size in the N dimension.

8. An apparatus for semi-precision sparse matrix multiplication multi-core parallelism of a vector-oriented processor, characterized by, The device is applied to the method of any one of claims 1-7, the sparse matrix multiplication is a process of multiplying the weight matrix compressed in a sparse format with the dense input matrix to obtain the dense output matrix, and the device includes a mode construction unit, an efficiency calculation unit and a calculation unit. The mode construction unit is configured to construct a plurality of multi-core parallel modes according to the matrix dimension and the number of calculation cores; the matrix dimension is the number of rows and the number of columns in the matrix, wherein the matrix dimension of the weight matrix is [M][K], the matrix dimension of the dense input matrix is [K][N], and the matrix dimension of the dense output matrix is [M][N]; the multi-core parallel modes include a first mode, a second mode and a third mode; the first mode is that each calculation core is parallel in the M dimension of the weight matrix; the second mode is that each calculation core is parallel in the N dimension of the dense input matrix; and the third mode is that each calculation core is parallel in the M dimension and the N dimension simultaneously; The efficiency calculation unit is configured to obtain the theoretical calculation efficiency of the sparse matrix multiplication calculation in each of the multi-core parallel modes based on the values of M and N; The calculation unit is configured to select the multi-core parallel mode with the maximum theoretical calculation efficiency for the sparse matrix multiplication calculation.

Images