Sparse matrix storage method, vector calculation method and electronic device

Abstract

The present application relates to a sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device, a vector calculation method performed by the computing device, an electronic device, and a computer-readable storage medium. The sparse matrix storage method includes: determining the number of columns of a vector register for vector calculation in the computing device; determining the number of non-zero elements in each row of the sparse matrix; based on the number of non-zero elements in each row of the sparse matrix and the number of columns of the vector register for vector calculation, determining a group of the sparse matrix comprising a plurality of matrix rows, wherein the difference between the number of non-zero elements of each matrix row in the plurality of matrix rows in each group is less than a first threshold value; and storing the non-zero elements corresponding to the same matrix row in each group in the memory by column. In this way, the storage performance of the sparse matrix is ​​improved and the vector calculation process is simplified.

Description

Technical Field

[0001] The present invention generally relates to the field of data processing, and in particular to a sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device, a vector calculation method performed by the computing device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Sparse matrix-vector multiplication (SpMV) is a fundamental and important operator in high-performance computing, widely used in fields such as graph computing, social network analysis, circuit simulation, structural mechanics, fluid dynamics, oil exploration, and protein analysis. Many algorithms rely on SpMV operators, including PageRank for web page ranking, the conjugate gradient method for solving large linear systems, the High-Performance Conjugate Gradient (HPCG) benchmark for measuring supercomputer performance, the 2016 Supercomputing Application Award for atmospheric simulation, and the Gordon Bell Prize-winning work "Ten Million-Core Scalable Fully Implicit Simulation of Atmospheric Dynamics." Therefore, accelerating SpMV operations is crucial to improving the performance of these algorithms.

[0003] SpMV calculation can be expressed as y=A*x, where A is a sparse matrix, x is a dense input vector, and y is a dense output vector. In actual production, sparse matrices may be very sparse, for example, only 0.001% of the matrix elements are non-zero. Therefore, the industry usually uses formats such as CSR (compressed sparse row) and COO (coordinate) to store sparse matrices. These formats only record the row and column numbers of non-zero elements. In this way, indexed-access modes such as a[b[i]] are required in SpMV calculations, which will result in random access to array a, reduce data locality, and thus reduce the performance of SpMV.

[0004] In summary, SpMV operations face the following challenges: 1. Random memory access: SpMV's random access pattern reduces data locality, increases cache miss rates, and reduces computational performance. 2. Memory bandwidth: In the commonly used CSR storage format, one SpMV multiplication requires three memory access operations, which increases the access-to-computation ratio of SpMV. In a multi-threaded environment, memory bandwidth becomes a bottleneck restricting SpMV performance. 3. Short vector problem: Currently, many general-purpose processors support vectorized computing, such as Intel processors that support 512-bit vector computing (AVX-512). However, a row of a sparse matrix may have too few elements to fill a vector. This leads to the short vector problem, reducing computational density and utilization.

[0005] Therefore, it is desirable to provide an improved sparse matrix storage method for vector calculation and a corresponding vector calculation method. Summary of the Invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a sparse matrix storage method for storing a sparse matrix for vector calculation in the memory of a computing device, a vector calculation method performed by the computing device, an electronic device, and a computer-readable storage medium. The method improves the storage performance of the sparse matrix and simplifies the vector calculation process by grouping the matrix rows based on the number of non-zero elements in the matrix rows and storing the non-zero elements corresponding to the same matrix row in the group by column.

[0007] According to one aspect of the present application, a sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device is provided, characterized by comprising:

[0008] determining the number of columns of a vector register used for vector calculations in a computing device;

[0009] Determining the number of non-zero elements in each row of the sparse matrix; and

[0010] Determining, based on the number of non-zero elements in each row of the sparse matrix and the number of columns of the vector register for vector calculation, groups of the sparse matrix comprising a plurality of matrix rows, wherein a difference between the numbers of non-zero elements of respective matrix rows of the plurality of matrix rows in each group is less than a first threshold, such that the plurality of columns of the vector register are filled and all columns of the vector register require the same or a similar number of multiplication-addition operations that meet a predetermined threshold; and

[0011] The non-zero elements in each group corresponding to the same matrix row are stored in the memory by column, respectively, so that the result of the vector calculation can be saved to the memory using a vector save instruction.

[0012] In the technical solution of this aspect, the difference between the number of non-zero elements of each matrix row in the multiple matrix rows in each group is less than a first threshold value, so that multiple columns of the vector register are filled, and all columns of the vector register require the same or a similar number of multiplication and addition operations that meet the predetermined threshold, thereby avoiding the introduction of conditional branch instructions for judging whether certain vector columns of the current vector register are the last elements of a row and the need to save the scalar data of these columns to the memory in sequence, and the operation of extracting the data of a certain column from the vector register and saving it to the scalar register; and the non-zero elements of the same matrix row are stored in the memory by column respectively so that the results of the vector calculation can be saved to the memory using a vector save instruction, so as to realize that different columns of the vector register save the accumulated results of different rows, and the accumulation process of N rows is performed simultaneously during vector multiplication and addition, and the addition process of the reduced summation of the multiplication results of one row is simplified.

[0013] In the above-mentioned sparse matrix storage method, it further includes: determining the remaining ungrouped matrix rows of the sparse matrix; and storing the non-zero elements in the remaining matrix rows in the memory in rows based on the number of columns of the vector register and in units of each matrix row.

[0014] In the above-mentioned sparse matrix storage method, before determining the number of non-zero elements in each row of the sparse matrix, it further includes: obtaining an initial matrix to be divided; dividing the initial matrix to be divided into multiple blocks in two dimensions based on a fixed size; and merging multiple horizontally continuous blocks that meet preset conditions into large blocks, wherein the large blocks serve as the sparse matrix to improve the hit rate of each row of the sparse matrix in the cache.

[0015] In the above-mentioned sparse matrix storage method, the preset conditions include: the number of non-zero elements in all blocks and the merged large blocks does not exceed a second threshold; and / or the distance between the blocks is less than or equal to a compression threshold, wherein the block distance is the number of horizontal blocks included in the large block, and the compression threshold enables all the data of the input and output vectors corresponding to the blocks to be stored in the cache.

[0016] In the above-mentioned sparse matrix storage method, after merging multiple horizontally continuous blocks that meet preset conditions into a large block as a sparse matrix, it further includes: renumbering the row numbers and column numbers of the non-zero elements of the sparse matrix from zero, and splicing them into 32-bit integers, so that during vector calculation, the row numbers and column numbers are obtained by bit operations and logical right shift operations, and the row numbers and column numbers are stored in four bytes of an integer type.

[0017] In the above-mentioned sparse matrix storage method, the preset condition further includes: the column distance between the first column of the block at one end of the large block and the last column of the block at the other end of the large block is less than or equal to 65536, so that the column number is stored in two bytes of short type.

[0018] In the above sparse matrix storage method, obtaining the initial matrix to be divided includes: obtaining the initial matrix to be divided allocated to each thread of the CPU, so that each thread has the same or similar storage operands.

[0019] In the above sparse matrix storage method, the first threshold is determined by a machine learning method.

[0020] According to another aspect of the present application, a vector calculation method performed by a computing device is provided, comprising: obtaining a sparse matrix stored according to the sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of the computing device as described above; performing vector calculation using non-zero elements of the sparse matrix that are stored in the memory by column and correspond to the group including multiple matrix rows; and performing vector calculation using non-zero elements of the sparse matrix that are not stored in the memory by column and do not correspond to the group including multiple matrix rows.

[0021] The above-mentioned vector calculation method further includes: each thread calculates the sparse matrix-vector multiplication operation corresponding to the first and last rows of the continuous area allocated to it; temporarily stores the calculation results in an array buffer; and, after all threads complete the calculation, serially accumulates the values ​​in the array buffer to the corresponding positions of the output vector.

[0022] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the sparse matrix storage method for storing a sparse matrix for vector calculation in the memory of a computing device as described above, or the vector calculation method executed by the computing device as described above.

[0023] According to another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a computing device, the medium can be operated to execute the sparse matrix storage method for storing sparse matrices for vector calculations in the memory of the computing device as described above, or the vector calculation method executed by the computing device as described above.

[0024] The present application provides a sparse matrix storage method for storing a sparse matrix for vector calculation in the memory of a computing device, a vector calculation method performed by the computing device, an electronic device and a computer-readable storage medium. The method improves the storage performance of the sparse matrix and simplifies the vector calculation process by grouping the matrix rows based on the number of non-zero elements in the matrix rows of the sparse matrix and storing the non-zero elements corresponding to the same matrix row in the group by column. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] These and / or other aspects and advantages of the present invention will become more apparent and more readily understood from the following detailed description of embodiments of the present invention in conjunction with the accompanying drawings, in which:

[0026] Figure 1 A flowchart of a sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device according to an embodiment of the present application is illustrated;

[0027] Figure 2 FIG2 is a schematic diagram illustrating a regular arrangement in a sparse matrix storage method according to an embodiment of the present application;

[0028] Figure 3 The figure illustrates the effect of different first thresholds T1 on the number of zero elements filled in different sparse matrices in the sparse matrix storage method according to an embodiment of the present application;

[0029] Figure 4 A schematic pseudo-code diagram of a dynamic programming algorithm in a sparse matrix storage method according to an embodiment of the present application is illustrated;

[0030] Figure 5 The pseudo code of SpMV in CSR format is shown (without two-dimensional partitioning)

[0031] Figure 6 A schematic diagram illustrating an example of block merging in a sparse matrix storage method according to an embodiment of the present application is shown;

[0032] Figure 7 A schematic diagram illustrating a pseudo code of a block merging process in a sparse matrix storage method according to an embodiment of the present application is shown;

[0033] Figure 8 The figure shows the execution time of different threads and the number of scatter instructions for the sparse matrix flickr under 28 threads (both have been normalized)

[0034] Figure 9 An example of load balancing in a sparse matrix storage method according to an embodiment of the present application is illustrated;

[0035] Figure 10A schematic diagram illustrating non-zero elements allocated to different threads in a multi-threaded environment in a sparse matrix storage method according to an embodiment of the present application is shown;

[0036] Figure 11 FIG2 is a schematic diagram illustrating the overall calculation process of SpMV according to an embodiment of the present application;

[0037] Figure 12 A schematic flow chart illustrating a vector calculation method performed by a computing device according to an embodiment of the present application is shown.

[0038] Figure 13 FIG2 is a pseudo code diagram illustrating a calculation process of regularly arranged elements in a vector calculation method according to an embodiment of the present application;

[0039] Figure 14 A pseudo code diagram illustrating a calculation process of irregularly arranged elements in a vector calculation method according to an embodiment of the present application is illustrated;

[0040] Figure 15 The figure shows a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0042] Application Overview

[0043] As mentioned above, the storage format of sparse matrices will significantly affect the performance of vector calculations. Currently, there are mainly the following storage methods for sparse matrices.

[0044] In 2015, Wai Teng Tang et al. proposed the VHCC format at the CGO international academic conference. This format first arranges all nonzero elements of a sparse matrix in row-major order, then computes the SpMV multiplication vector by vector. It then calculates the segment sum within a vector based on whether the nonzero elements in the vector belong to the same row. The VHCC format minimizes the number of vector multiplications during the multiplication phase, and the calculation process is unaffected by the distribution of the nonzero elements in the sparse matrix.

[0045] However, after each vector multiplication, the VHCC format requires a conditional branch during the reduction phase to determine whether the last element of a row is included in the vector multiplication. If so, the segment sum is calculated; if not, the entire vector sum is reduced. Both operations require log2(N) additions, where N is the number of vector lanes. Therefore, the ratio of the number of vector multiplications to additions in the VHCC format is 1:log2(N). Under Intel's AVX-512 instruction set, a single multiplication and addition of double-precision floating-point numbers takes four clock cycles. Therefore, the excessive number of additions in VHCC increases the computational workload and reduces SpMV performance. Furthermore, the conditional branch statements introduced by the VHCC format make branch prediction difficult, which can block the processor pipeline and reduce computational performance. During the storage phase, VHCC uses scalar instructions to store calculation results, underutilizing vector storage instructions.

[0046] In addition, in 2015, Weifeng Liu et al. proposed the CSR5 format at the International Academic Conference (ICS). This format is similar to VHCC, but differs in that after all non-zero elements of the sparse matrix are arranged row-first, VHCC arranges them row-first in each vector lane, while CSR5 arranges them column-first. The advantage of this is that hardware multiplication-add instructions (fused multiply-add, FMA) can be used to accelerate calculations during the multiplication stage. Under Intel's AVX-512 instruction set, a multiplication-add operation on double-precision floating-point numbers takes four clock cycles, so directly using multiplication-add instructions reduces the time overhead by half compared to the method of multiplying first and then adding.

[0047] CSR5 requires several arrays to record whether the last element of a row is included in each vector calculation, whether there are empty rows in the matrix, and other information. The CSR5 format minimizes the number of vector multiplications in the multiplication stage, and the calculation process is not affected by the distribution of non-zero elements in the sparse matrix. However, in the reduction stage, CSR5 also needs to introduce conditional branch statements like VHCC, and additional index values ​​need to be calculated to skip empty rows in the sparse matrix. In addition, the CSR5 format also requires a calculation part and a method for handling the situation where a row in the sparse matrix is ​​split into multiple vector columns.

[0048] In 2018, Biwei Xie et al. proposed the CVR format at the international academic conference CGO. This format is similar to CSR5, but differs in that CVR ensures that the nonzero elements in a row of a sparse matrix are arranged in the same column of the vector, while CSR5 is likely to split the nonzero elements of a row into different columns of the vector. This feature of CVR completely avoids vector addition, replacing it with vector multiplication and addition (FMA), which reduces the amount of computation. However, like VHCC and CSR5, CVR requires conditional branch instructions to determine whether the last element of a row is included in each vector calculation. If so, CVR uses scalar instructions rather than vector instructions to store the calculation results.

[0049] In 2014, Moritz Kreutzer et al. proposed the SELL-C-σ format in the SIAM Journal on Scientific Computing, a variant of the ELL (ELLPACK) format. The SELL format has two parameters, C and σ. C represents the number of vector columns, which is determined by the hardware; σ is the sorting window size, which can be set in the code. The core idea of ​​SELL is to sort each σ row by number of elements from largest to smallest, and then arrange the non-zero elements of each C row into a vector by column, just like the ELL format. This avoids the short vector problem and improves the density and utilization of vector computations. However, to fill the vector, SELL requires some zero elements, introducing an average of up to 20% of zero elements. This increases the storage capacity and memory bandwidth of the sparse matrix, reducing the performance of SpMV.

[0050] Based on this, the applicant of this application proposed a sparse matrix storage method for storing sparse matrices for vector calculations in the memory of a computing device and a vector calculation method performed by the corresponding computing device, which can ensure that "all columns of a vector register require the same number of multiplication and addition operations" through regular arrangement, avoiding the conditional branch instructions introduced in the above-mentioned CVR, CSR5, VHCC and other formats in order to "determine whether certain vector lanes of the current vector register are the last elements of a row and therefore the scalar data of these columns need to be saved in memory in sequence" and the operation of "extracting data from a certain column from the vector register and saving it to the scalar register".

[0051] In addition, the sparse matrix storage method and corresponding vector calculation method proposed by the applicant of this application can simplify the addition process of "reducing and summing the multiplication results of a row", that is, avoiding the reduction operation on a vector register (as mentioned above, VHCC fully adopts this method, and CSR5 partially adopts this method), and changing it to "different columns of the vector register store the accumulation results of different rows, and perform the accumulation process of N rows simultaneously during vector multiplication and addition". This is because in the sparse matrix storage method for vector calculation and the corresponding vector calculation method of this application, the non-zero elements are transposed before being saved, and the transposition operation also enables the subsequent use of the vector instruction scatter to save the calculation results to the memory.

[0052] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0053] Schematic method

[0054] Regular arrangement

[0055] Figure 1 The flowchart of a sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device according to an embodiment of the present application is illustrated.

[0056] like Figure 1 As shown, a sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device according to an embodiment of the present application includes the following steps.

[0057] Step S110, determining the number of columns of the vector register used for vector calculation. Here, the sparse matrix used for vector calculation is stored in the memory of the computing device, and the memory can be a volatile memory or a non-volatile memory, such as a disk of the computing device. In addition, when the computing device performs vector calculation, it needs to store the vector in a register. Therefore, the register used for vector calculation is also called a vector register, which corresponds to the scalar register and is an ordinary register of the computing device. In addition, those skilled in the art will understand that the computing device may also include special registers, but special registers are generally used for internal control of hardware, not for calculation. Ordinary registers are used for calculation.

[0058] Here, the vector register can be assumed to be N, where N depends on the hardware used for vector calculations. Specifically, considering the double-precision (double) data type, the hardware modules for vector calculations of different processors will have different numerical values ​​N, such as N of the Intel AVX256 instruction set is 4, N of the Intel AVX512 instruction set is 8, and ARM's SVE instruction set can support up to 2048 bits, that is, N = 32. The number of threads of a warp on the GPU is 32, that is, N = 32. Therefore, in the embodiment of the present application, the specific value of N is not limited, but the storage format according to the embodiment of the present application can be applied to various different numerical values ​​N to adapt to different vector calculation hardware platforms and increase flexibility.

[0059] Step S120: Determine the number of non-zero elements in each row of the sparse matrix. For example, the number of non-zero elements in each row of the sparse matrix is ​​defined as num[i]. Then, based on the number of non-zero elements in each row, a regular arrangement is performed in the sparse matrix storage method for vector calculation according to an embodiment of the present application.

[0060] Step S130, based on the number of non-zero elements in each row of the sparse matrix and the number of columns of the vector register used for vector calculation, determine the group of the sparse matrix including multiple matrix rows, wherein the difference between the number of non-zero elements of each matrix row in the multiple matrix rows in each group is less than a first threshold value, so that the multiple columns of the vector register are filled, and all columns of the vector register require the same or similar number of multiplication and addition operations that meet a predetermined standard. Here, in an embodiment of the present application, determining the group of the sparse matrix including multiple matrix rows, and the difference between the number of non-zero elements of each matrix row in the multiple matrix rows in each group is less than a first threshold value can be referred to as "regular arrangement".

[0061] here, Figure 2 The figure shows a schematic diagram of regular arrangement in the sparse matrix storage method according to an embodiment of the present application.

[0062] like Figure 2 As shown, in the sparse matrix storage method according to the embodiment of the present application, in order to avoid the inability to fill multiple columns (vector lanes) of a vector register (that is, the short vector problem), a regularly arranged format as defined in the embodiment of the present application is adopted. Specifically, N rows with the same or similar number of non-zero elements are selected, and the i-th element of the j-th row is arranged in the j-th column (vector lane) of the i-th vector register. For example, Figure 2The case of length = 3 shown on the right side of selects four rows with similar numbers of nonzero elements: the 7th, 5th, 9th, and 4th rows of the original matrix. The first element (g) of the 9th row of the original matrix is ​​placed in the second column of the first vector register, the second element (h) in the second column of the second vector register, and the third element (i) in the second column of the third vector register. The same arrangement applies to the remaining rows.

[0063] Here, the number of columns of the vector register is N, so the number of rows of the matrix for regular arrangement in each group is also N. And, as Figure 2 As shown, when determining a group including a plurality of matrix rows, the number of non-zero elements in the plurality of matrix rows in the group may be the same, for example, Figure 2 In addition, the number of non-zero elements in the multiple matrix rows in the group may not be exactly the same, then it is necessary to add zero elements, for example Figure 2 In the case of length = 3 shown on the right, the added zero elements are represented by white squares.

[0064] Furthermore, in an embodiment of the present application, in order to control the number of zero elements filled in as described above, the difference in the number of non-zero elements in a group can be limited to not exceed a first threshold value T1. Here, the larger the value of the first threshold value T1, the more rows (and corresponding non-zero elements) that can be found that are suitable for regular arrangement, which can increase the regularity of the calculation, but will increase the number of zero elements that need to be filled in, increase the amount of memory access during calculation, and reduce the overall computing performance.

[0065] Therefore, in the sparse matrix storage method for vector calculation according to an embodiment of the present application, the difference between the numbers of non-zero elements of each matrix row in the plurality of matrix rows in each group is less than the first threshold T1.

[0066] Figure 3 The figure shows the effect of different first thresholds T1 on the number of zero elements filled in different sparse matrices in the sparse matrix storage method according to an embodiment of the present application. The horizontal axis is different sparse matrices, each sparse matrix uses 6 parameters from T1=0 to T1=5, and the vertical axis is the percentage of the number of regularly arranged non-zero elements (no filling), other non-zero elements (shaded) and filled zero elements (black filling). Figure 3It can be seen that the larger T1 is, the greater the percentage of regularly arranged non-zero elements (no fill) and filled zero elements (black fill), which is consistent with the above analysis. At the same time, when T1 is greater than or equal to 3, the percentage of regularly arranged non-zero elements (no fill) slowly increases with the increase of T1. Therefore, in the embodiment of the present application as described above, T1=3 is adopted. Here, those skilled in the art will understand that for other sparse matrices and the number of columns of the vector register, corresponding appropriate T1 values ​​can also be selected. The first threshold T1 can be preset by the inventor based on practical experience, or, for example, can be determined by learning using a machine learning method.

[0067] Step S140, the non-zero elements in each group corresponding to the same matrix row are stored in the memory by column, so that the result of the vector calculation can be saved to the memory using the vector save instruction. Figure 2 As shown, through regular arrangement, each group, for example, matrix rows of length = 1 and length = 3, can be stored in columns corresponding to the columns of the vector register, that is, the non-zero elements corresponding to the same matrix row in each group are stored in the memory in columns respectively, so that the results of the vector calculation can be saved to the memory using the vector save instruction.

[0068] In this way, the sparse matrix storage method according to the embodiment of the present application can ensure that "all columns of a vector register require the same number of multiplication and addition operations" through regular arrangement, avoiding the conditional branch instructions introduced in the above-mentioned CVR, CSR5, VHCC and other formats in order to "determine whether certain vector lanes of the current vector register are the last elements of a row and therefore the scalar data of these columns need to be saved in memory in sequence" and the operation of "extracting data from a certain column from the vector register and saving it to the scalar register".

[0069] In addition, according to the sparse matrix storage method of the embodiment of the present application, the non-zero elements can be transposed before being saved, wherein the transposition operation enables the subsequent use of the vector instruction scatter to save the calculation results to the memory, thereby simplifying the addition process of "reduction and summation of the multiplication results of a row", that is, avoiding the reduction operation of a vector register (as mentioned above, VHCC completely adopts this method, and CSR5 partially adopts this method), and changing it to "different columns of the vector register save the accumulated results of different rows, and the accumulation process of N rows is performed simultaneously during vector multiplication and addition".

[0070] Here, in accordance with the embodiment of the present application, in order to determine more groups containing multiple matrix rows while satisfying the constraint of the difference T1 between the non-zero elements of each row in the group, each row of the sparse matrix can be first sorted from large to small according to the number of non-zero elements, such as Figure 2 Then use a dynamic programming algorithm to find the rows that fit the regular arrangement, that is, Figure 2 Finally, put the non-zero elements of each row into Figure 2 The right side of the form is rearranged and saved to memory. In addition, note that Figure 2 The first and last rows in the middle of the are not included in the regular arrangement, so these two rows are Figure 2 The right side needs to be arranged separately, that is Figure 2 on the right side of "Other".

[0071] Therefore, in the sparse matrix storage method for vector calculation according to an embodiment of the present application, it further includes: determining the remaining ungrouped matrix rows of the sparse matrix; and storing the non-zero elements in the remaining matrix rows in the memory in rows based on the number of columns of the vector register in units of each matrix row.

[0072] Figure 4 FIG2 shows a schematic pseudo code diagram of a dynamic programming algorithm in a sparse matrix storage method according to an embodiment of the present application. Figure 4 As shown, the dynamic programming algorithm takes as input the number of nonzero elements num[i] in each row and outputs a regular arrangement of rows that are grouped. In the output, f[i] refers to the largest number of groups in the first i rows, g[i] refers to the smallest number of nonzero elements (including padding zeros) in the first i rows, and h[i] == 0 indicates that row i is not grouped. Line 1 of the pseudocode first sorts the num array from largest to smallest and sets h[i] = 0 (initializing all rows to be grouped). Line 2 initializes the dynamic programming, implementing special handling for rows less than N. Line 3 iterates over all rows, deciding for each row whether to group them with the preceding N-1 rows. Lines 4 through 9 determine if the difference between row i and the preceding N-1 rows is no greater than T1 and if grouping reduces either f[i] or g[i], then grouping is chosen and h[i] is set to 1. Lines 10 to 3 traverse all rows in reverse order, splicing N consecutive rows marked as "need to be spliced", and if the number of non-zero elements in N rows is different, they need to be filled with zeros.

[0073] Two-dimensional partitioning

[0074] During the storage process of the sparse matrix, in order to improve the cache hit rate, in the sparse matrix storage method according to the embodiment of the present application, adjustable two-dimensional partitioning is further performed.

[0075] In 2009, Aydin et al. proposed the CSB format (SPAA2009) at the SPAA International Conference. This format partitions sparse matrices into fixed-size, two-dimensional partitions to improve cache hit rates. The following explains how two-dimensional partitioning of sparse matrices can improve cache hit rates.

[0076] Figure 5 The figure shows the pseudocode for SpMV in CSR format (without two-dimensional partitioning). Line 4 of the pseudocode accesses the data and col arrays sequentially, but the data cannot be reused. Access to the x array is random, that is, accesses of the form x[b[i]]. In the case of two-dimensional partitioning, different col[j] values ​​are likely to have the same value. Therefore, the x array values ​​x[col[j]] corresponding to different j values ​​are likely to be reused in the cache, improving the cache hit rate of the x array. This is the benefit of two-dimensional partitioning.

[0077] However, the fixed-size two-dimensional partitioning may result in too few elements in some blocks, resulting in too few opportunities for x array reuse and an inability to further improve the cache hit rate. In addition, too few elements in the blocks are also not conducive to the regular arrangement in the sparse matrix storage method for vector calculation in the embodiment of the present application. Therefore, the applicant of the present application believes that different sparse matrices require different partitioning sizes, and even in different areas of the same sparse matrix, the distribution of non-zero elements may be uneven, and different partitioning sizes are also required.

[0078] Specifically, in the sparse matrix storage method for vector calculation according to the embodiment of the present application, an adjustable two-dimensional partitioning method is further performed. First, the sparse matrix is ​​divided into multiple blocks (chunks) in two dimensions using a fixed size P*Q, and then several horizontally continuous blocks that meet preset conditions are merged into a large block.

[0079] For example, in the embodiment of the present application, the preset conditions may be the following two points:

[0080] 1. The number of non-zero elements in all blocks (including the merged large blocks) does not exceed the preset threshold T2;

[0081] 2. The distance between blocks (that is, the difference between the first column of the leftmost block and the last column of the rightmost block) is not greater than the compression requirement.

[0082] Therefore, in the sparse matrix storage method according to an embodiment of the present application, before determining the number of non-zero elements in each row of the sparse matrix, it further includes: obtaining an initial matrix to be divided; dividing the initial matrix to be divided into multiple blocks in two dimensions based on a fixed size; and merging multiple horizontally continuous blocks that meet preset conditions into large blocks, wherein the large blocks serve as the sparse matrix to improve the hit rate of each row of the sparse matrix in the cache.

[0083] Furthermore, in the above-mentioned sparse matrix storage method for vector calculation, the preset conditions include: the number of non-zero elements in all blocks and the merged large blocks does not exceed a second threshold; and / or the distance between the blocks is less than or equal to a compression threshold, wherein the block distance is the number of horizontal blocks included in the large block, and the compression threshold ensures that all data of the input and output vectors corresponding to the blocks are stored in the cache.

[0084] Figure 6 FIG. 1 is a schematic diagram illustrating an example of block merging in a sparse matrix storage method according to an embodiment of the present application. Figure 6 As shown in , each square represents a block, the number in the block represents the number of non-zero elements in the block, and a blank represents no non-zero elements. Assuming the threshold T2 = 20, and the compression requirement is that the maximum span of the left and right blocks does not exceed 4 blocks, then Figure 6 The figure shows an optimal merging solution that minimizes the number of blocks (a total of 5). Large block 1 cannot include the 5th block (the number of nonzero elements is 1) because it cannot violate the rule that the maximum span between left and right blocks does not exceed 4 blocks. Furthermore, large block 3 cannot include the subsequent blocks (the number of nonzero elements is 2) because it cannot violate the rule that the total number of nonzero elements in the large blocks does not exceed the threshold T2 = 20. Thus, through block merging, the original 10 blocks are reduced to 5 large blocks.

[0085] The advantage of block merging is that it increases the number of nonzero elements within a block, facilitating subsequent regular arrangement, and improves data locality in the output vector, reducing write operations to the output vector (one block requires a single write operation to the output vector at the corresponding position). Furthermore, the preset threshold T2 prevents the input and output vector data corresponding to a block from not being fully cached due to the block size being too large.

[0086] In addition, in the block merging process described above, the selection of the fixed size P*Q and the threshold T2 is related to the size of the CPU cache: the larger the cache, the more input and output vector data it can accommodate (data is reused), and the more non-zero elements in a block can be (data is not reused), and the larger the fixed size P*Q and T2 can be.

[0087] Figure 7 FIG is a schematic diagram of a pseudo code of a block merging process in a sparse matrix storage method according to an embodiment of the present application. Figure 7 As shown, the inputs to the block merging process are the number of nonzero elements num[i] in each block and the second threshold T2. The output is the block merging strategy. The output array f[i] stores the minimum number of large blocks obtained by merging the previous i blocks. Line 1 of the pseudocode initializes the dynamic programming. Line 2 iterates through all blocks. Each block has two strategies: not merging with the previous block and becoming a large block on its own (line 3), or merging with several consecutive blocks. Line 4 enumerates all possible starting merging ranges j, but these must meet three conditions (line 5): 1. The number of nonzero elements in the previous consecutive blocks does not exceed T2; 2. The number of nonzero elements in the i-th block does not exceed T2; and 3. The distance between i and j is not too far. Line 6 selects the minimum number of large blocks among all possible merging scenarios. Line 9 exits the for loop early to speed up the dynamic programming process without affecting the calculation results. After calculating the f[i] array, lines 10-13 work backwards to get the optimal merging strategy. Figure 7 The core idea of ​​the pseudocode shown is that if f[i] == f[j], then blocks i and j belong to the same large block.

[0088] Furthermore, after the two-dimensional partitioning, the row and column numbers of the non-zero elements can be renumbered starting from 0 and concatenated into a 32-bit integer. In this way, during vector calculations, the row and column numbers can be obtained using the bitwise operation "and" and the logical right shift operation "srl", thereby compressing the two int types (8 bytes in total) originally used to store the row and column numbers into a single int type (4 bytes).

[0089] In addition, if the column numbers of all non-zero elements in the block do not exceed 2^16=65536, then only 2 bytes can be used to store these column numbers, reducing the amount of data transferred from the memory to the CPU during the calculation process. Specifically, in the sparse matrix storage method for vector calculation according to an embodiment of the present application, the numerical value of the non-zero element (double type, 8 bytes) and the corresponding column number (int type, 4 bytes) can be saved for each non-zero element, just like the original storage format such as CSR and COO. If the int type of the column number is replaced with a short type (2 bytes), the storage space and memory transfer amount of each non-zero element can be compressed from 12 bytes to 10 bytes. However, this requires that in the block merging process as described above, not only the number of blocks within the large block is limited, but also the constraint that "the column number of all non-zero elements in the large block does not exceed 2^16=65536" is introduced. In other words, the distance between the blocks is not greater than the compression requirement (i.e., the difference between the first column of the leftmost block and the last column of the rightmost block does not exceed 2^16=65536).

[0090] Therefore, in the sparse matrix storage method according to an embodiment of the present application, after merging multiple horizontally continuous blocks that meet preset conditions into large blocks as a sparse matrix, it further includes: renumbering the row numbers and column numbers of the non-zero elements of the sparse matrix from zero, and splicing them into 32-bit integers, so that during vector calculation, the row numbers and column numbers are obtained by bit operations and logical right shift operations, and the row numbers and column numbers are stored in four bytes of an integer type.

[0091] Furthermore, in the above-mentioned sparse matrix storage method for vector calculation, the preset condition further includes: the column distance between the first column of the block at one end of the large block and the last column of the block at the other end of the large block is less than or equal to 65536, so that the column number is stored in two bytes of short type.

[0092] In this way, the index compression process can reduce the storage and memory access of the sparse matrix. In a multi-threaded environment, memory bandwidth is the bottleneck that restricts the performance of SpMV. Therefore, reducing memory access can alleviate this bottleneck and improve performance.

[0093] Load Balancing

[0094] In a multi-core (or multi-threaded) environment, different processor cores (or threads) can compute different regions of a sparse matrix in parallel. Distributing the entire sparse matrix evenly across cores based on the number of nonzero elements is a straightforward load balancing strategy. However, because different regions of a sparse matrix have varying numbers of nonzero elements, simple equal distribution results in significant inequality in the number of storage operations, leading to uneven load and impacting the overall performance of SpMV. Figure 8 The figure shows the statistics of the execution time of different threads and the number of storage instructions (scatterinstruction) of the sparse matrix flickr under 28 threads (both have been normalized). Figure 8 As can be seen from the graph, the execution time and number of store instructions vary significantly between threads, while the trends are generally consistent. The number of nonzero elements is the same across threads, meaning the computational effort is the same. Therefore, we can infer that the difference in the number of store instructions leads to the different execution times.

[0095] Therefore, in an embodiment of the present application, when considering load balancing, the number of non-zero elements and the number of storage operations can be considered at the same time, and a reasonable allocation plan can be obtained by using methods such as machine learning. Figure 9 FIGURE 1 illustrates an example of load balancing in a sparse matrix storage method for vector calculation according to an embodiment of the present application. Figure 9 As shown in the figure, taking three threads as an example, if the entire sparse matrix is ​​evenly distributed according to the number of non-zero elements, without considering the number of storage operations, then the three threads are allocated the same number of non-zero elements (7 in the figure), but the storage operations of the calculation results required by the three threads (that is, the number of rows of non-zero elements in the sparse matrix) are 2, 4, and 5 times respectively. In large-scale matrices, the number of storage operations of different threads may vary greatly, which in turn affects the performance of the final SpMV. If the threads with fewer storage operations (such as Figure 9 Thread 1), using the weights obtained by machine learning, assign more non-zero elements, then the possible non-zero element allocation scheme is as follows Figure 9 As shown in the lower right corner of . The three threads require 3, 4, and 4 storage operations respectively.

[0096] In this way, after the load balancing distribution scheme is determined, each CPU thread will be assigned a continuous area of ​​non-zero elements, and the distribution of non-zero elements between different threads is continuous and non-overlapping, such as Figure 10 shown. Figure 10 The diagram shows a schematic diagram of non-zero elements allocated to different threads in a multi-threaded environment in a sparse matrix storage method according to an embodiment of the present application. Figure 10 As shown in the figure, to achieve load balancing, different threads may have different numbers of nonzero elements. Only the first and last rows of each thread may share the same sparse matrix row with adjacent threads. Therefore, storing the results of the first and last rows of each thread requires atomic operations (or serial operations). The results of other rows within the thread are exclusive and do not require atomic operations when storing the results.

[0097] In this way, after assigning all non-zero elements of the sparse matrix to different CPU threads, each thread then performs the "two-dimensional partitioning" and "regular arrangement" of its assigned non-zero elements as described above. The non-zero elements operated on by different threads do not overlap or affect each other, and the operations are identical, allowing OpenMP parallel processing on the CPU.

[0098] Therefore, in the sparse matrix storage method according to an embodiment of the present application, obtaining the initial matrix to be divided includes: obtaining the initial matrix to be divided allocated to each thread of the CPU, so that each thread has the same or similar storage operands.

[0099] Vector calculations

[0100] As described above, in an embodiment of the present application, a sparse matrix storage method performed by a computing device is provided, which is a storage format for a coefficient matrix for SpMV calculation in a vector computing scenario (such as Intel's AVX512 instruction set, ARM's SVE instruction set, SIMT computing mode on a GPU, etc.), which may include the regular arrangement of the sparse matrix as described above, and may further include two-dimensional partitioning and index compression of the sparse matrix. Moreover, by using, for example, a machine learning method, different parameters can be selected for different sparse matrices and different hardware conditions, thereby achieving optimal performance of the SpMV calculation.

[0101] Figure 11 FIG2 is a schematic diagram of the overall calculation process of SpMV according to an embodiment of the present application. Figure 11 As shown, first, the sparse matrix to be calculated is preprocessed based on the sparse matrix storage method described above, and the SpMV is calculated after the preprocessing is completed.

[0102] As described above, two-dimensional partitioning preprocessing can increase data locality in SpMV calculations, improving cache hit rates. Furthermore, regular arrangement preprocessing can avoid conditional branch instructions in SpMV calculations, simplify addition calculations, and use vector operations to store calculation results, accelerating SpMV calculations. Furthermore, index compression preprocessing can reduce memory accesses and improve SpMV performance.

[0103] Figure 12 The figure illustrates a schematic flowchart of a vector calculation method executed by a computing device according to an embodiment of the present application.

[0104] like Figure 12 As shown, the vector calculation method performed by the computing device according to an embodiment of the present application includes the following steps.

[0105] Step S210: Obtain a sparse matrix stored in a memory of a computing device according to the sparse matrix storage method for storing a sparse matrix for vector calculations as described above. That is, after preprocessing using the sparse matrix storage method as described above, SPMV calculation can be performed based on the stored sparse matrix.

[0106] Step S220, performing vector calculation using the non-zero elements of the sparse matrix stored in the memory by column, corresponding to the group including multiple matrix rows. That is, performing vector calculation with the elements arranged regularly in the sparse matrix storage method as described above. This corresponds to Figure 13 Pseudocode: Line 1 iterates over all groups. Line 2 gets the length of the non-zero elements of the group. Line 3 clears the vector accumulator. Line 4 iterates over the N non-zero elements of the group. Line 5 loads the value of the non-zero element and the corresponding column number. Line 6 loads the corresponding input vector. Line 7 performs vector multiplication and addition operations. Line 8 loads the row number of the corresponding output vector (if the row and column compression strategy is adopted, the row number is obtained from the 32-bit integer of the column number, and no memory access is required). Line 9 saves the result of the calculated output vector to memory. Here, Figure 13 The figure illustrates a pseudo code diagram of the calculation process of regularly arranged elements in the vector calculation method according to an embodiment of the present application.

[0107] As described above, in the sparse matrix storage method according to the embodiment of the present application, the non-zero elements are transposed before being saved, and the transposition operation enables the subsequent use of the vector instruction scatter to save the calculation results to the memory. In this way, the addition process of "reduction and summation of the multiplication results of a row" is simplified, that is, the reduction operation of a vector register is avoided (VHCC completely adopts this method, and CSR5 partially adopts this method), and it is changed to "different columns of the vector register save the accumulation results of different rows, and the accumulation process of N rows is performed simultaneously during vector multiplication and addition".

[0108] For example, in Figure 2In the case of length = 3 shown in the middle, the four columns of the vector register correspond to the 7th, 5th, 9th, and 4th rows of the original matrix, respectively. After the first vector multiplication and addition, the four columns of the vector register are the multiplication and addition results of the four non-zero elements "d, a, g, 7". After the second vector multiplication and addition, the four columns of the vector register are the multiplication and addition results of the eight non-zero elements "d+e, a+b, g+h, 7+8". After the third vector multiplication and addition, the four columns of the vector register are the multiplication and addition results of the 12 non-zero elements "d+e+empty, a+b+empty, g+h+i, 7+8+9", where "empty" represents a zero element. Finally, the vector instruction scatter can be used to save the entire vector register to memory.

[0109] Step S230, performing vector calculation using the non-zero elements of the sparse matrix stored in the memory by column, corresponding to the group including multiple matrix rows. That is, performing vector calculation using the irregularly arranged elements in the sparse matrix storage method as described above. This can be done by using the traditional "row-by-row vector calculation" method to calculate SpMV, corresponding to Figure 14 Pseudocode. Line 1 iterates over all unjoined rows. Line 2 clears the vector accumulator. Line 3 iterates over all elements of the row in the form of a vector. Line 4 loads the value of the non-zero element and the corresponding column number, and loads the corresponding input vector by the column number. Line 5 performs vector multiplication and addition. Line 6 reduces the calculation result from the vector to a scalar and saves it to memory. Here, Figure 14 The figure illustrates a pseudo code diagram of the calculation process of irregularly arranged elements in the vector calculation method according to an embodiment of the present application.

[0110] Furthermore, if multithreading is used, each thread can calculate the SpMV for the corresponding region in parallel. Furthermore, if load balancing is implemented, each thread must calculate the SpMV for the first and last rows of its assigned continuous region and temporarily store the results (one scalar per row, for a total of two scalars) in the array buf. After all threads have completed their calculations, the values ​​in the array buf are serially added to the corresponding positions in the output vector, avoiding atomic operations.

[0111] That is, in the vector calculation method according to the embodiment of the present application, it further includes: each thread calculates the sparse matrix-vector multiplication operation corresponding to the first and last rows of the continuous area allocated to it; temporarily stores the calculation results in the array buffer; and, after all threads complete the calculation, serially accumulates the values ​​in the array buffer to the corresponding positions of the output vector.

[0112] The vector calculation method according to the embodiment of the present application can improve the performance of sparse matrix-vector multiplication (SpMV), thereby accelerating the performance of important algorithms in high-performance fields such as PageRank and conjugate gradient.

[0113] Experiments show that on an Intel Xeon processor (Skylake architecture), among the 30 irregular sparse matrices in the Florida sparse matrix library, the average performance of SpMV calculation according to the vector calculation method of the embodiment of the present application is 1.93 times that of CVR (up to 2.93 times), and 1.4 times that of CSR5 (up to 2.19 times); in the other 16 regular sparse matrices, the average performance of SpMV calculation according to the vector calculation method of the embodiment of the present application is 1.89 times that of CVR (up to 2.85 times), and 1.34 times that of CSR5 (up to 2.17 times).

[0114] In summary, the sparse matrix storage method for storing sparse matrices for vector calculation in the memory of a computing device according to an embodiment of the present application and the vector calculation method performed by the corresponding computing device can increase data locality and improve cache hit rate through two-dimensional partitioning; simplify the calculation process through regular arrangement, reduce the amount of data transmission in the memory through row and column compression, and obtain a reasonable load balancing distribution scheme and other key parameters through machine learning. For example, in an embodiment of the present application, several parameters can be set to adapt to different sparse matrices, including (1) different numbers of threads; (2) the size of the two-dimensional partitioning; (3) the threshold T2 for block merging; (4) the parameter T1 of the regular arrangement. Through these modifications, the optimal performance of SpMV can be achieved in different sparse matrices, increasing flexibility and adaptability.

[0115] Schematic electronic equipment

[0116] Below, reference Figure 15 To describe the electronic device according to the embodiment of the present application.

[0117] Figure 15 The figure shows a block diagram of an electronic device according to an embodiment of the present application.

[0118] like Figure 15 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0119] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0120] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the sparse matrix storage method and vector calculation method for vector calculation of the various embodiments of the present application described above and / or other desired functions. Various contents such as non-zero elements and control parameters may also be stored in the computer-readable storage medium.

[0121] In one example, the electronic device 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0122] For example, the input device 13 may be a keyboard, a mouse, or the like.

[0123] The output device 14 can output various information to the outside, including calculation results of sparse matrix-vector multiplication, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0124] Of course, to simplify, Figure 15 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 10 may further include any other appropriate components according to specific application scenarios.

[0125] Exemplary computer program product

[0126] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the sparse matrix storage method and vector calculation method for vector calculation according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0127] The computer program product can be written in one or more hardware description languages ​​used on FPGA platforms to write program code for performing the operations of the embodiments of the present application, such as Verilog / VHDL. In addition, Xilinx FPGAs also support FPGA development using C or C++. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0128] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the sparse matrix storage method and vector calculation method for vector calculation according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0129] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0130] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0131] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0132] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0133] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0134] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device, characterized in that: include: determining the number of columns of a vector register used for vector calculations in a computing device; Determining the number of non-zero elements in each row of the sparse matrix; Determining, based on the number of non-zero elements in each row of the sparse matrix and the number of columns of the vector register for vector calculation, groups of the sparse matrix including a plurality of matrix rows, wherein a difference between the numbers of non-zero elements of respective matrix rows of the plurality of matrix rows in each group is less than a first threshold, such that the plurality of columns of the vector register are filled and all columns of the vector register require the same or a similar number of multiplication-addition operations that meet a predetermined standard; and The non-zero elements in each group corresponding to the same matrix row are stored in the memory by column, respectively, so that the result of the vector calculation can be saved to the memory using a vector save instruction.

2. The sparse matrix storage method according to claim 1, further comprising: Determining remaining ungrouped matrix rows of the sparse matrix; as well as, The non-zero elements in the remaining matrix rows are stored in the memory in rows based on the number of columns of the vector register and in units of each matrix row.

3. The sparse matrix storage method according to claim 1, wherein: Before determining the number of non-zero elements in each row of the sparse matrix, the method further includes: Get the initial matrix to be divided; Dividing the initial matrix to be divided into a plurality of blocks in two dimensions based on a fixed size; and A plurality of horizontally continuous blocks that meet preset conditions are merged into a large block, wherein the large block serves as the sparse matrix to improve the hit rate of each row of the sparse matrix in the cache.

4. The sparse matrix storage method according to claim 3, wherein: The preset conditions include: The number of non-zero elements in all blocks and the merged large block does not exceed a second threshold; and / or, The distance of the blocks is less than or equal to a compression threshold, wherein the block distance is the number of horizontal blocks included in the large block, and the compression threshold enables all data of the input and output vectors corresponding to the blocks to be stored in the cache.

5. The sparse matrix storage method according to claim 3, wherein: After merging a plurality of horizontally continuous blocks that meet preset conditions into a large block as a sparse matrix, the method further includes: The row numbers and column numbers of the non-zero elements of the sparse matrix are renumbered starting from zero and concatenated into 32-bit integers, so that the row numbers and column numbers can be obtained by bit operations and logical right shift operations during vector calculations, and the row numbers and column numbers are stored in four bytes of an integer type.

6. The sparse matrix storage method according to claim 4, wherein: The preset conditions further include: The column distance between the first column of the block at one end of the large block and the last column of the block at the other end of the large block is less than or equal to 65536, so that the column number is stored in two bytes of short type.

7. The sparse matrix storage method according to claim 3, wherein: Obtaining the initial matrix to be divided includes: The initial matrix to be divided allocated to each thread of the CPU is obtained so that each thread has the same or similar number of storage operations.

8. The sparse matrix storage method according to claim 1, wherein the first threshold is determined by a machine learning method.

9. A vector calculation method executed by a computing device, characterized in that: include: Obtaining a sparse matrix stored by the sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device according to any one of claims 1 to 8; performing vector calculations using non-zero elements of the sparse matrix stored in the memory by column corresponding to the group including the plurality of matrix rows; and Vector calculations are performed using non-zero elements of the sparse matrix that are not stored in the memory by column and do not correspond to the group including the plurality of matrix rows.

10. The vector calculation method according to claim 9, further comprising: Each thread calculates the sparse matrix-vector multiplication operation corresponding to the first and last two rows of the assigned continuous area; Temporarily store the calculation results in the array buffer; as well as, After all threads have finished calculating, the values ​​in the array buffer are serially accumulated to the corresponding positions of the output vector.

11. An electronic device comprising: processor; as well as, A memory having computer program instructions stored therein, wherein the computer program instructions, when executed by the processor, cause the processor to execute a sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device as claimed in any one of claims 1 to 8 or a vector calculation method executed by a computing device as claimed in claim 9 or 10.

12. A computer-readable storage medium having computer program instructions stored thereon, which, when executed by a computing device, can be operated to execute a sparse matrix storage method for storing a sparse matrix for vector calculation in a memory of a computing device as claimed in any one of claims 1 to 8, or a vector calculation method executed by a computing device as claimed in claim 9 or 10.

Images