Matrix compression method, matrix processing method, and related device

Abstract

The present application relates to the field of data processing, and discloses a matrix compression method, a matrix processing method, and a related device. The matrix compression method comprises: dividing a first matrix stored in a memory to obtain a plurality of first sub-matrices; compressing a non-zero element group in each of the plurality of first sub-matrices in a first direction to obtain a plurality of second sub-matrices corresponding to the first matrix; dividing each of the plurality of second sub-matrices to obtain a plurality of third sub-matrices; and generating a compression result of the first matrix on the basis of the plurality of third sub-matrices. The present application enables only the non-zero element groups to exist in the first sub-matrices, thereby achieving removal of a portion of zero-valued elements in the first matrix, and therefore, the determined compression result of the first matrix can effectively reduce the number of zero elements that need to be stored, thereby saving data storage space.

Description

Matrix compression methods, matrix processing methods, and related devices

[0001] This application claims priority to Chinese Patent Application No. 202510060130.2, filed on January 14, 2025, entitled "Matrix Compression Method, Matrix Processing Method and Related Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing, and in particular to a matrix compression method, a matrix processing method, and related apparatus. Background Technology

[0003] In a matrix, if the number of zero elements far exceeds the number of non-zero elements, and the distribution of non-zero elements is irregular, then the matrix is ​​called a sparse matrix. Since many large-scale datasets and mathematical models can be represented as sparse matrices in practical applications, sparse matrices are widely used in machine learning, high-performance computing (HPC) scientific computing, search and recommendation, graph neural networks, and large sparse models in artificial intelligence (AI).

[0004] Since sparse matrices have fewer non-zero elements and more zero elements, using a conventional two-dimensional array to store them would waste a lot of storage space to store the zero elements. Therefore, how to compress sparse matrices is a current research hotspot. Summary of the Invention

[0005] This application provides a matrix compression method, a matrix processing method, and related apparatus, which can compress sparse matrices. The technical solution is as follows:

[0006] In a first aspect, a matrix compression method is provided, applied in a computer system, the computer system including a first processor and a memory, the method being executed by the first processor, comprising: dividing a first matrix stored in the memory to obtain multiple first sub-matrices, the first matrix being input information or intermediate results in a model training or inference process; compressing the non-zero element group in each of the multiple first sub-matrices along a first direction to obtain multiple second sub-matrices corresponding to the first matrix, the non-zero element group being a vector in the first sub-matrices along a second direction, and the non-zero element group containing elements with non-zero values; dividing each of the multiple second sub-matrices to obtain multiple third sub-matrices; and generating a compression result of the first matrix based on the multiple third sub-matrices, the compression result indicating the values ​​of non-zero elements in all third sub-matrices and the positions of the non-zero elements in the corresponding third sub-matrices.

[0007] This application can first divide the first matrix to be compressed into smaller sub-matrices, and then compress the non-zero elements in the resulting sub-matrices to obtain a second sub-matrix. This ensures that the second sub-matrix contains only non-zero elements and no all-zero elements, effectively eliminating some zero-valued elements from the first matrix. In this case, based on multiple third sub-matrices, the compression result of the first matrix can effectively reduce the number of zero elements that need to be stored, thus saving data storage space. Furthermore, in scenarios requiring matrix multiplication, the matrix can be divided according to computational needs to obtain sub-matrices of suitable size for computation, thereby effectively improving computational efficiency.

[0008] It should be noted that the input information or intermediate results in the model training or inference process are usually in the form of tensors. A tensor is a multidimensional array that can represent multidimensional data and complex multilinear relationships. The order (or rank) of a tensor refers to the number of dimensions represented by the tensor, including zero-order tensors (i.e., scalars), first-order tensors (i.e., vectors), second-order tensors (i.e., matrices), and third-order and higher-order tensors. Therefore, the input information or intermediate results in the model training or inference process in this application can be second-order tensors (i.e., matrices) or third-order and higher-order tensors. When the input information or intermediate results in the model training or inference process are second-order tensors, the first matrix is ​​the second-order tensor. When the input information or intermediate results in the model training or inference process are third-order and higher-order tensors, slicing operations (i.e., slicing the tensor according to specific dimensions), reshaping operations (i.e., reshaping the high-dimensional tensor into a low-dimensional matrix), etc., can be performed on the third-order and higher-order tensors to obtain at least one matrix. In this case, any one of the at least one matrices is determined as the first matrix.

[0009] The first matrix in this application is a sparse matrix, which is a matrix in which the number of elements with a value of 0 is far greater than the number of non-zero elements, and the distribution of non-zero elements is irregular. In one possible implementation, the matrix is ​​determined to be a sparse matrix if its density is less than or equal to a sparsity threshold. Here, density refers to the total number of non-zero elements in the matrix divided by the total number of elements in the matrix.

[0010] The process of dividing the first matrix stored in memory to obtain multiple first sub-matrices includes: dividing the first matrix in a second direction to obtain multiple first sub-matrices. The second direction can be either a row direction or a column direction; if the second direction is a row direction, the first direction is a column direction; if the second direction is a column direction, the first direction is a row direction.

[0011] Since the second direction can be either row-oriented or column-oriented, the process of dividing the first matrix into multiple first sub-matrices in the second direction varies depending on the circumstances. These will be described separately below.

[0012] In the first case, the second direction is the column direction. Dividing the first matrix in the column direction means dividing the first matrix into rows. In this case, the first value is used as the step size to divide the rows of the first matrix, thereby obtaining multiple first sub-matrices. The number of rows of each first sub-matrice is the first value, the number of columns of each first sub-matrice is equal to the number of columns of the first matrix, and the sum of the number of rows of multiple first sub-matrices is the number of rows of the first matrix.

[0013] It should be noted that the first value is any value less than the number of rows in the first matrix. For example, to facilitate subsequent matrix multiplication operations, the first value is a positive integer multiple of M and less than the number of rows in the first matrix, where M is the number of rows in the matrix to be multiplied.

[0014] In the second case, the second direction is the row direction. Dividing the first matrix in the row direction means dividing the first matrix into columns. In this case, the columns of the first matrix are divided with the second value as the step size, resulting in multiple first submatrices. The number of columns in each first submatric is the first value, the number of rows in each first submatric is equal to the number of rows in the first matrix, and the sum of the number of columns in multiple first submatrices is the number of columns in the first matrix.

[0015] It should be noted that the second value is any value less than the number of columns in the first matrix. For example, to facilitate subsequent matrix multiplication operations, the second value is a positive integer multiple of N and less than the number of columns in the first matrix, where N is the number of columns in the matrix to be multiplied.

[0016] The compression of the non-zero element group in each of the multiple first sub-matrices in the first direction can be understood as squeezing the non-zero element group in each of the multiple first sub-matrices toward the first direction, so that the non-zero element groups in the multiple first sub-matrices are sequentially spliced ​​together in the first direction.

[0017] The process of compressing the non-zero element groups in each of the multiple first submatrices along a first direction to obtain multiple second submatrices corresponding to the first matrix includes: for each of the multiple first submatrices, concatenating the non-zero element groups in the first submatrices along a first direction to obtain the second submatrices. Here, the first direction is the row direction of the first submatrices and the second direction is the column direction of the first submatrices, or the first direction is the column direction of the first submatrices and the second direction is the row direction of the first submatrices.

[0018] Since the second direction can be either row-oriented or column-oriented, the process of concatenating the non-zero elements of each first submatrix in the first direction to obtain the second submatrix varies depending on the circumstances. These processes will be described separately below.

[0019] In the first case, the first direction is the row direction, the second direction is the column direction, and the non-zero element group is a column vector containing non-zero elements (i.e., non-zero columns). The first processor keeps the row numbers of each element in the non-zero element group of the first submatrix unchanged, and concatenates the non-zero element groups of the first submatrix in ascending column order to obtain the second submatrix. The number of columns in the second submatrix is ​​the same as the number of non-zero columns in the submatrix before concatenation, while the number of rows remains unchanged.

[0020] In the second case, the first direction is the column direction, the second direction is the row direction, and the non-zero element group is a row vector containing non-zero elements (i.e., non-zero rows). The first processor keeps the column number of each element in the non-zero element group of the first submatrix unchanged, and concatenates the non-zero element groups of the first submatrix in ascending order of row number to obtain the second submatrix. The second submatrix has the same number of rows as the submatrix before concatenation, and the number of columns remains unchanged.

[0021] It is understandable that since the third submatrix is ​​obtained by partitioning the second submatrix, multiple third submatrixes are obtained by compressing the first matrix. There is no set of all-zero elements between any two non-zero element groups in the third submatrix. The non-zero element group is a vector along the second direction in the third submatrix, and there are elements with non-zero values ​​in the non-zero element group. The values ​​of all-zero elements are all zero in the set of all-zero elements. The second direction is either the row direction or the column direction.

[0022] When the second direction is a row direction, the first direction is a column direction; when the second direction is a column direction, the first direction is a row direction. Therefore, when the first direction is a row direction, the second direction is a column direction, and the non-zero element group is a column vector containing non-zero elements, there is no all-zero column vector between any two column vectors containing non-zero elements in the third submatrix, and the values ​​of the elements in the all-zero column vector are all zero; when the first direction is a column direction, the second direction is a row direction, and the non-zero element group is a row vector containing non-zero elements, there is no all-zero row vector between any two row vectors containing non-zero elements in the third submatrix, and the values ​​of the elements in the all-zero row vector are all zero.

[0023] Since the second direction can be either row-oriented or column-oriented, the process of dividing each second submatrix into multiple third submatrixes differs depending on the circumstances. These processes will be described separately below.

[0024] In the first case, the second direction is the column direction. The first processor divides the columns of each of the multiple second submatrices to obtain multiple third submatrices.

[0025] In one possible implementation, for any one of the multiple second submatrices, the columns of the second submatric are divided with a third value as the step size to obtain multiple third submatrices corresponding to the second submatric. Here, the number of columns in each of the multiple third submatrices corresponding to the second submatric is the third value, the number of rows in each third submatric is equal to the number of rows in the second submatric, and the sum of the number of columns in the multiple third submatrices is equal to the number of columns in the second submatric.

[0026] In one possible implementation, if the second direction is a column direction and the number of columns of the second submatrix is ​​not a multiple of the third value, then before partitioning the second submatrix, z columns of all zeros can be added to the second submatrix so that the number of columns of the second submatrix is ​​X times the third value, where X is the floor value of the second submatrix divided by the third value, and z is the difference between X multiplied by the third value and the number of columns of the second submatrix.

[0027] In other words, before dividing the columns of the second submatrix, it is necessary to ensure that the columns of the second submatrix are multiples of the third value. This ensures that the resulting third submatrix has a consistent number of rows and columns, thereby guaranteeing that the data structure of the compressed information corresponding to the third submatrix in the subsequent compression results is consistent, thus improving the accuracy and efficiency of subsequent matrix processing or analysis.

[0028] In the second case, the second direction is the row direction. The first processor divides the rows of each second submatrix in the multiple second submatrixes to obtain multiple third submatrixes.

[0029] In one possible implementation, for any one of the multiple second submatrices, the rows of the second submatric are divided with a fourth value as the step size to obtain multiple third submatrices corresponding to the second submatric. The number of rows in each of the multiple third submatrices corresponding to the second submatric is the fourth value, the number of columns in each third submatric is equal to the number of columns in the second submatric, and the sum of the number of rows in the multiple third submatrices is equal to the number of rows in the second submatric.

[0030] In one possible implementation, if the second direction is a row direction and the number of rows of the second submatrix is ​​not a multiple of the fourth value, then before dividing the second submatrix, C rows of all zeros can be added to the second submatrix so that the number of rows of the second submatrix is ​​Y times the fourth value, where Y is the floor value of the second submatrix divided by the fourth value, and C is the difference between the value of Y multiplied by the fourth value and the number of rows of the second submatrix.

[0031] In other words, before dividing the rows of the second submatrix, it is necessary to ensure that the number of rows of the second submatrix is ​​a multiple of the fourth value. This ensures that the resulting third submatrix has a consistent number of rows and columns, thereby guaranteeing that the data structure of the compressed information corresponding to the third submatrix in the subsequent compression results is consistent, thus improving the accuracy and efficiency of subsequent matrix processing or analysis.

[0032] In one possible implementation, the compression result includes submatrix position information and submatrix element position information. The submatrix position information indicates the relative positions between multiple third submatrixes, and the submatrix element position information indicates the position of non-zero elements in the corresponding third submatrix.

[0033] In one possible implementation, the submatrix location information includes a submatrix row pointer and a submatrix column index. The submatrix row pointer indicates the number of third submatrixes in each row, and the submatrix column index indicates the column in which each third submatrix is ​​located.

[0034] In one possible implementation, the submatrix row pointer includes multiple elements, and the difference between the value of the (i+1)th element and the value of the ith element is the number of the third submatrix in the ith row, where i is greater than or equal to 1 and less than or equal to L-1, and L is the total number of elements.

[0035] In one possible implementation, the submatrix element position information includes a submatrix bitmap, which includes bitmaps corresponding to multiple third submatrixes. The bitmaps indicate the position of non-zero elements in the corresponding third submatrixes. Each bitmap includes multiple bits of data, and the number of bits is the same as the number of elements in the corresponding third submatrix.

[0036] In one possible implementation, the compression result also includes non-zero element information, which includes the values ​​of all non-zero elements in the third submatrix and the indices of the non-zero elements in the second direction of the first matrix.

[0037] In one possible implementation, the values ​​of non-zero elements in multiple third submatrices are placed in a first array, which is a one-dimensional array, and the positions of the non-zero elements in the same third submatrix within the first array are consecutive. Of course, the positions of the non-zero elements in the same third submatrix within the first array can also be non-consecutive.

[0038] If data is scattered across different locations in the array, additional data structures (such as hash tables or index trees) are needed to manage it, increasing the complexity of data management. This application stores the values ​​and indices of non-zero elements contiguously within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the indices and values ​​of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine their positions, thus improving data access efficiency. Furthermore, since caches typically load recently accessed data items and their adjacent data items together, when the indices and values ​​of non-zero elements are stored contiguously in the array, these elements are more likely to be loaded into the cache consecutively, reducing cache misses and improving data access speed.

[0039] When the second direction is a column direction, the index of a non-zero element in the second direction of the first matrix is ​​the column index of the non-zero element in the first matrix (i.e., the column number where the non-zero element is located in the first matrix). When the second direction is a row direction, the index of a non-zero element in the second direction of the first matrix is ​​the row index of the non-zero element in the first matrix (i.e., the row number where the non-zero element is located in the first matrix).

[0040] In one possible implementation, elements at the same position in the first and second arrays correspond to the same non-zero element.

[0041] Since an array is a linear data structure in which elements are placed in a specific order, and each element in an array can be uniquely identified by its array index (i.e., position or sequence number), when the value and index of a non-zero element are located in the same position in two arrays, the corresponding value can be accessed from both arrays based on the same array index without additional lookup or mapping operations, thus effectively improving access efficiency.

[0042] In one possible implementation, after obtaining the compression result of the first matrix, it may be necessary to further process the first matrix based on the compression result (such as determining the target matrix based on the first matrix and the second matrix, where the target matrix is ​​the product of the first matrix and the second matrix). Therefore, the first processor can store the compression result of the first matrix in the memory for later retrieval.

[0043] Secondly, a matrix processing method is provided, applied in a computer system, the computer system including a second processor and a memory, the method being executed by the second processor, comprising: determining a target matrix based on a compression result of a first matrix and a second matrix stored in the memory, the target matrix being the product of the first matrix and the second matrix; wherein, the first matrix is ​​input information or intermediate result in the model training or inference process, the compression result of the first matrix is ​​generated based on multiple third sub-matrices, the compression result indicating the values ​​of non-zero elements in all third sub-matrices and the positions of the non-zero elements in the corresponding third sub-matrices, the multiple third sub-matrices being obtained by partitioning each of the multiple second sub-matrices, the multiple second sub-matrices being obtained by compressing the non-zero element groups in each of the multiple first sub-matrices according to a first direction, the multiple first sub-matrices being obtained by partitioning the first matrix, the non-zero element groups being vectors in the first sub-matrices along a second direction, and the non-zero element groups containing elements with non-zero values.

[0044] This application can first divide the first matrix to be compressed into smaller sub-matrices, and then compress the non-zero elements in the resulting sub-matrices to obtain a second sub-matrix. This ensures that the second sub-matrix contains only non-zero elements and no all-zero elements, effectively eliminating some zero-valued elements from the first matrix. In this case, based on multiple third sub-matrices, the compression result of the first matrix can effectively reduce the number of zero elements that need to be stored, thus saving data storage space. Furthermore, in scenarios requiring matrix multiplication, the matrix can be divided according to computational needs to obtain sub-matrices of suitable size for computation, thereby effectively improving computational efficiency.

[0045] It should be noted that the input information or intermediate results in the model training or inference process are usually in the form of tensors. A tensor is a multidimensional array that can represent multidimensional data and complex multilinear relationships. The order (or rank) of a tensor refers to the number of dimensions represented by the tensor, including zero-order tensors (i.e., scalars), first-order tensors (i.e., vectors), second-order tensors (i.e., matrices), and third-order and higher-order tensors. Therefore, the input information or intermediate results in the model training or inference process in this application can be second-order tensors (i.e., matrices) or third-order and higher-order tensors. When the input information or intermediate results in the model training or inference process are second-order tensors, the first matrix is ​​the second-order tensor. When the input information or intermediate results in the model training or inference process are third-order and higher-order tensors, slicing operations (i.e., slicing the tensor according to specific dimensions), reshaping operations (i.e., reshaping the high-dimensional tensor into a low-dimensional matrix), etc., can be performed on the third-order and higher-order tensors to obtain at least one matrix. In this case, any one of the at least one matrices is determined as the first matrix.

[0046] In one possible implementation, based on the compression result of the first matrix, the plurality of third sub-matrices are restored; the position of each element in the plurality of third sub-matrices in the first matrix is ​​determined; based on the position of each element in the plurality of third sub-matrices in the first matrix, the computational matrices corresponding to the plurality of third sub-matrices are determined from the second matrix; and the plurality of third sub-matrices are multiplied by their respective corresponding computational matrices to obtain the target matrix.

[0047] In one possible implementation, the number of rows of the third submatrix is ​​a positive integer multiple of M, the number of columns is a positive integer multiple of N, and M×N is the size of the matrix for matrix multiplication. Both M and N are positive integers. In this case, the multiple third submatrixes are divided to obtain multiple first computation matrices, each of which is M×N in size. Based on the position of each element in the multiple first computation matrices in the first matrix, the second computation matrices corresponding to the multiple first computation matrices are determined from the second matrix.

[0048] In one possible implementation, the size of the third submatrix is ​​M×N. In this case, the second processor can directly determine multiple third submatrices as multiple first computation matrices. When the size of the third submatrix is ​​the size of the matrix used for matrix multiplication, it means that the size of the third submatrix matches the standard size of matrix multiplication, and the compressed result is stored in units of the third submatrix. Therefore, when performing matrix multiplication, it is only necessary to retrieve the data corresponding to the third submatrix from the compressed result, without frequent access to external storage or memory swapping, thereby improving the computational efficiency of matrix multiplication.

[0049] In one possible implementation, the position of each element in the multiple first computation matrices includes the row number and column number of the element in the first matrix; the process of determining the second computation matrix corresponding to each of the multiple first computation matrices from the second matrix based on the position of each element in the multiple first computation matrices includes: determining the elements in the second matrix whose column number is the same as the row number of the target element and whose row number is the same as the column number of the target element as the elements in the second computation matrix corresponding to the target computation matrix; wherein, the target computation matrix is ​​any one of the multiple first computation matrices, and the target element is any one element of the target computation matrix.

[0050] In one possible implementation, multiple first computation matrices are multiplied by their respective second computation matrices to obtain the target matrix.

[0051] Since matrix multiplication includes left and right multiplication, the methods for multiplying multiple third submatrices with their respective computational matrices to obtain the target matrix differ depending on the circumstances. These methods will be described separately below. Furthermore, for ease of description, if the first matrix is ​​right-multiplied by the second matrix, the first matrix is ​​called the left matrix and the second matrix is ​​called the right matrix; if the first matrix is ​​left-multiplied by the second matrix, the first matrix is ​​called the right matrix and the second matrix is ​​called the left matrix.

[0052] In the first case, the first matrix is ​​the left matrix, and the second matrix is ​​the right matrix. For any one of the multiple first computation matrices, this first computation matrix is ​​loaded into the on-chip register corresponding to the left matrix, and the corresponding second computation matrix is ​​loaded into the on-chip register corresponding to the right matrix. Then, the first computation matrix is ​​multiplied by the corresponding second computation matrix through the tensor computation unit to obtain the target submatrix. Processing each first computation matrix in the same way yields multiple target submatrices. Based on the positions of the first computation matrices corresponding to each target submatrix in the intermediate results, the multiple target submatrices are concatenated to obtain the target matrix.

[0053] In the second case, the first matrix is ​​the right matrix, and the second matrix is ​​the left matrix. For any one of the multiple first computation matrices, this first computation matrix is ​​loaded into the on-chip register corresponding to the right matrix, and the corresponding second computation matrix is ​​loaded into the on-chip register corresponding to the left matrix. Then, the first computation matrix is ​​multiplied by the corresponding second computation matrix through the tensor computation unit to obtain the target submatrix. Processing each first computation matrix in the same way yields multiple target submatrices. Based on the positions of the first computation matrices corresponding to each target submatrix in the intermediate results, the multiple target submatrices are concatenated to obtain the target matrix.

[0054] Thirdly, a compression result is provided, which includes submatrix position information, submatrix element position information, and non-zero element information; wherein, the submatrix position information indicates the relative positions between multiple third submatrixes, the submatrix element position information indicates the position of a non-zero element in the corresponding third submatrix, the non-zero element information includes the values ​​of non-zero elements in all third submatrixes and the index of the non-zero element in the second direction of the first matrix, the multiple third submatrixes are obtained by compressing the first matrix, there is no all-zero element group between any two non-zero element groups in the third submatrix, the non-zero element group is a vector in the third submatrix along the second direction, and there are elements with non-zero values ​​in the non-zero element group, the values ​​of all elements in the all-zero element group are zero, and the second direction is a row direction or a column direction.

[0055] In one possible implementation, the submatrix location information includes a submatrix row pointer and a submatrix column index. The submatrix row pointer indicates the number of third submatrixes in each row, and the submatrix column index indicates the column in which each third submatrix is ​​located.

[0056] In one possible implementation, the submatrix row pointer includes multiple elements, and the difference between the value of the (i+1)th element and the value of the ith element is the number of the third submatrix in the ith row, where i is greater than or equal to 1 and less than or equal to L-1, and L is the total number of elements.

[0057] In one possible implementation, the submatrix element position information includes a submatrix bitmap, which includes bitmaps corresponding to multiple third submatrixes, and the bitmaps indicate the positions of non-zero elements in the corresponding third submatrixes.

[0058] Each bitmap includes multiple bits of data, the number of which is the same as the number of elements in the corresponding third submatrix.

[0059] In one possible implementation, the values ​​of non-zero elements in multiple third submatrices are placed in a first array, which is a one-dimensional array, and the positions of the non-zero elements in the same third submatrix within the first array are consecutive. Of course, the positions of the non-zero elements in the same third submatrix within the first array can also be non-consecutive.

[0060] In one possible implementation, the indices of non-zero elements in the second direction of the first matrix are placed in a second array, which is a one-dimensional array, and the indices of non-zero elements in the same third submatrix in the second direction of the first matrix are consecutive in the second array. Of course, the indices of non-zero elements in the same third submatrix in the second direction of the first matrix can also be non-consecutive in the second array.

[0061] If data is scattered across different locations in the array, additional data structures (such as hash tables or index trees) are needed to manage it, increasing the complexity of data management. This application stores the values ​​and indices of non-zero elements contiguously within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the indices and values ​​of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine their positions, thus improving data access efficiency. Furthermore, since caches typically load recently accessed data items and their adjacent data items together, when the indices and values ​​of non-zero elements are stored contiguously in the array, these elements are more likely to be loaded into the cache consecutively, reducing cache misses and improving data access speed.

[0062] When the second direction is a column direction, the index of a non-zero element in the second direction of the first matrix is ​​the column index of the non-zero element in the first matrix (i.e., the column number where the non-zero element is located in the first matrix). When the second direction is a row direction, the index of a non-zero element in the second direction of the first matrix is ​​the row index of the non-zero element in the first matrix (i.e., the row number where the non-zero element is located in the first matrix).

[0063] In one possible implementation, elements at the same position in the first and second arrays correspond to the same non-zero element.

[0064] Since an array is a linear data structure in which elements are placed in a specific order, and each element in an array can be uniquely identified by its array index (i.e., position or sequence number), when the value and index of a non-zero element are located in the same position in two arrays, the corresponding value can be accessed from both arrays based on the same array index without additional lookup or mapping operations, thus effectively improving access efficiency.

[0065] Fourthly, a matrix compression device is provided, comprising a first processor in a computer system, the computer system further comprising a memory, the device comprising:

[0066] The first partitioning module is used to partition the first matrix stored in the memory to obtain multiple first sub-matrices. The first matrix is ​​the input information or intermediate result in the model training or inference process.

[0067] The compression module is used to compress the non-zero element group in each of the multiple first sub-matrices in a first direction to obtain multiple second sub-matrices corresponding to the first matrix. The non-zero element group is a vector in the first sub-matrices along the second direction, and there are elements with non-zero values ​​in the non-zero element group.

[0068] The second partitioning module is used to partition each of the multiple second submatrices to obtain multiple third submatrices.

[0069] The generation module is used to generate a compressed result of the first matrix based on multiple third sub-matrices. The compressed result indicates the values ​​of non-zero elements in all third sub-matrices and the positions of the non-zero elements in the corresponding third sub-matrices.

[0070] In one possible implementation, the compression module is specifically used for:

[0071] For each of the multiple first submatrices, the non-zero element groups in the first submatrices are concatenated in the first direction to obtain the second submatrices, where the first direction is the row direction of the first submatrices and the second direction is the column direction of the first submatrices, or the first direction is the column direction of the first submatrices and the second direction is the row direction of the first submatrices.

[0072] In one possible implementation, the compression module is specifically used for:

[0073] Given that the first direction is the row direction, the second direction is the column direction, and the non-zero element group is a column vector containing non-zero elements, keep the row number of each element in the non-zero element group of the first submatrix unchanged, and concatenate the non-zero element groups of the first submatrix in ascending order of column number to obtain the second submatrix.

[0074] Given that the first direction is the column direction, the second direction is the row direction, and the non-zero element group is a row vector containing non-zero elements, keep the column number of each element in the non-zero element group of the first submatrix unchanged, and concatenate the non-zero element groups of the first submatrix in ascending order of row number to obtain the second submatrix.

[0075] In one possible implementation, the first direction is the row direction, the second direction is the column direction, and the non-zero element group is a column vector containing non-zero elements; in the third submatrix, there are no all-zero column vectors between any two column vectors containing non-zero elements, and the values ​​of the elements in the all-zero column vectors are all zero; or

[0076] The first direction is the column direction, the second direction is the row direction, and the non-zero element group is a row vector containing non-zero elements; in the third submatrix, there is no all-zero row vector between any two row vectors containing non-zero elements, and the values ​​of the elements in the all-zero row vector are all zero.

[0077] In one possible implementation, the compression result includes submatrix position information and submatrix element position information. The submatrix position information indicates the relative positions between multiple third submatrixes, and the submatrix element position information indicates the position of non-zero elements in the corresponding third submatrix.

[0078] In one possible implementation, the submatrix location information includes a submatrix row pointer and a submatrix column index. The submatrix row pointer indicates the number of third submatrixes in each row, and the submatrix column index indicates the column in which each third submatrix is ​​located.

[0079] In one possible implementation, the submatrix element position information includes a submatrix bitmap, which includes bitmaps corresponding to multiple third submatrixes, and the bitmaps indicate the positions of non-zero elements in the corresponding third submatrixes.

[0080] In one possible implementation, the compression result includes non-zero element information, which includes the values ​​of all non-zero elements in the third submatrix and the indices of the non-zero elements in the second direction of the first matrix.

[0081] In one possible implementation, the values ​​of non-zero elements in multiple third submatrices are placed within a first array, which is a one-dimensional array, and the positions of non-zero elements in the same third submatrix within the first array are consecutive; and / or

[0082] The indices of non-zero elements in the second direction of the first matrix are placed in the second array, which is a one-dimensional array, and the indices of non-zero elements in the same third submatrix in the second direction of the first matrix are consecutive in the second array.

[0083] In one possible implementation, the generation module is specifically used for:

[0084] The compression result of the first matrix is ​​stored in memory.

[0085] Fifthly, a matrix processing apparatus is provided, comprising a second processor in a computer system, the computer system further comprising a memory, the apparatus comprising:

[0086] The determination module is used to determine the target matrix based on the compression result of the first matrix and the second matrix stored in the memory. The target matrix is ​​the product of the first matrix and the second matrix. The first matrix is ​​the input information or intermediate result in the model training or inference process. The compression result of the first matrix is ​​generated based on multiple third sub-matrices. The compression result indicates the value of the non-zero element in all third sub-matrices and the position of the non-zero element in the corresponding third sub-matrices. The multiple third sub-matrices are obtained by partitioning each of the multiple second sub-matrices. The multiple second sub-matrices are obtained by compressing the non-zero element group in each of the multiple first sub-matrices according to the first direction. The multiple first sub-matrices are obtained by partitioning the first matrix. The non-zero element group is the vector in the first sub-matrices along the second direction, and there are elements with non-zero values ​​in the non-zero element group.

[0087] In one possible implementation, the module is specifically used for:

[0088] Based on the compression result of the first matrix, multiple third sub-matrices are restored;

[0089] Determine the position of each element in the first matrix within multiple third submatrices;

[0090] Based on the position of each element in the first matrix in the multiple third sub-matrices, the computational matrices corresponding to the multiple third sub-matrices are determined from the second matrix;

[0091] The target matrix is ​​obtained by multiplying each third submatrix by its corresponding computation matrix.

[0092] In one possible implementation, the number of rows of the third submatrix is ​​a positive integer multiple of M, the number of columns is a positive integer multiple of N, and M×N is the size of the matrix to be multiplied, where M and N are both positive integers.

[0093] The module is specifically used for:

[0094] Multiple third submatrices are divided to obtain multiple first computation matrices, each of which has a size of M×N;

[0095] Based on the position of each element in the first matrix, the second calculation matrix corresponding to each of the first calculation matrices is determined from the second matrix;

[0096] Multiplying each of the third submatrices by its corresponding computational matrix yields the target matrix, including:

[0097] The target matrix is ​​obtained by multiplying the multiple first calculation matrices by their respective second calculation matrices.

[0098] In one possible implementation, the position of each element in the multiple first computation matrices includes the row number and column number of the element in the first matrix;

[0099] The module is specifically used for:

[0100] The elements in the second matrix whose column number is the same as the row number of the target element and whose row number is the same as the column number of the target element are identified as the elements in the second calculation matrix corresponding to the target calculation matrix.

[0101] The target calculation matrix is ​​any one of the multiple first calculation matrices, and the target element is any one of the elements in the target calculation matrix.

[0102] In a sixth aspect, a computer system is provided, the computer system comprising a first processor, a second processor, and a memory; the memory is used to store a first matrix; the first processor is used to divide the first matrix stored in the memory into a plurality of first sub-matrices, the first matrix being input information or intermediate results in a model training or inference process; to compress the non-zero element group in each of the plurality of first sub-matrices along a first direction to obtain a plurality of second sub-matrices corresponding to the first matrix, the non-zero element group being a vector in the first sub-matrices along a second direction, and the non-zero element group containing elements with non-zero values; to divide each of the plurality of second sub-matrices into a plurality of third sub-matrices; and to generate a compression result of the first matrix based on the plurality of third sub-matrices, the compression result indicating the values ​​of non-zero elements in all third sub-matrices and the positions of the non-zero elements in the corresponding third sub-matrices; the memory is further used to store the compression result of the first matrix and a second matrix; the second processor is used to determine a target matrix based on the compression result of the first matrix stored in the memory and the second matrix, the target matrix being the product of the first matrix and the second matrix.

[0103] In a seventh aspect, a chip is provided, the chip including a processor configured to perform the matrix compression method of the first aspect described above, and / or configured to perform the matrix processing method of the second aspect described above.

[0104] Optionally, the processor may further include an interface circuit for receiving instructions and transmitting them to the processor.

[0105] Eighthly, a computer-readable storage medium is provided, wherein the storage medium stores a computer program that, when running on a computer or processor, causes the computer or processor to perform the steps of the matrix compression method described in the first aspect, and / or causes the computer or processor to perform the steps of the matrix processing method described in the second aspect, or stores the compression result described in the third aspect.

[0106] A ninth aspect provides a computer program product comprising computer instructions that, when executed on a computer or processor, cause the computer to perform the steps of the matrix compression method described in the first aspect, and / or cause the computer to perform the steps of the matrix processing method described in the second aspect. Alternatively, a computer program is provided that, when executed on a computer or processor, causes the computer or processor to perform the steps of the matrix compression method described in the first aspect, and / or cause the computer or processor to perform the steps of the matrix processing method described in the second aspect.

[0107] The technical effects achieved by the third to ninth aspects mentioned above are similar to those achieved by the corresponding technical means in the first and second aspects, and will not be repeated here. Attached Figure Description

[0108] Figure 1 is a schematic diagram of a computer system provided in an embodiment of this application;

[0109] Figure 2 is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0110] Figure 3 is a flowchart of a matrix compression method provided in an embodiment of this application;

[0111] Figure 4 is a schematic diagram of a first matrix provided in an embodiment of this application;

[0112] Figure 5 is a schematic diagram of a first sub-matrix provided in an embodiment of this application;

[0113] Figure 6 is a schematic diagram of a second sub-matrix provided in an embodiment of this application;

[0114] Figure 7 is a schematic diagram of another first sub-matrix provided in an embodiment of this application;

[0115] Figure 8 is a schematic diagram of another second sub-matrix provided in an embodiment of this application;

[0116] Figure 9 is a schematic diagram of a third sub-matrix provided in an embodiment of this application;

[0117] Figure 10 is a schematic diagram of another third sub-matrix provided in an embodiment of this application;

[0118] Figure 11 is a schematic diagram of a submatrix row pointer and a submatrix column index provided in an embodiment of this application;

[0119] Figure 12 is a schematic diagram of a sub-matrix bitmap provided in an embodiment of this application;

[0120] Figure 13 is a schematic diagram of the compression result of a first matrix provided in an embodiment of this application;

[0121] Figure 14 is a schematic diagram of a matrix processing method provided in an embodiment of this application;

[0122] Figure 15 is a schematic diagram of another matrix processing method provided in an embodiment of this application;

[0123] Figure 16 is a schematic diagram of a matrix compression device provided in an embodiment of this application;

[0124] Figure 17 is a schematic diagram of the structure of a matrix processing device provided in an embodiment of this application. Detailed Implementation

[0125] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0126] Before providing a detailed explanation of the methods provided in the embodiments of this application, the application scenarios and implementation environments involved in the embodiments of this application will be introduced first.

[0127] First, the application scenarios involved in the embodiments of this application will be introduced.

[0128] In a matrix, if the number of elements with a value of 0 far exceeds the number of non-zero elements, and the distribution of non-zero elements is irregular, then the matrix is ​​called a sparse matrix. In practical applications, many large-scale datasets and mathematical models can be represented as sparse matrices. For example, mathematical models in fields such as social problems, computational fluid dynamics, statistical physics, circuit simulation, and image processing often involve a large number of variables and parameters, but only a few of these variables or parameters have non-zero values ​​or significant effects under specific conditions. These mathematical models can be represented as sparse matrices. For instance, in social relationship problems, an individual's social circle is finite relative to the total population. When representing social relationships between people using a matrix, most elements in the matrix are 0 (i.e., indicating no social relationship). In this case, the matrix corresponding to the data model of social relationships is a sparse matrix. Therefore, sparse matrices are widely used in machine learning, HPC scientific computing, search and recommendation, graph neural networks, and large sparse AI models. However, since sparse matrices have fewer non-zero elements and more zero elements, using a conventional two-dimensional array to store them would waste a lot of storage units to store the zero elements. Therefore, how to compress sparse matrices is a current research hotspot.

[0129] Related technologies provide a scheme for denserening sparse matrices. In this scheme, a reordering algorithm is used to transform the sparse matrix (i.e., column and row transformations) to cluster the non-zero elements together. Then, the transformed sparse matrix is ​​compressed and stored using a relevant compression format (such as CSR). Subsequently, when performing matrix multiplication, the compressed sparse matrix is ​​multiplied by another matrix, or the directly transformed sparse matrix is ​​multiplied by another matrix. Regardless of the method used, to ensure the accuracy of the matrix multiplication result, the other matrix also needs to undergo the corresponding row and column transformations according to the sparse matrix transformation method.

[0130] Therefore, it is evident that the above-mentioned algorithm for row and column transformation of sparse matrices has high computational complexity and significant data processing overhead. Moreover, since different types of sparse matrices have different distribution characteristics of non-zero elements, the reordering algorithm used for sparse matrix densification may perform well on one type of sparse matrix but poorly on another. In other words, the generalization ability of the above-mentioned sparse matrix densification is poor.

[0131] Based on this, this application provides a matrix compression method. In this method, a first matrix is ​​divided into multiple first sub-matrices. Then, the non-zero element groups in each first sub-matrice are compressed along a first direction to obtain multiple second sub-matrices. Each of the multiple second sub-matrices is then divided to obtain multiple third sub-matrices. Based on these multiple third sub-matrices, the compression result of the first matrix is ​​determined. In other words, this application can first divide the first matrix to be compressed into smaller sub-matrices, and then compress the non-zero element groups in the resulting sub-matrices to obtain second sub-matrices. This ensures that the second sub-matrices contain only non-zero element groups and no all-zero element groups, effectively eliminating some zero-valued elements from the first matrix. In this case, the compression result of the first matrix determined based on the multiple third sub-matrices can effectively reduce the number of zero elements that need to be stored, thereby saving data storage space. Furthermore, in scenarios requiring matrix multiplication, the matrix can be divided according to computational requirements to obtain sub-matrices of suitable size for computation, thereby effectively improving computational efficiency.

[0132] Next, the implementation environment involved in the embodiments of this application will be introduced.

[0133] Please refer to Figure 1, which is a schematic diagram of a computer system provided in an embodiment of this application. The computer system includes a first processor 01 and a memory 03. The first processor 01 can divide a first matrix stored in the memory 03 to obtain multiple first sub-matrices. The first matrix represents input information or intermediate results during model training or inference. The non-zero element groups in each of the multiple first sub-matrices are compressed along a first direction to obtain multiple second sub-matrices corresponding to the first matrix. The non-zero element groups are vectors along a second direction in the first sub-matrices, and each non-zero element group contains elements with non-zero values. Each of the multiple second sub-matrices is further divided to obtain multiple third sub-matrices. Based on the multiple third sub-matrices, a compression result of the first matrix is ​​generated. The compression result indicates the values ​​of the non-zero elements in all third sub-matrices and the positions of the non-zero elements in the corresponding third sub-matrices.

[0134] In some embodiments, the compression result obtained by the first processor 01 after performing a matrix compression method on the first matrix can be stored in the memory 03.

[0135] In some embodiments, the computer system further includes a second processor 02, which is capable of accessing the memory 03. When matrix multiplication is required, the second processor 02 can obtain the compression result of the first matrix from the memory 03 and determine the target matrix based on the compression result of the first matrix and the second matrix stored in the memory 03. The target matrix is ​​the product of the first matrix and the second matrix.

[0136] It should be noted that the first processor 01 and the second processor 02 mentioned above can be two independent processors or the same processor. That is, a processor can execute both the matrix compression method and the matrix processing method provided in the embodiments of this application.

[0137] It is understood that the matrix compression method in this application embodiment is mainly implemented through the general computing power of the processor, while the matrix processing method in this application embodiment is mainly implemented through the parallel computing power of the processor. Therefore, whether to use two independent processors or one processor depends on the actual application scenario and the computing power of the processor.

[0138] For example, a central processing unit (CPU) is a general-purpose processor, adept at handling complex logic and computational processes, including data conversion and format processing. Therefore, in scenarios where high matrix compression efficiency is required, the matrix compression method in this application embodiment can be executed by the CPU. A graphics processing unit (GPU) contains a large number of cores (typically hundreds of times the number of CPU cores). These cores can process data simultaneously, making them suitable for executing highly parallel computational tasks. A neural network processing unit (NPU) is a processor specifically designed for neural network computation, possessing extremely high parallel computing capabilities and hardware acceleration for specific operations (such as matrix multiplication). When processing large-scale matrix multiplication, the NPU can fully utilize its internal hardware acceleration units to achieve efficient and fast computation. Therefore, in scenarios where high matrix multiplication efficiency is required, the matrix processing method in this application embodiment can be executed by both the GPU and the NPU.

[0139] While CPUs can perform a certain degree of parallel computing, their core count and design are primarily geared towards handling general-purpose computing tasks, rather than being optimized for large-scale parallel operations. Compared to NPUs and GPUs, CPUs are less efficient at performing matrix multiplication. Therefore, in scenarios where matrix multiplication efficiency is not critical, the matrix processing method provided in this application embodiment can also be executed using a CPU. As for GPUs and NPUs, both excel at handling large-scale parallel operations. However, compared to CPUs, GPUs and NPUs have weaker general-purpose computing capabilities. Therefore, in scenarios where matrix compression efficiency is not critical, the matrix compression method provided in this application embodiment can also be executed using GPUs and NPUs.

[0140] The following section will use the first processor 01 and the second processor 02 as two independent processors as examples.

[0141] In some embodiments, the memory 03 includes a cache of the first processor 01, and the second processor 02 is able to access the cache of the first processor 01. In this case, the first processor 01 is able to store the compression result of the first matrix into its own cache. When matrix multiplication is required, the second processor 02 is able to retrieve the compression result of the first matrix from the cache of the first processor 01, and determine the target matrix based on the compression result of the first matrix and the second matrix.

[0142] In other embodiments, memory 03 includes a cache for the first processor and a cache for the second processor 02, and the first processor 02 can access the cache for the second processor 01. In this case, the cache of the first processor stores the first matrix. Furthermore, after determining the compression result of the first matrix, the first processor 01 can copy the compression result of the first matrix stored in its own cache to the cache of the second processor 02. When matrix multiplication is required, the cache of the second processor 02 also stores the second matrix, allowing the second processor to retrieve the compression result of the first matrix and the second matrix from its own cache, and determine the target matrix based on the compression result of the first matrix and the second matrix.

[0143] In other embodiments, memory 03 is a permanent storage medium (also known as non-volatile storage), such as a hard disk drive (HDD), a solid-state drive (SSD), etc. Both the first processor 01 and the second processor 02 can access memory 03. In this case, the first processor 01 can copy the compressed result of the first matrix to memory 03. When matrix multiplication is required, the second processor 02 can obtain the compressed result of the first matrix from memory 03 and determine the target matrix based on the compressed result of the first matrix and the second matrix.

[0144] It should be noted that the above description of the form of memory 03 and the interaction between the first processor 01, the second processor 02 and memory 03 are merely examples for better understanding the solutions of the embodiments of this application. The form of memory 03 and the interaction between the first processor 01, the second processor 02 and memory 03 can also be other forms, and the embodiments of this application do not limit them.

[0145] It is understood that before determining the multiple second sub-matrices corresponding to the first matrix, the first processor 01 can also obtain information about the first matrix and determine the first matrix based on the information about the first matrix. The information about the first matrix can be input by the user into the first processor 01 or can be stored in the first processor 01 in advance; this embodiment of the application does not limit this.

[0146] The information of the first matrix can be its original data, such as the position of each element and its value. Alternatively, the information can be represented using a compression format, such as compressed sparse row format (CSR). In this case, before determining the multiple second sub-matrices corresponding to the first matrix, the first matrix can be restored based on its information to obtain its original data. It should be noted that the detailed process of restoring the first matrix based on its information is well-known to those skilled in the art, and will not be elaborated upon in this embodiment.

[0147] It should be noted that CSR includes three types of data: value arrays, column index arrays, and row pointer arrays. The value array stores the values ​​of all non-zero elements, the column index array stores the column index corresponding to each non-zero element, and the row pointer array, also known as the row offset array, indicates the starting position of each row's non-zero element in the value array and column index array, as well as the number of non-zero elements in each row.

[0148] It is understood that when the first processor 01, the second processor 02, and the memory 03 are in the same computer system, any two of the first processor 01, the second processor 02, and the memory 03 can be integrated together, or all of the first processor 01, the second processor 02, and the memory 03 can be integrated together. For example, the first processor 01, the second processor 02, and the memory 03 can be integrated into the same chip, and this application embodiment does not limit this.

[0149] In the above description, the first processor 01, the second processor 02, and the memory 03 reside in the same computer system. Of course, in one possible implementation, the first processor 01, the second processor 02, and the memory 03 may reside in different computer systems, or any two of the first processor 01, the second processor 02, and the memory 03 may reside in the same computer system; this embodiment does not limit this. For example, referring to Figure 2, the first processor 01, the second processor 02, and the memory 03 are located in three independent physical devices.

[0150] It should be noted that the first processor 01 and the second processor 02 can be a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a network processor (NP), a microprocessor, or one or more integrated circuits for implementing the scheme of this application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. For example, the first processor 01 is a CPU, and the second processor 02 is an NPU or a GPU.

[0151] In some embodiments, memory 03 may also be referred to as storage medium 03, which may be a cache, a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile optical disc, a Blu-ray disc, etc.), a magnetic disk storage medium, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but not limited thereto.

[0152] In some embodiments, the computer system further includes a communication bus 04 and / or a communication interface 05. The communication bus 04 is used to transmit information between the aforementioned components. The communication bus 04 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 05 uses any transceiver-like device for communicating with other devices or communication networks. The communication interface 05 includes a wired communication interface and may also include a wireless communication interface. The wired communication interface can be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface can be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0153] Those skilled in the art should understand that the first processor, second processor, and memory described above are merely examples. Other existing or future processors and memories that are applicable to the embodiments of this application should also be included within the scope of protection of the embodiments of this application, and are hereby incorporated by reference. The hardware included in the computer system in the embodiments of this application can be deployed in one computer device or in different computer devices, and the embodiments of this application do not limit this.

[0154] It should be noted that the application scenarios and implementation environments described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0155] The methods provided in the embodiments of this application will now be explained in detail.

[0156] Figure 3 is a flowchart of a matrix compression method provided in an embodiment of this application. Referring to Figure 3, the method is applied to a computer system, which includes a first processor 01 and a memory 03 as shown in Figure 1 or Figure 2. The matrix compression method is executed by the first processor 01, and the method includes the following steps.

[0157] Step 301: The first processor 01 divides the first matrix stored in the memory 03 to obtain multiple first sub-matrices. The first matrix is ​​the input information or intermediate result in the model training or inference process.

[0158] It should be noted that the input information or intermediate results in the model training or inference process are usually in the form of tensors. A tensor is a multidimensional array that can represent multidimensional data and complex multilinear relationships. The order (or rank) of a tensor refers to the number of dimensions represented by the tensor, including zero-order tensors (i.e., scalars), first-order tensors (i.e., vectors), second-order tensors (i.e., matrices), and third-order and higher-order tensors. Therefore, the input information or intermediate results in the model training or inference process in this application embodiment can be second-order tensors (i.e., matrices) or third-order and higher-order tensors. When the input information or intermediate results in the model training or inference process are second-order tensors, the first matrix is ​​the second-order tensor. When the input information or intermediate results in the model training or inference process are third-order and higher-order tensors, slicing operations (i.e., selecting specific dimensions to slice the tensor), reshaping operations (i.e., reshaping a high-dimensional tensor into a low-dimensional matrix), etc., can be performed on the third-order and higher-order tensors to obtain at least one matrix. In this case, any one of the at least one matrices is determined as the first matrix.

[0159] The first matrix in this embodiment is a sparse matrix, which is a matrix in which the number of elements with a value of 0 is far greater than the number of non-zero elements, and the distribution of non-zero elements is irregular. In some embodiments, if the density of the matrix is ​​less than or equal to the sparsity threshold, the matrix is ​​determined to be a sparse matrix. Here, density refers to the value of the total number of non-zero elements in the matrix divided by the total number of all elements in the matrix. The sparsity threshold is preset, for example, the sparsity threshold can be 0.05. Of course, the sparsity threshold can also be adjusted according to needs, and this embodiment does not limit it.

[0160] The process of dividing the first matrix stored in memory 03 to obtain multiple first sub-matrices includes: dividing the first matrix in a second direction to obtain multiple first sub-matrices. The second direction can be either a row direction or a column direction; if the second direction is a row direction, the first direction is a column direction; if the second direction is a column direction, the first direction is a row direction.

[0161] Since the second direction can be either row-oriented or column-oriented, the process of dividing the first matrix into multiple first sub-matrices in the second direction varies depending on the circumstances. These will be described separately below.

[0162] In the first case, the second direction is the column direction. Dividing the first matrix in the column direction means dividing the first matrix into rows. In this case, the first value is used as the step size to divide the rows of the first matrix, thereby obtaining multiple first sub-matrices. The number of rows of each first sub-matrice is the first value, the number of columns of each first sub-matrice is equal to the number of columns of the first matrix, and the sum of the number of rows of multiple first sub-matrices is the number of rows of the first matrix.

[0163] It should be noted that the first value is any value less than the number of rows in the first matrix. For example, to facilitate subsequent matrix multiplication operations, the first value is a positive integer multiple of M and less than the number of rows in the first matrix, where M is the number of rows in the matrix to be multiplied. For example, if M is 4 and the number of rows in the first matrix is ​​16, then the first value can be any value among 4, 8, and 12.

[0164] In the second case, the second direction is the row direction. Dividing the first matrix in the row direction means dividing the first matrix into columns. In this case, the columns of the first matrix are divided with the second value as the step size, resulting in multiple first submatrices. The number of columns in each first submatric is the first value, the number of rows in each first submatric is equal to the number of rows in the first matrix, and the sum of the number of columns in multiple first submatrices is the number of columns in the first matrix.

[0165] It should be noted that the second value is any value less than the number of columns in the first matrix. For example, to facilitate subsequent matrix multiplication operations, the second value is a positive integer multiple of N and less than the number of columns in the first matrix, where N is the number of columns in the matrix to be multiplied. For example, if N is 4 and the number of columns in the first matrix is ​​16, then the second value can be any value among 4, 8, and 12.

[0166] Before dividing the first matrix, the first processor 01 can also acquire information about the first matrix, determine the first matrix based on the information, and then store it in the memory 03. The information about the first matrix can be input by the user into the first processor 01, or it can be pre-stored in the first processor 01; this embodiment does not limit this.

[0167] The information of the first matrix can be its original data, such as the position of each element and its value. Alternatively, the information can be represented using a compressed format, such as CSR. In this case, before determining the multiple second sub-matrices corresponding to the first matrix, the first matrix can be restored based on its information to obtain its original data. It should be noted that the detailed process of restoring the first matrix based on its information is well-known to those skilled in the art, and will not be elaborated upon in this embodiment.

[0168] Step 302: The first processor 01 compresses the non-zero element group in each of the multiple first sub-matrices in the first direction to obtain multiple second sub-matrices corresponding to the first matrix. The non-zero element group is a vector in the first sub-matrices along the second direction, and there are elements with non-zero values ​​in the non-zero element group.

[0169] The compression of the non-zero element group in each of the multiple first sub-matrices in the first direction can be understood as squeezing the non-zero element group in each of the multiple first sub-matrices toward the first direction, so that the non-zero element groups in the multiple first sub-matrices are sequentially spliced ​​together in the first direction.

[0170] The process of compressing the non-zero element groups in each of the multiple first submatrices along a first direction to obtain multiple second submatrices corresponding to the first matrix includes: for each of the multiple first submatrices, concatenating the non-zero element groups in the first submatrices along a first direction to obtain the second submatrices. Here, the first direction is the row direction of the first submatrices and the second direction is the column direction of the first submatrices, or the first direction is the column direction of the first submatrices and the second direction is the row direction of the first submatrices.

[0171] Since the second direction can be either row-oriented or column-oriented, the process of concatenating the non-zero elements of each first submatrix in the first direction to obtain the second submatrix varies depending on the circumstances. These processes will be described separately below.

[0172] In the first case, the first direction is the row direction, the second direction is the column direction, and the non-zero element group is a column vector containing non-zero elements (i.e., non-zero columns). The first processor 01 keeps the row numbers of each element in the non-zero element group of the first submatrix unchanged, and concatenates the non-zero element groups of the first submatrix in ascending order of column numbers to obtain the second submatrix. The number of columns in the second submatrix is ​​the same as the number of non-zero columns in the submatrix before concatenation, while the number of rows remains unchanged.

[0173] For example, please refer to Figure 4, which is a schematic diagram of a first matrix provided in an embodiment of this application. The first matrix is ​​an 8-row, 8-column matrix, and the non-zero elements of the first matrix are a, b, c, d, e, f, g, h, i, j, k, and m. The element value not shown in the first matrix is ​​0 (i.e., the blank cell in Figure 4 represents the zero element), and the first value is 4. With the second direction being the column direction, the rows of the first matrix are divided with a step size of 4 rows, thereby obtaining two 4-row, 8-column matrices as shown in Figure 5 (i.e., first submatrices 1 and 2). In submatrices 1, columns 2 and 4 are all zero columns, and columns 0, 1, 3, and 5 to 7 are non-zero columns. In submatrices 2, columns 0, 2, 4, and 6 are all zero columns, and columns 1, 3, 5, and 7 are non-zero columns. Keeping the row numbers of each element in the non-zero columns unchanged, the non-zero columns in the first submatrix are concatenated in ascending order of column numbers to obtain the second submatrix 1 and the second submatrix 2 as shown in Figure 6.

[0174] In the second case, the first direction is the column direction, the second direction is the row direction, and the non-zero element group is a row vector containing non-zero elements (i.e., non-zero rows). The first processor 01 keeps the column number of each element in the non-zero element group of the first submatrix unchanged, and concatenates the non-zero element groups of the first submatrix in ascending order of row number to obtain the second submatrix. The number of rows in the second submatrix is ​​the same as the number of non-zero rows in the submatrix before concatenation, while the number of columns remains unchanged.

[0175] Continuing with the first matrix shown in Figure 4 as an example, with the second direction being the row direction, the columns of the first matrix are divided with a step size of 4 columns, resulting in two 8x4 matrices (i.e., first submatrices 3 and 4) as shown in Figure 7. In submatrice 3, row 5 is all zeros, and rows 0 to 4, 6, and 7 are non-zero. In submatrice 4, rows 3, 5, and 7 are all zeros, and rows 0 to 2, 4, and 6 are non-zero. Keeping the column numbers of the elements in the non-zero rows unchanged, the non-zero rows in the first submatrices are concatenated in ascending order of row numbers to obtain the second submatrice 3 and the second submatrice 4 as shown in Figure 8.

[0176] Step 303: The first processor 01 divides each of the multiple second submatrices to obtain multiple third submatrices.

[0177] It is understandable that since the third submatrix is ​​obtained by partitioning the second submatrix, multiple third submatrixes are obtained by compressing the first matrix. There is no set of all-zero elements between any two non-zero element groups in the third submatrix. The non-zero element group is a vector along the second direction in the third submatrix, and there are elements with non-zero values ​​in the non-zero element group. The values ​​of all-zero elements are all zero in the set of all-zero elements. The second direction is either the row direction or the column direction.

[0178] When the second direction is a row direction, the first direction is a column direction; when the second direction is a column direction, the first direction is a row direction. Therefore, when the first direction is a row direction, the second direction is a column direction, and the non-zero element group is a column vector containing non-zero elements, there is no all-zero column vector between any two column vectors containing non-zero elements in the third submatrix, and the values ​​of the elements in the all-zero column vector are all zero; when the first direction is a column direction, the second direction is a row direction, and the non-zero element group is a row vector containing non-zero elements, there is no all-zero row vector between any two row vectors containing non-zero elements in the third submatrix, and the values ​​of the elements in the all-zero row vector are all zero.

[0179] Since the second direction can be either row-oriented or column-oriented, the process of dividing each second submatrix into multiple third submatrixes differs depending on the circumstances. These processes will be described separately below.

[0180] In the first case, the second direction is the column direction. The first processor 01 divides the columns of each of the multiple second submatrices to obtain multiple third submatrices.

[0181] In some embodiments, for any one of the plurality of second submatrices, the columns of the second submatrices are divided with a third value as the step size to obtain a plurality of third submatrices corresponding to the second submatrices. The column number of the plurality of third submatrices corresponding to the second submatrices is the third value, the number of rows of each third submatric is equal to the number of rows of the second submatric, and the sum of the column numbers of the plurality of third submatrices is equal to the number of columns of the second submatric.

[0182] In some embodiments, when the second direction is the column direction, if the number of columns of the second submatrix is ​​not a multiple of the third value, then before dividing the second submatrix, z columns of all zeros can be added to the second submatrix so that the number of columns of the second submatrix is ​​X times the third value, where X is the value obtained by dividing the number of columns of the second submatrix by the third value and rounding it up, and z is the difference between the value obtained by multiplying X by the third value and the number of columns of the second submatrix.

[0183] In other words, before dividing the columns of the second submatrix, it is necessary to ensure that the columns of the second submatrix are multiples of the third value. This ensures that the resulting third submatrix has a consistent number of rows and columns, thereby guaranteeing that the data structure of the compressed information corresponding to the third submatrix in the subsequent compression results is consistent, thus improving the accuracy and efficiency of subsequent matrix processing or analysis.

[0184] It should be noted that the third value is any value less than the number of columns in the first matrix. For example, to facilitate subsequent matrix multiplication operations, the third value is a positive integer multiple of N and less than the number of columns in the first matrix, where N is the number of columns in the matrix to be multiplied. For example, if N is 4 and the number of columns in the first matrix is ​​16, then the third value can be any value among 4, 8, and 12.

[0185] Continuing with the examples of the second submatrix 1 and second submatrix 2 shown in Figure 6, if the third value is 4 and the second direction is a column direction, since the number of columns in the second submatrix 1 is 6, which is not a multiple of 4, we add two columns of all zeros to the second submatrix 1. Then, we divide the columns of the second submatrix 1 with a step size of 4 columns, resulting in the third submatrix 1 and third submatrix 2 shown in Figure 9. Since the number of columns in the second submatrix 2 is 4, we can directly divide the columns of the second submatrix 2 with a step size of 4 columns, resulting in the third submatrix 3 shown in Figure 9.

[0186] In the second case, the second direction is the row direction. The first processor 01 divides the rows of each second submatrix in the multiple second submatrixes to obtain multiple third submatrixes.

[0187] In some embodiments, for any one of the plurality of second submatrices, the rows of the second submatrices are divided with a fourth value as the step size to obtain a plurality of third submatrices corresponding to the second submatrices. The number of rows in each of the plurality of third submatrices corresponding to the second submatrices is the fourth value, the number of columns in each third submatrices is equal to the number of columns in the second submatrices, and the sum of the number of rows in the plurality of third submatrices is equal to the number of rows in the second submatrices.

[0188] In some embodiments, when the second direction is the row direction, if the number of rows of the second submatrix is ​​not a multiple of the fourth value, then before dividing the second submatrix, C rows of all zeros can be added to the second submatrix so that the number of rows of the second submatrix is ​​Y times the fourth value, where Y is the value obtained by dividing the number of rows of the second submatrix by the fourth value and rounding it up, and C is the difference between the value of Y multiplied by the fourth value and the number of rows of the second submatrix.

[0189] In other words, before dividing the rows of the second submatrix, it is necessary to ensure that the number of rows of the second submatrix is ​​a multiple of the fourth value. This ensures that the resulting third submatrix has a consistent number of rows and columns, thereby guaranteeing that the data structure of the compressed information corresponding to the third submatrix in the subsequent compression results is consistent, thus improving the accuracy and efficiency of subsequent matrix processing or analysis.

[0190] It should be noted that the fourth value is any value less than the number of rows in the first matrix. For example, to facilitate subsequent matrix multiplication operations, the fourth value is a positive integer multiple of M and less than the number of rows in the first matrix, where M is the number of rows in the matrix to be multiplied. For example, if M is 4 and the first matrix has 16 rows, then the fourth value can be any value among 4, 8, and 12.

[0191] Continuing with the examples of the second submatrix 1 and 2 shown in Figure 8, if the fourth value is 4 and the second direction is the row direction, since the second submatrix 3 has 7 rows, which is not a multiple of 4, we add one row of all zeros to the second submatrix 3, and then divide the rows of the second submatrix 3 with a step size of 4, resulting in the third submatrix 1 and the third submatrix 2 shown in Figure 10. Since the second submatrix 4 has 5 rows, we add 3 rows of all zeros to the second submatrix 4, and then divide the rows of the second submatrix 4 with a step size of 4, resulting in the third submatrix 3 and the third submatrix 4 shown in Figure 10.

[0192] It should be noted that the first, second, third, and fourth values ​​in the embodiments of this application may be different, or two or more of them may be the same. The embodiments of this application do not limit this.

[0193] Step 304: The first processor 01 generates a compressed result of the first matrix based on multiple third sub-matrices. The compressed result indicates the value of the non-zero element in all third sub-matrices and the position of the non-zero element in the corresponding third sub-matrices.

[0194] In some embodiments, the compression result includes submatrix position information and submatrix element position information. The submatrix position information indicates the relative positions between multiple third submatrixes, and the submatrix element position information indicates the position of non-zero elements in the corresponding third submatrix.

[0195] In one possible implementation, the submatrix location information includes a submatrix row pointer and a submatrix column index. The submatrix row pointer indicates the number of third submatrixes in each row, and the submatrix column index indicates the column in which each third submatrix is ​​located.

[0196] In some embodiments, the submatrix row pointer includes multiple elements, and the difference between the value of the (i+1)th element and the value of the ith element is the number of the third submatrix in the ith row, where i is greater than or equal to 1 and less than or equal to L-1, and L is the total number of elements.

[0197] The process of determining the row pointer of a submatrix based on multiple third submatrices includes: setting the value of the first element in the row pointer of the submatrix to 0; letting i = 1, and determining the value of the (i+1)th element in the row pointer of the submatrix by summing the value of the i-th element in the row pointer of the submatrix with the number of submatrices in the i-th row of the intermediate result; if the intermediate result includes H rows of submatrices and i is less than H, then setting i = i+1, and returning to the previous step of determining the value of the (i+1)th element in the row pointer of the submatrix by summing the value of the i-th element in the row pointer of the submatrix with the number of submatrices in the i-th row of the intermediate result, until i is greater than or equal to H. The intermediate result is the arrangement of the multiple third submatrices obtained in the above manner, meaning that the arrangement of the multiple third submatrices in the intermediate result is the same as the arrangement of the multiple third submatrices in the first matrix.

[0198] For example, please refer to Figure 11, which is a schematic diagram of an intermediate result provided by an embodiment of this application. It is easy to see from Figure 11 that the number of submatrices in the first row (i.e., the row with row number 0) of the intermediate result is 2 (i.e., the first row includes two third submatrices). Therefore, the value of the first element in the submatrix row pointer is set to 0, and the value of the second element in the submatrix row pointer is set to 0 + 2 = 2. The number of submatrices in the second row (i.e., the row with row number 1) of the intermediate result is 1 (i.e., the second row includes one third submatrix). Therefore, the value of the second element in the submatrix row pointer is set to 2 + 1 = 3.

[0199] In some embodiments, the submatrix column index includes multiple elements, the number of which is the same as the number of third submatrixes. The value of each element indicates the column number of the corresponding third submatrix in the intermediate result. The first processor 01 determines the priority of the multiple third submatrixes based on their row and column numbers in the intermediate result, wherein the third submatrix with a smaller row number has a higher priority, and if the row numbers are the same, the third submatrix with a smaller column number has a higher priority. Based on the priority of the multiple third submatrixes, the processor determines the corresponding elements in the submatrix column index for each of the multiple third submatrixes, and based on their column numbers in the intermediate result, determines the values ​​of the corresponding elements in the submatrix column index for each of the multiple third submatrixes.

[0200] In one possible implementation, the column indices (i.e., column numbers) of each third submatrix are mapped sequentially from the first element to the last element of the submatrix column index, according to priority from high to low. Alternatively, the opposite can be true, that is, the column indices (i.e., column numbers) of each third submatrix are mapped sequentially from the first element to the last element of the submatrix column index, according to priority from low to high. This application does not limit this implementation.

[0201] The process of determining the values ​​of the elements corresponding to the column indices of the multiple third submatrices in the intermediate results based on the column indices of the multiple third submatrices includes: for any element in the column index of the submatrices, determining the column indices of the corresponding third submatrices in the intermediate results as the value of that element.

[0202] Taking Figure 11 as an example again, based on the row and column numbers of multiple third submatrices in the intermediate results, the priority of the multiple third submatrices is determined from high to low as follows: third submatrix 1, third submatrix 2, and third submatrix 3. Following the order of priority from high to low, the column indices (i.e., column numbers) of each third submatrix are mapped sequentially from the first element of the submatrix column index to the last element. Specifically, second submatrix 1 corresponds to the first element in the submatrix column index, and the value of this first element is the column number of third submatrix 1 (i.e., 0); second submatrix 2 corresponds to the second element in the submatrix column index, and the value of this second element is the column number of third submatrix 2 (i.e., 1); second submatrix 3 corresponds to the third element in the submatrix column index, and the value of this third element is the column number of third submatrix 3 (i.e., 0).

[0203] In some embodiments, the submatrix element position information includes a submatrix bitmap, which includes bitmaps corresponding to multiple third submatrixes respectively. The bitmaps indicate the positions of non-zero elements in the corresponding third submatrixes. Each bitmap includes multiple bits of data, and the number of bits is the same as the number of elements in the corresponding third submatrix.

[0204] In this context, a bitmap refers to a data structure whose basic idea is to use at least one binary bit to mark the information corresponding to a certain element. In the embodiments of this application, the bitmap corresponding to the third submatrix can be a 1-bit bitmap. This 1-bit bitmap is a special form of bitmap. Each element of the 1-bit bitmap corresponds to one binary bit, and the state of this binary bit (0 or 1) is used to represent some information, such as whether the data exists or not.

[0205] In some embodiments, the correspondence between the multiple third sub-matrices and the multiple bitmaps is determined based on the priority of the multiple third sub-matrices, and the value of each bit in the bitmap corresponding to each third sub-matrice is determined based on the correspondence between the multiple third sub-matrices and the multiple bitmaps.

[0206] For any three submatrix containing multiple elements, the first processor 01 determines the priority of the multiple elements based on their row and column numbers in the three submatrix; based on the priority of the multiple elements, it determines the bit of each of the multiple elements in the bitmap corresponding to the three submatrix; and based on the value of each of the multiple elements, it determines the value of the corresponding bit in the bitmap.

[0207] As an example, according to the priority of the multiple third submatrices from high to low, each third submatric is mapped sequentially from the first bitmap to the last bitmap of the submatric bitmap to obtain the correspondence between the multiple third submatrices and the multiple bitmaps.

[0208] When the second direction is column-oriented, elements with smaller row numbers have higher priority; when row numbers are the same, elements with smaller column numbers have higher priority. When the second direction is row-oriented, elements with smaller column numbers have higher priority; when column numbers are the same, elements with smaller row numbers have higher priority.

[0209] In one possible implementation, the first processor 01 maps each element from the least significant bit to the most significant bit of the bitmap in descending order of priority. In this case, the element with the highest priority corresponds to the least significant bit of the bitmap, and the element with the lowest priority corresponds to the most significant bit. Alternatively, the elements can be mapped from the least significant bit to the most significant bit in ascending order of priority. In this case, the element with the lowest priority corresponds to the least significant bit, and the element with the highest priority corresponds to the most significant bit.

[0210] The number of bits in the bitmap corresponding to the third submatrix is ​​the same as the number of elements in the third submatrix, and the bits in the bitmap correspond one-to-one with the elements in the third submatrix. In this case, the process by which the first processor 01 determines the values ​​of the bits corresponding to the multiple elements in the bitmap based on the values ​​of the multiple elements includes: for any one of the multiple elements, if the value of the element is 0, then the value of the bit corresponding to the element is determined to be 0; if the element is a non-zero value, then the value of the bit corresponding to the element is determined to be 1.

[0211] For example, please refer to Figure 12, which is a schematic diagram of a sub-matrix bitmap provided in an embodiment of this application. When the second direction is the column direction, each element is mapped sequentially from the lowest bit to the highest bit of the bitmap according to its priority from high to low. For example, for the first element in the third sub-matrix 3, i.e., the element in the 4th row and 0th column, this element is mapped to the lowest bit of the bitmap with a value of 1. For the second element in the third sub-matrix 3, i.e., the element in the 4th row and 1st column, this element is mapped to the second bit of the bitmap with a value of 0. The other elements in the third sub-matrix 3 are mapped in the same way, ultimately obtaining the bitmap 0x1A05 = 0001101000000101 corresponding to the third sub-matrix 3. Here, 0x1A05 means that in hexadecimal, 0001101000000101 can also be represented as 1A05.

[0212] In some embodiments, the compression result also includes non-zero element information, which includes the values ​​of non-zero elements in all third submatrices and the indices of the non-zero elements in the second direction of the first matrix.

[0213] In one possible implementation, the values ​​of non-zero elements in multiple third submatrices are placed in a first array, which is a one-dimensional array, and the positions of the non-zero elements in the same third submatrix within the first array are consecutive. Of course, the positions of the non-zero elements in the same third submatrix within the first array may not be consecutive; this embodiment does not limit this.

[0214] It is understandable that for multiple non-zero elements within the same third submatrix, the position of these non-zero elements in the first array can be determined by their priority. Similarly, for multiple non-zero elements in different third submatrixes, their position in the first array can be determined by the priority of the third submatrix. For example, within the same third submatrix, elements with higher priority rank earlier in the first array. In different third submatrixes, elements with higher priority rank earlier in the first array. Conversely, within the same third submatrix, elements with lower priority rank earlier in the first array. In different third submatrixes, elements with lower priority rank earlier in the first array. This application does not limit this specific approach.

[0215] In one possible implementation, the indices of non-zero elements in the second direction of the first matrix are placed in a second array, which is a one-dimensional array, and the indices of non-zero elements in the same third submatrix in the second direction of the first matrix are consecutive in the second array. Of course, the indices of non-zero elements in the same third submatrix in the second direction of the first matrix may not be consecutive in the second array, and this embodiment does not limit this.

[0216] If data is scattered across different locations in the array, additional data structures (such as hash tables or index trees) are needed to manage it, increasing the complexity of data management. However, this embodiment stores the values ​​and indices of non-zero elements consecutively within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the indices and values ​​of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine their positions, thus improving data access efficiency. Furthermore, since caches typically load recently accessed data items and their adjacent data items together, when the indices and values ​​of non-zero elements are stored consecutively in the array, these elements are more likely to be loaded consecutively into the cache, reducing cache misses and improving data access speed.

[0217] When the second direction is a column direction, the index of a non-zero element in the second direction of the first matrix is ​​the column index of the non-zero element in the first matrix (i.e., the column number where the non-zero element is located in the first matrix). When the second direction is a row direction, the index of a non-zero element in the second direction of the first matrix is ​​the row index of the non-zero element in the first matrix (i.e., the row number where the non-zero element is located in the first matrix).

[0218] It is understood that for multiple non-zero elements in the same third submatrix, the positions of their indices in the second array can be determined based on their priority. Similarly, for multiple non-zero elements in different third submatrixes, their positions in the second array can be determined based on the priority of the third submatrix. For example, within the same third submatrix, elements with higher priority have earlier indices in the second array. In different third submatrixes, elements with higher priority have earlier indices in the second array. Conversely, within the same third submatrix, elements with lower priority have earlier indices in the second array. In different third submatrixes, elements with lower priority have earlier indices in the second array. This embodiment does not limit the specific implementation of this method.

[0219] In some embodiments, elements at the same position in the first array and the second array correspond to the same non-zero element. For example, the first element in the first array is the value of the non-zero element 1, and the first element in the second array is the index of the non-zero element 1.

[0220] Since an array is a linear data structure in which elements are placed in a specific order, and each element in an array can be uniquely identified by its array index (i.e., position or sequence number), when the value and index of a non-zero element are located in the same position in two arrays, the corresponding value can be accessed from both arrays based on the same array index without additional lookup or mapping operations, thus effectively improving access efficiency.

[0221] In other embodiments, the Q-th element in the first array and the element at position P+1-Q in the second array correspond to the same non-zero element, where P is the total number of elements in the second array.

[0222] To facilitate understanding, we will continue with the examples in Figures 4-6, 9, 11, and 12 above, and further illustrate the compression result of the first matrix through examples. Please refer to Figure 13, which is a schematic diagram of the compression result of the first matrix provided in an embodiment of this application. The multiple bitmaps included in the submatrix bitmap are sorted in ascending order of row number and column number of the corresponding third submatrix, with row number having higher priority than column number. As shown in Figure 12, the submatrix bitmap includes bitmaps corresponding to third submatrixes 1-3 respectively. The row number and column number of third submatrix 1 are the smallest among the multiple third submatrixes. Therefore, the first bitmap in the submatrix bitmap is the bitmap corresponding to third submatrix 1. The row number of the third submatrix 2 is the smallest among all the third submatrixes, and its column number is greater than that of the third submatrix 1. Therefore, the second bitmap in the submatrix bitmap is the bitmap corresponding to the third submatrix 2. The row number and column number of the third submatrix 3 are the largest among all the third submatrixes. Therefore, the third bitmap in the submatrix bitmap is the bitmap corresponding to the third submatrix 3. Furthermore, for the explanation of submatrix row pointers and submatrix column indices, please refer to the relevant content in Figure 11 above. For the meaning and determination method of each bitmap included in the matrix bitmap, please also refer to the relevant content in Figure 12 above. Further explanation is not provided here.

[0223] In some embodiments, after obtaining the compression result of the first matrix, it may be necessary to further process the first matrix based on the compression result of the first matrix (such as determining the target matrix based on the first matrix and the second matrix, where the target matrix is ​​the product of the first matrix and the second matrix). Therefore, the first processor 01 can store the compression result of the first matrix in the memory 03 for subsequent retrieval.

[0224] In some embodiments, when the compression result in the first matrix is ​​stored in the memory 03, the first processor 01 may also delete the first matrix stored in the memory 03 to save memory space. This application embodiment does not limit this.

[0225] In some embodiments, the second processor 02 can process the first matrix based on the compression result of the first matrix. If the memory 03 includes a cache of the first processor 01, and the second processor 02 can access the cache of the first processor 01, then the first processor 01 can store the compression result of the first matrix in its own cache. When matrix multiplication is required, the second processor 02 can retrieve the compression result of the first matrix from the cache of the first processor 01, and determine the target matrix based on the compression result of the first matrix and the second matrix.

[0226] If memory 03 includes the cache of the first processor and the cache of the second processor 02, and the first processor 02 can access the cache of the second processor 01, then the first processor's cache stores the first matrix. Furthermore, after determining the compression result of the first matrix, the first processor 01 can copy the compression result of the first matrix stored in its own cache to the cache of the second processor 02. When matrix multiplication is required, the second processor 02's own cache also stores the second matrix, allowing the second processor to retrieve the compression result of the first matrix and the second matrix from its own cache, and determine the target matrix based on the compression result of the first matrix and the second matrix.

[0227] If memory 03 is a permanent storage medium (also known as non-volatile storage), such as a hard disk drive (HDD) or a solid-state drive (SSD), both the first processor 01 and the second processor 02 can access memory 03. In this case, the first processor 01 can copy the compressed result of the first matrix to memory 03. When matrix multiplication is required, the second processor 02 can retrieve the compressed result of the first matrix from memory 03 and determine the target matrix based on the compressed result of the first matrix and the second matrix.

[0228] It should be noted that the above description of the form of memory 03 and the interaction between the first processor 01, the second processor 02 and memory 03 are merely examples for better understanding the solutions of the embodiments of this application. The form of memory 03 and the interaction between the first processor 01, the second processor 02 and memory 03 can also be other forms, and the embodiments of this application do not limit them.

[0229] This embodiment of the application can first divide the first matrix to be compressed into smaller sub-matrices, and then compress the non-zero element groups in the resulting sub-matrices to obtain a second sub-matrix. This ensures that the second sub-matrix contains only non-zero element groups and no all-zero element groups, effectively eliminating a portion of the zero-valued elements in the first matrix. In this case, based on multiple third sub-matrices, the compression result of the first matrix can effectively reduce the number of zero elements that need to be stored, thereby saving data storage space. Furthermore, in scenarios requiring matrix multiplication, the matrix can be divided according to computational needs to obtain sub-matrices of suitable size for computation, thereby effectively improving computational efficiency.

[0230] Furthermore, in this embodiment, the values ​​and indices of non-zero elements are stored consecutively according to their positions within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the indices and values ​​of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine the positions of other non-zero elements in the array, thus improving data access efficiency. Moreover, since caches typically load recently accessed data items and their adjacent data items together, when the indices and values ​​of non-zero elements are stored consecutively in the array, these elements are more likely to be loaded consecutively into the cache, thereby reducing cache misses and improving data access speed. Because an array is a linear data structure where elements are stored in a specific order, and each element in the array can be uniquely identified by its array index (i.e., position or sequence number), in this embodiment, when the values ​​and indices of non-zero elements are directly stored at the same position in two arrays, the corresponding values ​​can be accessed from both arrays based on the same array index without additional lookup or mapping operations, thus effectively improving access efficiency.

[0231] This application also provides a matrix processing method. The method is applied to a computer system, which includes a second processor 02 and a memory 03 as shown in Figure 1 or Figure 2. The matrix processing method is executed by the second processor 02 and includes the following steps.

[0232] Step 1: The second processor 02 determines the target matrix based on the compression result of the first matrix and the second matrix stored in memory 03. The target matrix is ​​the product of the first matrix and the second matrix. The first matrix represents input information or intermediate results during model training or inference. The compression result of the first matrix is ​​generated based on multiple third sub-matrices. The compression result indicates the values ​​of non-zero elements in all third sub-matrices and their positions within their respective sub-matrices. Multiple third sub-matrices are obtained by partitioning each of the multiple second sub-matrices. Multiple second sub-matrices are obtained by compressing the non-zero element groups in each of the multiple first sub-matrices along a first direction. Multiple first sub-matrices are obtained by partitioning the first matrix. The non-zero element groups are vectors in the first sub-matrices along a second direction, and each non-zero element group contains elements with non-zero values. In other words, based on the compression result of the first matrix and the second matrix, matrix multiplication is performed on the first and second matrices to obtain the target matrix.

[0233] It should be noted that the input information or intermediate results in the model training or inference process are usually in the form of tensors. A tensor is a multidimensional array that can represent multidimensional data and complex multilinear relationships. The order (or rank) of a tensor refers to the number of dimensions represented by the tensor, including zero-order tensors (i.e., scalars), first-order tensors (i.e., vectors), second-order tensors (i.e., matrices), and third-order and higher-order tensors. Therefore, the input information or intermediate results in the model training or inference process in this application embodiment can be second-order tensors (i.e., matrices) or third-order and higher-order tensors. When the input information or intermediate results in the model training or inference process are second-order tensors, the first matrix is ​​the second-order tensor. When the input information or intermediate results in the model training or inference process are third-order and higher-order tensors, slicing operations (i.e., selecting specific dimensions to slice the tensor), reshaping operations (i.e., reshaping a high-dimensional tensor into a low-dimensional matrix), etc., can be performed on the third-order and higher-order tensors to obtain at least one matrix. In this case, any one of the at least one matrices is determined as the first matrix.

[0234] It should also be noted that matrix multiplication is only meaningful when the number of columns in the first matrix is ​​the same as the number of rows in the second matrix. For example, matrix A is shown in formula (1) below, matrix B is shown in formula (2) below, and the product AB of matrix A and matrix B is shown in formula (3) below.

[0235] In this embodiment, the second direction is either a row direction or a column direction. When the second direction is a row direction, the first direction is a column direction; when the second direction is a column direction, the first direction is a row direction. Furthermore, the method for determining the compression result of the first matrix is ​​detailed in the above embodiments and will not be repeated here.

[0236] In some embodiments, the target matrix is ​​determined according to the following steps (1)-(4).

[0237] (1) The second processor 02 restores multiple third sub-matrices based on the compression result of the first matrix.

[0238] In some embodiments, the compression result includes position indication information and the values ​​of non-zero elements in a plurality of third submatrices, wherein the position indication information indicates the relative positions between the plurality of third submatrices and the position of the non-zero elements in each third submatric.

[0239] In one possible implementation, the position indication information includes submatrix position information, submatrix element position information, and non-zero element information. The submatrix position information indicates the relative positions between multiple third submatrixes, and the submatrix element position information indicates the position of the non-zero element in the corresponding third submatrix. The non-zero element information includes the values ​​of all non-zero elements in the third submatrixes and the index of the non-zero element in the second direction of the first matrix.

[0240] In some embodiments, the submatrix position information includes a submatrix row pointer and a submatrix column index, wherein the submatrix row pointer indicates the number of third submatrixes in each row, and the submatrix column index indicates the column in which each third submatrix is ​​located. The submatrix element position information includes a submatrix bitmap, which indicates the position of non-zero elements in each third submatrix.

[0241] The second processor 02 determines the positions of multiple third submatrices in the intermediate result based on the submatric row pointers and column indices. Based on these positions, the submatric bitmap, and the values ​​of the non-zero elements in the multiple third submatrices, it reconstructs the multiple third submatrices. The intermediate result includes H rows of submatrices, and the intermediate result is an arrangement of the multiple third submatrices obtained in the above manner; that is, the arrangement of the multiple third submatrices in the intermediate result is the same as their arrangement in the first matrix.

[0242] Determining the position of multiple third submatrices in the intermediate result based on the submatric row pointer and submatric column index includes: determining the number of third submatrices included in each row of the intermediate result based on the submatric row pointer, and determining the position of multiple third submatrices in the intermediate result based on the submatric column index and the number of third submatrices included in each row of the intermediate result.

[0243] In some embodiments, the submatrix row pointer includes multiple elements, and the difference between the value of the (i+1)th element and the value of the ith element is the number of third submatrices in the ith row, where i is greater than or equal to 1 and less than or equal to L-1, and L is the total number of elements. In this case, for the ith row in the intermediate result, the difference between the value of the (i+1)th element and the value of the ith element in the submatrix pointer is determined as the number of third submatrices included in the ith row of the intermediate result.

[0244] In some embodiments, the submatrix column index includes multiple elements, the number of which is the same as the number of third submatrices. Each element's value indicates the column number of the corresponding third submatrix in the intermediate result. In this case, let j = 1 and n = 0. Starting from the (n+1)th element of the submatrix column index, retrieve the values ​​of V elements sequentially, where V is the number of third submatrices included in the j-th row of the intermediate result. Based on the values ​​of these V elements, determine the column containing the third submatrix in the j-th row of the intermediate result. Let j = j + 1 and n = n + V. If j is not greater than the number of rows in the intermediate result, return to the previous step of retrieving the values ​​of V elements sequentially starting from the first element of the submatrix column index, until j is greater than the number of rows in the intermediate result. The number of rows in the intermediate result is one less than the number of elements included in the submatrix row pointer.

[0245] The process of determining the column number of the third submatrix in the j-th row of the intermediate result based on the values ​​of V elements includes: for any one of the V values, determining that value as the column number of a third submatrix in the j-th row. This process is repeated for each of the V values ​​to obtain the column number of each third submatrix in the j-th row.

[0246] In some embodiments, the submatrix bitmap includes bitmaps corresponding to multiple third submatrices, each bitmap indicating the position of a non-zero element in the corresponding third submatrix. Each bitmap includes multiple bits of data, the number of which is the same as the number of elements in the corresponding third submatrix.

[0247] The process of restoring multiple third submatrices based on their positions in the intermediate results, the submatric bitmaps, and the values ​​of non-zero elements in the multiple third submatrices includes: determining the priority of the multiple third submatrices based on their positions in the intermediate results, where the third submatrices with smaller row numbers have higher priority, and when the row numbers are the same, the third submatrices with smaller column numbers have higher priority; determining the correspondence between the multiple third submatrices and multiple bitmaps based on their priorities; and restoring the multiple third submatrices based on the correspondence between the multiple third submatrices and multiple bitmaps, as well as the values ​​of non-zero elements in the multiple third submatrices.

[0248] In some embodiments, according to the priority of the multiple third submatrices from high to low, each third submatric is sequentially mapped from the first bitmap of the submatric bitmap to the last bitmap, thereby obtaining the correspondence between the multiple third submatrices and the multiple bitmaps.

[0249] Based on the correspondence between multiple third submatrices and multiple bitmaps, and the values ​​of non-zero elements in the multiple third submatrices, the process of reconstructing multiple third submatrices includes: for any multiple elements in any third submatric, determining the priority of the multiple elements based on their row and column numbers; determining the bit positions of the multiple elements in the corresponding bitmap based on their priority; and determining the number of non-zero elements based on the corresponding bit positions and the values ​​of each bit in the bitmap. This process is repeated for each third submatric to obtain the number of non-zero elements in each submatric. Based on the number and values ​​of the non-zero elements in each third submatric, the multiple third submatrices are reconstructed.

[0250] When the second direction is column-oriented, elements with smaller row numbers have higher priority; when row numbers are the same, elements with smaller column numbers have higher priority. When the second direction is row-oriented, elements with smaller column numbers have higher priority; when column numbers are the same, elements with smaller row numbers have higher priority.

[0251] In one possible implementation, elements are mapped sequentially from the least significant bit to the most significant bit of the bitmap, according to their priority from highest to lowest. In this case, the highest priority element corresponds to the least significant bit, and the lowest priority element corresponds to the most significant bit. Alternatively, elements can be mapped sequentially from the least significant bit to the most significant bit, according to their priority from lowest to highest. In this case, the lowest priority element corresponds to the least significant bit, and the highest priority element corresponds to the most significant bit.

[0252] In some embodiments, the number of bits in the bitmap corresponding to the third submatrix is ​​the same as the number of elements in the third submatrix, and the bits in the bitmap correspond one-to-one with the elements in the third submatrix. In this case, the process of determining the number of non-zero elements among the multiple elements based on the bits corresponding to each element in the bitmap and the values ​​of each bit in the bitmap includes: for any element among the multiple elements, if the value of the bit corresponding to that element is 0, then the value of that element is determined to be 0; if the value of the bit corresponding to that element is 1, then that element is determined to be a non-zero element. By processing each element in the multiple elements in the same way, the number of non-zero elements among the multiple elements can be determined.

[0253] In one possible implementation, the values ​​of non-zero elements in multiple third submatrices are placed in a first array, which is a one-dimensional array, and the positions of the non-zero elements in the same third submatrix within the first array are consecutive. Of course, the positions of the non-zero elements in the same third submatrix within the first array may not be consecutive; this embodiment does not limit this. The following description uses the example of consecutive positions of the non-zero elements in the same third submatrix within the first array as an example.

[0254] If data is scattered across different locations in the array, additional data structures (such as hash tables or index trees) are needed to manage it, increasing the complexity of data management. However, this embodiment stores the values ​​of non-zero elements consecutively according to their positions within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the values ​​of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine their positions, thus improving data access efficiency. Furthermore, since caches typically load recently accessed data items and their adjacent data items together, when the values ​​of non-zero elements are stored consecutively in the array, these elements are more likely to be loaded consecutively into the cache, thereby reducing cache misses and improving data access speed.

[0255] For multiple non-zero elements within the same third submatrix, their positions in the first array are determined by their priority. Similarly, for multiple non-zero elements in different third submatrixes, their positions are determined by the priority of the third submatrix itself. For example, within the same third submatrix, elements with higher priority rank earlier in the first array. In different third submatrixes, elements with higher priority rank earlier in the first array. Conversely, within the same third submatrix, elements with lower priority rank earlier in the first array. In different third submatrixes, elements with lower priority rank earlier in the first array.

[0256] The following section will introduce the implementation process of reconstructing multiple third submatrices based on the number of non-zero elements in each third submatric and the values ​​of non-zero elements in multiple third submatrices, taking the example that in the same third submatric, the higher the priority of the element, the earlier its position in the array, and in different third submatrices, the higher the priority of the third submatric, the earlier its position.

[0257] Let r = 1 and t = 0. Starting from the (t+1)th element of the first array, sequentially retrieve the values ​​of B elements, where B is the number of non-zero elements in the r-th third submatrix. Place the values ​​of these B elements in the positions of the non-zero elements in the r-th third submatrix according to their priority from highest to lowest. Set all values ​​in the r-th third submatrix except for the non-zero elements to 0. Let r = r+1 and t = t+B. If r is not greater than the number of third submatrixes, return to the previous step of sequentially retrieving the values ​​of B elements starting from the (t+1)th element of the first array, until r is greater than the number of third submatrixes.

[0258] (2) The second processor 02 determines the position of each element in the first matrix in the multiple third submatrices.

[0259] In some embodiments, the compression result also includes the index of the non-zero element in the second direction of the first matrix. In this case, the second processor 02 concatenates the restored third sub-matrices according to their positions in the intermediate result to obtain the intermediate result. Based on the intermediate result and the index of the non-zero element in the second direction of the first matrix, the position of each element in the multiple third sub-matrices in the first matrix is ​​determined.

[0260] The process of determining the position of each element in a plurality of third submatrices in the first matrix based on intermediate results and the indices of non-zero elements in the second direction of the first matrix includes: determining the position of each element in a plurality of third submatrices in the first direction of the first matrix based on intermediate results; determining the index of non-zero elements in the second direction of the first matrix based on the indices of non-zero elements in the second direction of the first matrix; and determining the position of each element in a plurality of third submatrices in the second direction of the first matrix based on the indexes of non-zero elements in the second direction of the first matrix.

[0261] Since matrix multiplication includes left multiplication and right multiplication, the left multiplication of the first matrix by the second matrix and the right multiplication of the first matrix by the second matrix are different. In the case of the left multiplication of the first matrix by the second matrix, the second direction mentioned above is the row direction; in the case of the right multiplication of the first matrix by the second matrix, the second direction mentioned above is the column direction. In different cases, the implementation process of determining the position of each element in the first direction of the first matrix based on the intermediate result is different, which will be introduced separately below.

[0262] In the first case, the second direction is the column direction and the first direction is the row direction. For any third submatrix among multiple third submatrixes, the second processor 02 determines the row number of each element in the intermediate result of the third submatrix as the row number of the element in the first matrix.

[0263] In the second case, the second direction is the row direction and the first direction is the column direction. For any third submatrix among multiple third submatrixes, the second processor 02 determines the column number of each element in the intermediate result of the third submatrix as the column number of the element in the first matrix.

[0264] In some embodiments, the indices of non-zero elements in the second direction of the first matrix are placed in a second array, which is a one-dimensional array, and the indices of non-zero elements in the same third submatrix in the second direction of the first matrix are consecutive in the second array. Of course, the indices of non-zero elements in the same third submatrix may not be consecutive in the first array, and this embodiment does not limit this. The following description uses the example of consecutive indices of non-zero elements in the second direction of the first matrix in the second array as an example.

[0265] If data is scattered across different locations in the array, additional data structures (such as hash tables or index trees) are needed to manage it, increasing the complexity of data management. However, this embodiment stores the indices of non-zero elements consecutively according to their positions within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the indices of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine their positions, thus improving data access efficiency. Furthermore, since caches typically load recently accessed data items and their adjacent data items together, when the indices of non-zero elements are stored consecutively in the array, these elements are more likely to be loaded consecutively into the cache, thereby reducing cache misses and improving data access speed.

[0266] In some embodiments, elements at the same position in the first array and the second array correspond to the same non-zero element. For example, the first element in the first array is the value of the non-zero element 1, and the first element in the second array is the index of the non-zero element 1.

[0267] Because an array is a linear data structure where elements are placed in a specific order, and each element can be uniquely identified by its array index (i.e., position or sequence number), when the value and index of a non-zero element are located in the same position in two arrays, the corresponding value can be accessed from both arrays based on the same array index without additional lookup or mapping operations, thus effectively improving access efficiency.

[0268] For multiple non-zero elements within the same third submatrix, their positions in the second array can be determined by their priority. Similarly, for multiple non-zero elements in different third submatrixes, their positions can be determined by the priority of the third submatrix itself. For example, within the same third submatrix, elements with higher priority have earlier indices in the second array. In different third submatrixes, higher priority elements have earlier indices in the second array. Conversely, within the same third submatrix, lower priority elements have earlier indices in the second array. In different third submatrixes, lower priority elements have earlier indices in the second array.

[0269] The following section will introduce the implementation process of determining the index of non-zero elements in the second direction of multiple third submatrices based on the index of non-zero elements in the second direction of the first matrix, taking the example that in the same third submatric, the value of an element with higher priority is placed earlier in the array, and in different third submatrices, the value of an element with higher priority is placed earlier.

[0270] Let w = 1 and e = 0. Starting from the (e+1)th element of the first array, sequentially retrieve the values ​​of G elements, where G is the number of non-zero elements in the w-th third submatrix. Following the order of priority from highest to lowest, assign the values ​​of these G elements as the indices of the non-zero elements in the w-th third submatrix in the second direction. Let w = w+1 and e = e+G. If w is not greater than the number of third submatrixes, return to the previous step of sequentially retrieving the values ​​of G elements starting from the (e+1)th element of the first array, until w is greater than the number of third submatrixes.

[0271] When the second direction is a column direction, the index of a non-zero element in the second direction of the first matrix is ​​the column index of the non-zero element in the first matrix (i.e., the column number where the non-zero element is located in the first matrix). When the second direction is a row direction, the index of a non-zero element in the second direction of the first matrix is ​​the row index of the non-zero element in the first matrix (i.e., the row number where the non-zero element is located in the first matrix).

[0272] The implementation process of determining the position of each element in the second direction of the first matrix based on the index of the second direction of the non-zero elements in the multiple third sub-matrices differs depending on the second direction. These processes will be introduced separately below.

[0273] In the first case, the second direction is the column direction and the first direction is the row direction. For any element in any third submatrix among multiple third submatrixes, the second processor 02 determines the column number of the non-zero element in the column where the element is located in the third submatrix as the column number of the element.

[0274] In the second case, the second direction is the row direction and the first direction is the column direction. For any element in any third submatrix among multiple third submatrixes, the second processor 02 determines the row number of the non-zero element in the column where the element is located as the row number of the element.

[0275] (3) The second processor 02 determines the calculation matrix corresponding to each of the multiple third sub-matrices from the second matrix based on the position of each element in the first matrix.

[0276] In some embodiments, the number of rows of the third submatrix is ​​a positive integer multiple of M, the number of columns is a positive integer multiple of N, and M×N is the size of the matrix for matrix multiplication. Both M and N are positive integers. In this case, the multiple third submatrixes are divided to obtain multiple first computation matrices, and the size of each first computation matrix is ​​M×N. Based on the position of each element in the multiple first computation matrices in the first matrix, the second computation matrices corresponding to the multiple first computation matrices are determined from the second matrix.

[0277] In one possible implementation, the size of the third submatrix is ​​M×N. In this case, the second processor 02 can directly determine multiple third submatrices as multiple first computation matrices. When the size of the third submatrix is ​​the size of the matrix used for matrix multiplication, it indicates that the size of the third submatrix matches the standard size of matrix multiplication, and the compression result is stored in units of the third submatrix. Therefore, when performing matrix multiplication, it is only necessary to retrieve the data corresponding to the third submatrix from the compressed result, without frequent access to external storage or memory swapping, thereby improving the computational efficiency of matrix multiplication.

[0278] In some embodiments, the position of each element in the plurality of first calculation matrices in the first matrix includes the row number and column number of the element in the first matrix; the process of determining the second calculation matrix corresponding to the plurality of first calculation matrices from the second matrix based on the position of each element in the plurality of first calculation matrices includes: determining the element in the second matrix whose column number is the same as the row number of the target element and whose row number is the same as the column number of the target element as the element in the second calculation matrix corresponding to the target calculation matrix; wherein, the target calculation matrix is ​​any one of the plurality of first calculation matrices, and the target element is any one element of the target calculation matrix.

[0279] (4) The second processor 02 multiplies the multiple third submatrices with their respective computation matrices to obtain the target matrix.

[0280] In some embodiments, multiple first computation matrices are multiplied by their respective corresponding second computation matrices to obtain the target matrix.

[0281] Since matrix multiplication includes left and right multiplication, the methods for multiplying multiple third submatrices with their respective computational matrices to obtain the target matrix differ depending on the circumstances. These methods will be described separately below. Furthermore, for ease of description, if the first matrix is ​​right-multiplied by the second matrix, the first matrix is ​​called the left matrix and the second matrix is ​​called the right matrix; if the first matrix is ​​left-multiplied by the second matrix, the first matrix is ​​called the right matrix and the second matrix is ​​called the left matrix.

[0282] In the first scenario, the second processor 02 includes a tensor computation unit used to perform matrix multiplication. The first matrix is ​​the left matrix, and the second matrix is ​​the right matrix. For any one of the multiple first computation matrices, the second processor 02 loads the first computation matrix into the on-chip register corresponding to the left matrix, and loads the corresponding second computation matrix into the on-chip register corresponding to the right matrix. Then, the tensor computation unit multiplies the first computation matrix with the corresponding second computation matrix to obtain the target submatrix. Processing each first computation matrix in the same way yields multiple target submatrices. Based on the positions of the corresponding first computation matrices in the intermediate results, the multiple target submatrices are concatenated to obtain the target matrix.

[0283] For example, when the second direction is the column direction, please refer to Figure 14, which is a schematic diagram of a matrix processing method provided in an embodiment of this application. As shown in Figure 14, the first matrix is ​​the left matrix, the second matrix is ​​the right matrix, and the size of the third submatrix is ​​M×N, that is, the size of the third submatrix is ​​the size of the matrix to be used for matrix multiplication (i.e., the size required by the tensor computation unit). According to the storage order of the third submatrix (i.e., the row number and column number of the third submatrix in ascending order), the bitmaps of multiple third submatrixes are obtained sequentially. For the third submatrix 1 among multiple third submatrixes, count the number 'a' of non-zero elements in the bitmap corresponding to the third submatrix 1 (i.e., a = 5). Then, read 5 values ​​(i.e., a, b, c, e, g) from the non-zero elements in the multiple third submatrixes in order from front to back, and load them into the on-chip register corresponding to the left matrix. Then, read 5 column indices (also called column numbers, i.e., 1, 5, 3, 0, 1) from the global column index values ​​of the non-zero elements in order. Obtain the row data corresponding to these 5 column numbers from the right matrix (i.e., the second matrix). That is, determine the data corresponding to the 5 row numbers in the right matrix that are the same as 'a' column numbers as the row data corresponding to the 5 column numbers, which is the data in rows 0, 1, 3, and 5 of the second matrix. Obtain column b data (i.e., the data in columns 0-3) from the data in rows 0, 1, 3, and 5 of the second matrix in order from the smallest to the largest column number. b is equal to the number of rows of the third submatrix. The data located in rows 0, 1, 3, and 5 and columns 0-3 of the second matrix are loaded into the on-chip register corresponding to the right matrix. Then, the tensor computation unit performs calculations on the two sub-matrix blocks and outputs the calculation result (i.e., the target sub-matrix).

[0284] In the second scenario, the second processor 02 includes a tensor computation unit used to perform matrix multiplication. The first matrix is ​​the right matrix, and the second matrix is ​​the left matrix. For any one of the multiple first computation matrices, the second processor 02 loads the first computation matrix into the on-chip register corresponding to the right matrix, and loads the corresponding second computation matrix into the on-chip register corresponding to the left matrix. Then, the tensor computation unit multiplies the first computation matrix with the corresponding second computation matrix to obtain the target submatrix. Processing each first computation matrix in the same way yields multiple target submatrices. Based on the positions of the corresponding first computation matrices in the intermediate results, the multiple target submatrices are concatenated to obtain the target matrix.

[0285] As another example, when the second direction is the row direction, please refer to Figure 15, which is a schematic diagram of a matrix processing method provided in an embodiment of this application. As shown in Figure 15, the first matrix is ​​the right matrix, the second matrix is ​​the left matrix, and the size of the third submatrix is ​​M×N, that is, the size of the third submatrix is ​​the size of the matrix to be used for matrix multiplication (i.e., the size required by the tensor computation unit). According to the storage order of the third submatrix (i.e., the row number of the third submatrix in ascending order, and the column number in ascending order), the bitmaps of multiple third submatrixes are obtained sequentially. For the third submatrix 1 among multiple third submatrixes, count the number 'a' of non-zero elements in the bitmap corresponding to the third submatrix 1 (i.e., a = 5). Then, read 4 values ​​(e, a, g, c) from the non-zero elements in the multiple third submatrixes in order from front to back, and load them into the on-chip register corresponding to the right matrix. Then, read 4 row indices (also called row numbers, i.e., 2, 0, 3, 1) from the global row index values ​​of the non-zero elements in order. Obtain the column data corresponding to these 5 row numbers from the left matrix (i.e., the second matrix). That is, determine the data corresponding to the 5 column numbers with the same 5 row numbers in the left matrix as the column data corresponding to the 5 row numbers, which is the data in columns 0, 1, 2, and 3 of the second matrix. Obtain b rows of data (i.e., the data in rows 0-3) from the data in rows 0, 1, 2, and 3 of the second matrix in order from the smallest to the largest row number. b is equal to the number of columns of the third submatrix. The data located in columns 0, 1, 2, and 3 and rows 0-3 of the second matrix are loaded into the on-chip register corresponding to the left matrix. Then, the tensor computation unit performs calculations on the two sub-matrix blocks and outputs the calculation result (i.e., the target sub-matrix).

[0286] In some embodiments, before determining the target matrix based on the compression result of the first matrix and the second matrix, the second processor 02 can also obtain information about the compression result of the first matrix and the second matrix, and determine the second matrix based on the information about the second matrix.

[0287] For example, the compressed result of the first matrix is ​​input by the user into the second processor 01, or retrieved by the second processor 01 from the memory 03.

[0288] In one possible implementation, memory 03 includes a cache of a first processor 01, which determines the compression result of the first matrix, and a second processor 02 is able to access the cache of the first processor 01. In this case, after determining the compression result of the first matrix, the first processor 01 can store the compression result of the first matrix in its own cache, and then the second processor 02 can retrieve the compression result of the first matrix from the cache of the first processor 01.

[0289] In another possible implementation, memory 03 includes a cache for the first processor and a cache for the second processor 02, and the first processor 02 can access the cache for the second processor 01. In this case, the cache of the first processor stores the first matrix. Furthermore, after determining the compression result of the first matrix, the first processor 01 can copy the compression result of the first matrix stored in its own cache to the cache of the second processor 02. When matrix multiplication is required, the cache of the second processor 02 also stores the second matrix, allowing the second processor to retrieve the compression result of the first matrix and the second matrix from its own cache, and determine the target matrix based on the compression result of the first matrix and the second matrix.

[0290] In another possible implementation, memory 03 is a permanent storage medium, accessible to both the first processor 01 and the second processor 03. The first processor 01 determines the compression result of the first matrix. In this case, after determining the compression result of the first matrix, the first processor 01 copies the compression result to memory 03, and then the second processor 02 retrieves the compression result of the first matrix from its own cache.

[0291] It should be noted that the above description of the form of memory 03 and the interaction between the first processor 01, the second processor 02 and memory 03 are merely examples for better understanding the solutions of the embodiments of this application. The form of memory 03 and the interaction between the first processor 01, the second processor 02 and memory 03 can also be other forms, and the embodiments of this application do not limit them.

[0292] For example, the information of the second matrix may be input by the user into the second processor 01, or it may be pre-stored in the second processor 01; this embodiment of the application does not limit this. The information of the second matrix may be the original data of the second matrix, such as the position of each element in the second matrix and the value of the element at the corresponding position. The information of the first matrix may also be the information of the second matrix represented by a relevant compression format, for example, the format of the second matrix information may be CSR. In this case, the second matrix can be restored based on the information of the second matrix to obtain the original data of the second matrix. It should be noted that the detailed process of restoring the second matrix based on the information of the second matrix is ​​a technique well known to those skilled in the art, and this embodiment of the application will not elaborate on it.

[0293] This embodiment of the application can first divide the first matrix to be compressed into smaller sub-matrices, and then compress the non-zero element groups in the resulting sub-matrices to obtain a second sub-matrix. This ensures that the second sub-matrix contains only non-zero element groups and no all-zero element groups, effectively eliminating a portion of the zero-valued elements in the first matrix. In this case, based on multiple third sub-matrices, the compression result of the first matrix can effectively reduce the number of zero elements that need to be stored, thereby saving data storage space. Furthermore, in scenarios requiring matrix multiplication, the matrix can be divided according to computational needs to obtain sub-matrices of suitable size for computation, thereby effectively improving computational efficiency.

[0294] Furthermore, in this embodiment, the values ​​and indices of non-zero elements are stored consecutively according to their positions within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the indices and values ​​of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine the positions of other non-zero elements in the array, thus improving data access efficiency. Moreover, since caches typically load recently accessed data items and their adjacent data items together, when the indices and values ​​of non-zero elements are stored consecutively in the array, these elements are more likely to be loaded consecutively into the cache, thereby reducing cache misses and improving data access speed. Because an array is a linear data structure where elements are placed in a specific order, and each element in the array can be uniquely identified by its array index (i.e., position or sequence number), when the values ​​and indices of non-zero elements are located in the same position in two arrays, the corresponding values ​​can be accessed from both arrays based on the same array index without additional lookup or mapping operations, thus effectively improving access efficiency. Furthermore, when performing matrix multiplication, the matrix processing method provided in this application can perform operations on the data in the third submatrix, thereby avoiding a large number of zero-element calculations and improving the computational efficiency of matrix multiplication. When the size of the third submatrix is ​​the same as the size of the matrix used for matrix multiplication, it means that the size of the third submatrix matches the size required for matrix multiplication. Since the compression result is stored in units of the third submatrix, when performing matrix multiplication, it is only necessary to retrieve the data corresponding to the third submatrix from the compression result, without frequent access to external storage or memory swapping, thus improving the computational efficiency of matrix multiplication.

[0295] Figure 16 is a schematic diagram of a matrix compression device provided in an embodiment of this application. The matrix compression device can be implemented by software, hardware, or a combination of both as part or all of the first processor 01 described above. Referring to Figure 16, the device includes: a first partitioning module 1601, a compression module 1602, a second partitioning module 1603, and a generation module 1604.

[0296] The first partitioning module 1601 is used to partition the first matrix stored in the memory to obtain multiple first sub-matrices. The first matrix represents input information or intermediate results during model training or inference. For detailed implementation details, please refer to the corresponding content in the above embodiments; they will not be repeated here.

[0297] Compression module 1602 is used to compress the non-zero element group in each of the multiple first sub-matrices along a first direction to obtain multiple second sub-matrices corresponding to the first matrix. The non-zero element group is a vector in the first sub-matrices along a second direction, and there are elements with non-zero values ​​in the non-zero element group. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0298] The second partitioning module 1603 is used to partition each of the multiple second submatrices to obtain multiple third submatrices. For detailed implementation processes, please refer to the corresponding contents in the above embodiments; they will not be repeated here.

[0299] The generation module 1604 is used to generate a compressed result of the first matrix based on multiple third sub-matrices. The compressed result indicates the values ​​of non-zero elements in all third sub-matrices and the positions of the non-zero elements in their respective third sub-matrices. For detailed implementation processes, please refer to the corresponding contents in the above embodiments, which will not be repeated here.

[0300] In one possible implementation, the compression module 1602 is specifically used for:

[0301] For each of the multiple first submatrices, the non-zero element groups in the first submatrices are concatenated in the first direction to obtain the second submatrices, where the first direction is the row direction of the first submatrices and the second direction is the column direction of the first submatrices, or the first direction is the column direction of the first submatrices and the second direction is the row direction of the first submatrices.

[0302] In one possible implementation, the compression module 1602 is specifically used for:

[0303] Given that the first direction is the row direction, the second direction is the column direction, and the non-zero element group is a column vector containing non-zero elements, keep the row number of each element in the non-zero element group of the first submatrix unchanged, and concatenate the non-zero element groups of the first submatrix in ascending order of column number to obtain the second submatrix.

[0304] Given that the first direction is the column direction, the second direction is the row direction, and the non-zero element group is a row vector containing non-zero elements, keep the column number of each element in the non-zero element group of the first submatrix unchanged, and concatenate the non-zero element groups of the first submatrix in ascending order of row number to obtain the second submatrix.

[0305] In one possible implementation, the first direction is the row direction, the second direction is the column direction, and the non-zero element group is a column vector containing non-zero elements; in the third submatrix, there are no all-zero column vectors between any two column vectors containing non-zero elements, and the values ​​of the elements in the all-zero column vectors are all zero; or

[0306] The first direction is the column direction, the second direction is the row direction, and the non-zero element group is a row vector containing non-zero elements; in the third submatrix, there is no all-zero row vector between any two row vectors containing non-zero elements, and the values ​​of the elements in the all-zero row vector are all zero.

[0307] In one possible implementation, the compression result includes submatrix position information and submatrix element position information. The submatrix position information indicates the relative positions between multiple third submatrixes, and the submatrix element position information indicates the position of non-zero elements in the corresponding third submatrix.

[0308] In one possible implementation, the submatrix location information includes a submatrix row pointer and a submatrix column index. The submatrix row pointer indicates the number of third submatrixes in each row, and the submatrix column index indicates the column in which each third submatrix is ​​located.

[0309] In one possible implementation, the submatrix element position information includes a submatrix bitmap, which includes bitmaps corresponding to multiple third submatrixes, and the bitmaps indicate the positions of non-zero elements in the corresponding third submatrixes.

[0310] In one possible implementation, the compression result includes non-zero element information, which includes the values ​​of all non-zero elements in the third submatrix and the indices of the non-zero elements in the second direction of the first matrix.

[0311] In one possible implementation, the values ​​of non-zero elements in multiple third submatrices are placed within a first array, which is a one-dimensional array, and the positions of non-zero elements in the same third submatrix within the first array are consecutive; and / or

[0312] The indices of non-zero elements in the second direction of the first matrix are placed in the second array, which is a one-dimensional array, and the indices of non-zero elements in the same third submatrix in the second direction of the first matrix are consecutive in the second array.

[0313] In one possible implementation, the generation module 1604 is specifically used for:

[0314] The compression result of the first matrix is ​​stored in memory.

[0315] This embodiment of the application can first divide the first matrix to be compressed into smaller sub-matrices, and then compress the non-zero element groups in the resulting sub-matrices to obtain a second sub-matrix. This ensures that the second sub-matrix contains only non-zero element groups and no all-zero element groups, effectively eliminating a portion of the zero-valued elements in the first matrix. In this case, based on multiple third sub-matrices, the compression result of the first matrix can effectively reduce the number of zero elements that need to be stored, thereby saving data storage space. Furthermore, in scenarios requiring matrix multiplication, the matrix can be divided according to computational needs to obtain sub-matrices of suitable size for computation, thereby effectively improving computational efficiency.

[0316] Furthermore, in this embodiment, the values ​​and indices of non-zero elements are stored consecutively according to their positions within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the indices and values ​​of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine the positions of other non-zero elements in the array, thus improving data access efficiency. Moreover, since caches typically load recently accessed data items and their adjacent data items together, when the indices and values ​​of non-zero elements are stored consecutively in the array, these elements are more likely to be loaded consecutively into the cache, thereby reducing cache misses and improving data access speed. Because an array is a linear data structure where elements are stored in a specific order, and each element in the array can be uniquely identified by its array index (i.e., position or sequence number), in this embodiment, when the values ​​and indices of non-zero elements are directly stored at the same position in two arrays, the corresponding values ​​can be accessed from both arrays based on the same array index without additional lookup or mapping operations, thus effectively improving access efficiency.

[0317] It should be noted that the matrix compression device provided in the above embodiments is only illustrated by the division of the functional modules described above when performing matrix compression. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the matrix compression device and the matrix compression method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0318] Figure 17 is a schematic diagram of a matrix processing device provided in an embodiment of this application. This matrix processing device can be implemented as part or all of the aforementioned second processor 02 by software, hardware, or a combination of both. Referring to Figure 17, the device includes: a determination module 1701.

[0319] The determination module 1701 is used to determine a target matrix based on the compression result of the first matrix and the second matrix stored in the memory. The target matrix is ​​the product of the first matrix and the second matrix. The first matrix is ​​input information or intermediate results during model training or inference. The compression result of the first matrix is ​​generated based on multiple third sub-matrices. The compression result indicates the values ​​of non-zero elements in all third sub-matrices and the positions of the non-zero elements in their respective third sub-matrices. The multiple third sub-matrices are obtained by partitioning each of the multiple second sub-matrices. The multiple second sub-matrices are obtained by compressing the non-zero element groups in each of the multiple first sub-matrices along a first direction. The multiple first sub-matrices are obtained by partitioning the first matrix. The non-zero element groups are vectors in the first sub-matrices along a second direction, and each non-zero element group contains elements with non-zero values. For detailed implementation processes, please refer to the corresponding contents in the above embodiments; they will not be repeated here.

[0320] In one possible implementation, the determining module 1701 is specifically used for:

[0321] Based on the compression result of the first matrix, multiple third sub-matrices are restored;

[0322] Determine the position of each element in the first matrix within multiple third submatrices;

[0323] Based on the position of each element in the first matrix in the multiple third sub-matrices, the computational matrices corresponding to the multiple third sub-matrices are determined from the second matrix;

[0324] The target matrix is ​​obtained by multiplying each third submatrix by its corresponding computation matrix.

[0325] In one possible implementation, the number of rows of the third submatrix is ​​a positive integer multiple of M, the number of columns is a positive integer multiple of N, and M×N is the size of the matrix to be multiplied, where M and N are both positive integers.

[0326] Module 1701 is specifically used for:

[0327] Multiple third submatrices are divided to obtain multiple first computation matrices, each of which has a size of M×N;

[0328] Based on the position of each element in the first matrix, the second calculation matrix corresponding to each of the first calculation matrices is determined from the second matrix;

[0329] Multiplying each of the third submatrices by its corresponding computational matrix yields the target matrix, including:

[0330] The target matrix is ​​obtained by multiplying the multiple first calculation matrices by their respective second calculation matrices.

[0331] In one possible implementation, the position of each element in the multiple first computation matrices includes the row number and column number of the element in the first matrix;

[0332] Module 1701 is specifically used for:

[0333] The elements in the second matrix whose column number is the same as the row number of the target element and whose row number is the same as the column number of the target element are identified as the elements in the second calculation matrix corresponding to the target calculation matrix.

[0334] The target calculation matrix is ​​any one of the multiple first calculation matrices, and the target element is any one of the elements in the target calculation matrix.

[0335] This embodiment of the application can first divide the first matrix to be compressed into smaller sub-matrices, and then compress the non-zero element groups in the resulting sub-matrices to obtain a second sub-matrix. This ensures that the second sub-matrix contains only non-zero element groups and no all-zero element groups, effectively eliminating a portion of the zero-valued elements in the first matrix. In this case, based on multiple third sub-matrices, the compression result of the first matrix can effectively reduce the number of zero elements that need to be stored, thereby saving data storage space. Furthermore, in scenarios requiring matrix multiplication, the matrix can be divided according to computational needs to obtain sub-matrices of suitable size for computation, thereby effectively improving computational efficiency.

[0336] Furthermore, in this embodiment, the values ​​and indices of non-zero elements are stored consecutively according to their positions within the third submatrix. In this case, once the position of the first non-zero element in the third submatrix and the number of non-zero elements in the third submatrix are determined, the indices and values ​​of other non-zero elements in the array can be located using simple arithmetic operations (such as addition), without needing additional data structures to determine the positions of other non-zero elements in the array, thus improving data access efficiency. Moreover, since caches typically load recently accessed data items and their adjacent data items together, when the indices and values ​​of non-zero elements are stored consecutively in the array, these elements are more likely to be loaded consecutively into the cache, thereby reducing cache misses and improving data access speed. Because an array is a linear data structure where elements are placed in a specific order, and each element in the array can be uniquely identified by its array index (i.e., position or sequence number), when the values ​​and indices of non-zero elements are located in the same position in two arrays, the corresponding values ​​can be accessed from both arrays based on the same array index without additional lookup or mapping operations, thus effectively improving access efficiency. Furthermore, when performing matrix multiplication, the matrix processing method provided in this application can perform operations on the data in the third submatrix, thereby avoiding a large number of zero-element calculations and improving the computational efficiency of matrix multiplication. When the size of the third submatrix is ​​the same as the size of the matrix used for matrix multiplication, it means that the size of the third submatrix matches the size required for matrix multiplication. Since the compression result is stored in units of the third submatrix, when performing matrix multiplication, it is only necessary to retrieve the data corresponding to the third submatrix from the compression result, without frequent access to external storage or memory swapping, thus improving the computational efficiency of matrix multiplication.

[0337] It should be noted that the matrix processing device provided in the above embodiments is only illustrated by the division of the above functional modules when performing matrix processing. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the matrix processing device and the matrix processing method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0338] This application embodiment also provides a computer system, the computer system including a first processor, a second processor, and a memory. The memory is used to store a first matrix; the first processor is used to divide the first matrix stored in the memory into multiple first sub-matrices, the first matrix being input information or intermediate results in the model training or inference process; compress the non-zero element group in each of the multiple first sub-matrices according to a first direction to obtain multiple second sub-matrices corresponding to the first matrix, the non-zero element group being a vector in the first sub-matrices along a second direction, and the non-zero element group containing elements with non-zero values; divide each of the multiple second sub-matrices to obtain multiple third sub-matrices; based on the multiple third sub-matrices, generate a compression result of the first matrix, the compression result indicating the values ​​of non-zero elements in all third sub-matrices and the positions of the non-zero elements in the corresponding third sub-matrices; the memory is also used to store the compression result of the first matrix and a second matrix; the second processor is used to determine a target matrix based on the compression result of the first matrix stored in the memory and the second matrix, the target matrix being the product of the first matrix and the second matrix.

[0339] This application also provides a chip, which includes a processor capable of executing the matrix compression method provided in this application, and / or executing the matrix processing method provided in this application.

[0340] Optionally, the chip also includes an interface circuit for receiving instructions and transmitting them to the processor.

[0341] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the steps of the matrix compression method described in the above embodiments, and / or cause the computer or processor to perform the steps of the matrix processing method described in the above embodiments.

[0342] This application also provides a computer program product containing instructions that, when executed on a computer or processor, cause the computer or processor to perform the steps of the matrix compression method described in the above embodiments, and / or cause the computer or processor to perform the steps of the matrix processing method described in the above embodiments. Alternatively, a computer program is provided that, when executed on a computer or processor, causes the computer or processor to perform the steps of the matrix compression method described in the above embodiments, and / or causes the computer or processor to perform the steps of the matrix processing method described in the above embodiments.

[0343] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0344] It should be understood that "at least one" as mentioned herein refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0345] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the first matrix involved in the embodiments of this application was obtained under full authorization.

[0346] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art should be included within the protection scope of this application.

Claims

A matrix compression method, characterized in that, Applied to a computer system, the computer system including a first processor and a memory, the method is executed by the first processor and includes: The first matrix stored in the memory is divided into multiple first sub-matrices, where the first matrix is ​​the input information or intermediate result during the model training or inference process; The non-zero element group in each of the plurality of first sub-matrices is compressed in a first direction to obtain a plurality of second sub-matrices corresponding to the first matrix. The non-zero element group is a vector in the first sub-matrices along a second direction, and there are elements with non-zero values ​​in the non-zero element group. Each of the plurality of second submatrices is divided to obtain a plurality of third submatrices; Based on the plurality of third sub-matrices, a compressed result of the first matrix is ​​generated, wherein the compressed result indicates the value of the non-zero element in all third sub-matrices and the position of the non-zero element in the corresponding third sub-matrices. The method according to claim 1, characterized in that, The step of compressing the non-zero element groups in each of the plurality of first sub-matrices in a first direction to obtain a plurality of second sub-matrices corresponding to the first matrix includes: For each of the plurality of first submatrices, the non-zero element groups in the first submatrices are concatenated in the first direction to obtain the second submatrices, wherein the first direction is the row direction of the first submatrices and the second direction is the column direction of the first submatrices, or the first direction is the column direction of the first submatrices and the second direction is the row direction of the first submatrices. The method as described in claim 2, characterized in that, The step of concatenating the non-zero element groups in the first sub-matrix along the first direction to obtain the second sub-matrix includes: When the first direction is the row direction, the second direction is the column direction, and the non-zero element group is a column vector containing non-zero elements, the row number of each element in the non-zero element group in the first submatrix remains unchanged, and the non-zero element group in the first submatrix is ​​concatenated in ascending order of column number to obtain the second submatrix. When the first direction is a column direction, the second direction is a row direction, and the non-zero element group is a row vector containing non-zero elements, the column number of each element in the non-zero element group of the first submatrix remains unchanged, and the non-zero element group of the first submatrix is ​​concatenated in ascending order of row number to obtain the second submatrix. The method as described in any one of claims 1-3, characterized in that, The first direction is the row direction, the second direction is the column direction, and the non-zero element group is a column vector containing non-zero elements; in the third submatrix, there is no all-zero column vector between any two column vectors containing non-zero elements, and the values ​​of the elements in the all-zero column vector are all zero; or The first direction is the column direction, the second direction is the row direction, and the non-zero element group is a row vector containing non-zero elements; there is no all-zero row vector between any two row vectors containing non-zero elements in the third submatrix, and the values ​​of the elements in the all-zero row vector are all zero. The method as described in any one of claims 1-4, characterized in that, The compression result includes submatrix position information and submatrix element position information. The submatrix position information indicates the relative position between the plurality of third submatrixes, and the submatrix element position information indicates the position of the non-zero element in the corresponding third submatrix. The method as described in claim 5, characterized in that, The submatrix position information includes a submatrix row pointer and a submatrix column index. The submatrix row pointer indicates the number of third submatrixes in each row, and the submatrix column index indicates the column in which each third submatrix is ​​located. The method as described in claim 5 or 6, characterized in that, The submatrix element position information includes a submatrix bitmap, which includes bitmaps corresponding to the plurality of third submatrixes respectively, and the bitmaps indicate the positions of non-zero elements in the corresponding third submatrix. The method as described in any one of claims 5-7, characterized in that, The compression result includes non-zero element information, which includes the values ​​of all non-zero elements in the third submatrix and the indices of the non-zero elements in the second direction of the first matrix. The method as described in claim 8, characterized in that, The values ​​of non-zero elements in the plurality of third submatrices are placed in a first array, which is a one-dimensional array, and the values ​​of non-zero elements in the same third submatrix are consecutive in the first array; and / or The indices of the non-zero elements in the second direction of the first matrix are placed in a second array, which is a one-dimensional array, and the indices of the non-zero elements in the same third submatrix in the second direction of the first matrix are consecutive in the second array. The method as described in any one of claims 1-9, characterized in that, After determining the compression result of the first matrix based on the plurality of third sub-matrices, the method further includes: The compression result of the first matrix is ​​stored in the memory. A matrix processing method, characterized in that, Applied to a computer system, the computer system including a second processor and a memory, the method is executed by the second processor and includes: Based on the compression result of the first matrix and the second matrix stored in the memory, a target matrix is ​​determined, wherein the target matrix is ​​the product of the first matrix and the second matrix; Wherein, the first matrix is ​​the input information or intermediate result in the model training or inference process, the compression result of the first matrix is ​​generated based on multiple third sub-matrices, the compression result indicates the value of non-zero elements in all third sub-matrices and the position of non-zero elements in the corresponding third sub-matrices, the multiple third sub-matrices are obtained by partitioning each of the multiple second sub-matrices, the multiple second sub-matrices are obtained by compressing the non-zero element group in each of the multiple first sub-matrices according to the first direction, the multiple first sub-matrices are obtained by partitioning the first matrix, the non-zero element group is the vector in the first sub-matrices along the second direction, and the non-zero element group contains elements with non-zero values. The method as described in claim 11, characterized in that, The determination of the target matrix based on the compression result of the first matrix and the second matrix stored in the memory includes: Based on the compression result of the first matrix, the plurality of third sub-matrices are restored; Determine the position of each element in the plurality of third sub-matrices within the first matrix; Based on the position of each element in the plurality of third sub-matrices in the first matrix, the computational matrices corresponding to the plurality of third sub-matrices are determined from the second matrix; The target matrix is ​​obtained by multiplying each of the multiple third submatrices by its corresponding computation matrix. The method as described in claim 12, characterized in that, The number of rows in the third submatrix is ​​a positive integer multiple of M, and the number of columns is a positive integer multiple of N. M×N is the size of the matrix to be multiplied, and both M and N are positive integers. The step of determining the computational matrix corresponding to each of the plurality of third sub-matrices from the second matrix based on the position of each element in the plurality of third sub-matrices in the first matrix includes: The plurality of third sub-matrices are divided to obtain a plurality of first computation matrices, each of which has a size of M×N; Based on the position of each element in the plurality of first calculation matrices in the first matrix, the second calculation matrix corresponding to each of the plurality of first calculation matrices is determined from the second matrix; The step of multiplying the plurality of third sub-matrices by their respective corresponding computational matrices to obtain the target matrix includes: The target matrix is ​​obtained by multiplying the plurality of first calculation matrices by their respective second calculation matrices. The method as described in claim 13, characterized in that, The position of each element in the plurality of first calculation matrices includes the row number and column number of the element in the first matrix; The step of determining the second calculation matrix corresponding to each of the plurality of first calculation matrices from the second matrix based on the position of each element in the first matrix includes: The elements in the second matrix whose column number is the same as the row number of the target element and whose row number is the same as the column number of the target element are identified as elements in the second calculation matrix corresponding to the target calculation matrix. Wherein, the target calculation matrix is ​​any one of the plurality of first calculation matrices, and the target element is any one of the elements of the target calculation matrix. A matrix compression device, characterized in that, A first processor included in a computer system, the computer system further including a memory, the means comprising: The first partitioning module is used to partition the first matrix stored in the memory to obtain multiple first sub-matrices, wherein the first matrix is ​​input information or intermediate results in the model training or inference process; A compression module is used to compress the non-zero element group in each of the plurality of first sub-matrices in a first direction to obtain a plurality of second sub-matrices corresponding to the first matrix. The non-zero element group is a vector in the first sub-matrices along a second direction, and there are elements with non-zero values ​​in the non-zero element group. The second partitioning module is used to partition each of the plurality of second sub-matrices to obtain a plurality of third sub-matrices; The determining module is used to generate a compression result of the first matrix based on the plurality of third sub-matrices, wherein the compression result indicates the values ​​of non-zero elements in all third sub-matrices and the positions of the non-zero elements in the corresponding third sub-matrices. A matrix processing device, characterized in that, A second processor included in a computer system, the computer system further including a memory, the device comprising: The determining module is used to determine a target matrix based on the compression result of the first matrix and the second matrix stored in the memory, wherein the target matrix is ​​the product of the first matrix and the second matrix; Wherein, the first matrix is ​​the input information or intermediate result in the model training or inference process, the compression result of the first matrix is ​​generated based on multiple third sub-matrices, the compression result indicates the value of non-zero elements in all third sub-matrices and the position of non-zero elements in the corresponding third sub-matrices, the multiple third sub-matrices are obtained by partitioning each of the multiple second sub-matrices, the multiple second sub-matrices are obtained by compressing the non-zero element group in each of the multiple first sub-matrices according to the first direction, the multiple first sub-matrices are obtained by partitioning the first matrix, the non-zero element group is the vector in the first sub-matrices along the second direction, and the non-zero element group contains elements with non-zero values. A computer system, characterized in that, The computer system includes a first processor, a second processor, and a memory; The memory is used to store the first matrix; The first processor is configured to divide the first matrix stored in the memory into multiple first sub-matrices, wherein the first matrix is ​​input information or intermediate results in the model training or inference process; The non-zero element group in each of the plurality of first sub-matrices is compressed in a first direction to obtain a plurality of second sub-matrices corresponding to the first matrix. The non-zero element group is a vector in the first sub-matrices along a second direction, and there are elements with non-zero values ​​in the non-zero element group. Each of the plurality of second submatrices is divided to obtain a plurality of third submatrices; Based on the plurality of third sub-matrices, a compressed result of the first matrix is ​​generated, wherein the compressed result indicates the value of the non-zero element in all third sub-matrices and the position of the non-zero element in the corresponding third sub-matrices; The memory is also used to store the compression result of the first matrix and the second matrix; The second processor is configured to determine a target matrix based on the compression result of the first matrix stored in the memory and the second matrix, wherein the target matrix is ​​the product of the first matrix and the second matrix. A chip characterized in that, The chip includes a processor for implementing the steps of the method according to any one of claims 1 to 10, and / or for implementing the steps of the method according to any one of claims 11 to 14. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a computer or processor, cause the steps of the method as described in any one of claims 1 to 10 to be performed, and / or cause the steps of the method as described in any one of claims 11 to 14 to be performed.

Images