Decompression of expressive sparse matrix representations with finite element data

Abstract

The invention discloses decompression of expressive sparse matrix representations with finite element data. Systems and techniques for decompressing expressive sparse matrix representations with finite element data are disclosed. The techniques include receiving a sparse matrix and metadata corresponding to the sparse matrix. The sparse matrix is a compressed representation of the dense matrix. The sparse matrix contains a first number (N) of elements to be retained in the dense matrix, and the dense matrix comprises at least a second number (M) of elements. The metadata corresponding to the sparse matrix is based on a third number (P) of positions and a format determined during compression of the dense matrix. The techniques include generating an uncompressed matrix based on a sparse matrix and metadata corresponding to the sparse matrix.

Description

TECHNICAL FIELD

[0001] At least one embodiment relates to a system for decompressing an expressive sparse matrix representation with limited element data to create an uncompressed matrix. BACKGROUND

[0002] Neural networks use a large number of matrix mathematical operations (e.g., matrix multiplication, dot product, etc.) during training and during inference. For example, weights of a layer of a neural network can be stored in a matrix. To compute output values of a layer, the weights of the layer can be multiplied by input values of the layer. In some cases, setting certain weights to zero does not have a significant impact on the results of the neural network. During a matrix multiplication operation, weights with zero values do not need to be considered because the result will always be zero, so when there are more zero values in a matrix, mathematical throughput can be improved. A matrix can be compressed to retain a subset of values (e.g., non-zero values) while discarding the remaining values (e.g., zero values) to reduce storage space requirements and improve mathematical operation throughput (because the discarded values can be treated as zero). BRIEF DESCRIPTION OF DRAWINGS

[0003] Figure 1 is an example block diagram of a system capable of using an expressive sparse matrix with limited element data according to at least one embodiment;

[0004] Figure 2 is an example block diagram of compressing a dense matrix into an expressive sparse matrix with limited element data according to at least one embodiment;

[0005] Figure 3 is an example block diagram of decompressing an expressive sparse matrix with limited element data into an uncompressed matrix according to at least one embodiment;

[0006] Figure 4A is an example block diagram of decompressing an expressive sparse matrix with limited element data into an uncompressed matrix according to at least one embodiment;

[0007] Figure 4B is an example block diagram of decompressing an expressive sparse matrix with limited element data into an uncompressed matrix according to at least one embodiment;

[0008] Figure 4C is an example block diagram of decompressing an expressive sparse matrix with limited element data into an uncompressed matrix according to at least one embodiment;

[0009] Figure 4D is an example block diagram of decompressing an expressive sparse matrix with limited element data into an uncompressed matrix according to at least one embodiment;

[0010] Figure 5Ais an example block diagram of compressing a dense matrix into a fully-expressive sparse matrix with limited element data, according to at least one embodiment;

[0011] Figure 5B is an example block diagram of decompressing a fully-expressive sparse matrix with limited element data into an uncompressed matrix, according to at least one embodiment;

[0012] Figure 6 is an example block diagram of computing a dot product between a sparse matrix with corresponding metadata and a matrix operand, according to at least one embodiment;

[0013] Figure 7 is an example block diagram of an integrated circuit for performing matrix mathematical operations using a sparse matrix with corresponding metadata, according to at least one embodiment;

[0014] Figure 8 is a flow diagram of an example method for compressing a dense matrix into an expressive sparse matrix with limited element data, according to at least one embodiment;

[0015] Figure 9 is a flow diagram of an example method for decompressing an expressive sparse matrix with limited element data into an uncompressed matrix, according to at least one embodiment;

[0016] Figure 10 is a flow diagram of an example method for computing a dot product using an expressive sparse matrix with limited element data, according to at least one embodiment;

[0017] Figure 11 is a flow diagram of an example method for performing a matrix multiplication operation using an expressive sparse matrix with limited element data, according to at least one embodiment;

[0018] Figure 12 is a flow diagram of an example method for compressing a dense matrix into a fully-expressive sparse matrix with limited element data, according to at least one embodiment. DETAILED DESCRIPTION

[0019] For a given task, certain values of a matrix can be more important than others. By removing less important values from a matrix (e.g., converting a dense matrix to a sparse matrix), mathematical throughput can be improved. Existing sparse matrix representations preserve N non-zero values out of M consecutive elements of a dense matrix (N:M sparsity). For example, a 2:4 sparse matrix would have 2 non-zero values in every consecutive set of 4 elements of a dense matrix, while a 4:8 sparse matrix would have 4 non-zero values in every consecutive set of 8 elements of a dense matrix. Metadata can be used to indicate the index of each non-zero value (or in some cases, the index of each zero value). For example, a 2:4 sparse matrix requires 2 bits of metadata per non-zero element to represent all possible compressed matrices, while a 4:8 sparse matrix requires 3 bits of metadata per non-zero element to represent all possible compressed matrices. It can be advantageous to store less metadata, however, less metadata means that a sparse matrix can represent fewer dense matrices.

[0020] The present disclosure provides systems and techniques that allow expressive sparse matrix representations to be implemented using limited (e.g., a limited number of) metadata. Expressive sparse matrix representations can allow for N non-zero values out of M consecutive elements of a dense matrix, where each non-zero value has P available positions that can be interpreted according to a given format F(N:M:P, F-sparsity). An N:M sparse matrix (as discussed above) is equivalent to an N:M:P, full-sparse matrix (e.g., where P=M). For example, the 4:8 sparse matrix discussed above allows for 4 non-zero values out of 8 consecutive elements, and each non-zero value can be in any of 8 available positions. This can be represented as 4:8:8, a full-sparse matrix. This allows for any of 70 different data patterns (e.g., combinations of 4 non-zero values out of 8 consecutive elements) to be represented, but it requires 3 bits of metadata to indicate the index of each non-zero value (e.g., log2(8)=3). ) out of 8 consecutive elements, but it requires 2 bits of metadata to indicate the index of each non-zero value (e.g., log2(4)=2).

[0021] By choosing P < M, the amount of metadata needed to indicate the index of each non-zero value can be reduced. For example, a 4:8:4 sparse matrix only needs 2 bits of metadata to indicate the index of each non-zero value (because each non-zero value can only be in one of 4 positions (e.g., log2(4) = 2), instead of one of 8 positions). The number of data patterns that can be expressed using an N:M:P sparse matrix depends on the format F that is chosen. For example, using a simple "replicated" format, a 4:8:4 sparse matrix can be used to represent 36 data patterns. Using a "direct" format, the same 4:8:4 sparse matrix can represent 60 different data patterns. Using a "checkerboard" format, the same 4:8:4 sparse matrix can represent 66 different data patterns. Thus, a 4:8:4, checkerboard sparse matrix can represent 94% of the data patterns that can be represented by a 4:8:8, full sparse matrix, but it only needs 2 bits of metadata per non-zero value, instead of 3 bits of metadata per non-zero value (33% space savings).

[0022] The compression of a dense matrix into an N:M:P, F sparse matrix can be performed using software or hardware. We will discuss how to compress 8 elements into a 4:8:4 sparse matrix, but it is important to understand that N, M, and P can be arbitrary values, and that N < M and P < M. The compression can first calculate the "importance" of each source element according to an importance criterion. For example, the "importance" of an element can be determined based on the absolute magnitude of that element. The goal of the compression can be to represent the 8 elements with as high a cumulative "importance" as possible, that is, to maximize the sum of the importance of each non-zero element in the compressed representation of the data.

[0023] The present disclosure also provides systems and techniques that allow for a fully expressive sparse matrix representation to be achieved using a limited (e.g., a constrained amount) of metadata. A fully expressive sparse matrix representation can allow for N non-zero values to be taken from M consecutive elements of a dense matrix using B bits of metadata (N:M:Bb sparsity). A conventional implementation can compress a dense matrix into a 4:8 sparse matrix, where each non-zero value has a corresponding 3-bit metadata value to express the index of that non-zero value in the dense matrix. The disclosed techniques can compress a dense matrix into a 4:8:2b sparse matrix, where each non-zero value has a corresponding 2-bit metadata value (33% space savings), without any loss of expressiveness (e.g., all dense matrices that can be expressed using a 4:8 compression scheme and 3-bit metadata can be expressed using the 4:8:2b compression scheme disclosed herein). By giving special meaning to certain combinations of metadata, data patterns that cannot be expressed using a 4:8:4, checkerboard compression scheme can be expressed using a 4:8:2b compression scheme, thereby achieving full expressiveness.

[0024] Advantages of the disclosed technology include, but are not limited to, reduced storage space requirements for storing sparse matrices compared to corresponding dense matrices, and additional computational throughput when using sparse matrices in matrix computations.

[0025] System Architecture

[0026] Figure 1 is an example block diagram of a system 102 capable of using expressive sparse matrices with limited element data, according to at least one embodiment. The system 102 can include a central processing unit (CPU) 104, a sparse matrix subsystem 106, a memory 108, and a graphics processing unit (GPU) 110. In some embodiments, the memory 108 can store one or more matrices (or portions thereof) and sparse matrix metadata. For example, the memory 108 can store a dense matrix, a sparse matrix representing a compressed form of the dense matrix, and metadata corresponding to the sparse matrix. In some embodiments, the sparse matrix subsystem 106 of the system 102 can convert (e.g., compress) the dense matrix into a sparse matrix with corresponding metadata. In some embodiments, the sparse matrix subsystem 106 uses the CPU 104 to compress a dense matrix (e.g., from the memory 108) into a sparse matrix with corresponding metadata. In some embodiments, the sparse matrix subsystem 106 uses the GPU 110 to compress a dense matrix (e.g., from the memory 108) into a sparse matrix with corresponding metadata. The sparse matrix subsystem 106 can store the resulting sparse matrix and corresponding metadata to the memory 108.

[0027] In some embodiments, the sparse matrix subsystem 106 can decompress the sparse matrix based on the corresponding metadata to obtain a decompressed matrix. The decompressed matrix can include a subset of the values of the original dense matrix from which the sparse matrix was generated. For example, if the original dense matrix includes 8 elements, 6 of which are non-zero, and is compressed using a 4:8:4, F compression scheme, the decompressed matrix can include 4 of the 6 non-zero values of the dense matrix. Since the 4:8:4, F compression scheme only stores 4 non-zero elements of a set of 8 consecutive elements, 2 non-zero elements are lost in the compression-decompression process.

[0028] GPU 110 can be used to perform parallel operations. For example, GPU 110 can be used to perform mathematical computations in parallel. This can be advantageous when training and reasoning on artificial intelligence (AI) models, such as neural networks. In some embodiments, GPU 110 can include one or more sparse tensor cores 112. Sparse tensor cores 112 can include one or more integrated circuits to perform computations (e.g., matrix mathematical operations, matrix multiplication operations) using sparse matrices and corresponding metadata. As an example, sparse tensor cores 112 can be used to compute output values for a layer of a neural network using a sparse matrix representing weights for the layer. Based on metadata corresponding to the sparse matrix, sparse tensor cores 112 can select values from a matrix operand and compute a product (e.g., dot product) between the sparse matrix and a subset of values from the matrix operand.

[0029] In some embodiments, it can be advantageous to compress a dense matrix into a sparse matrix even if the dense matrix is not used in sparse tensor cores 112. For example, compressing a dense matrix into a sparse matrix can reduce storage space requirements (e.g., reduce storage space needed to store weights for a neural network or other AI model). Similarly, bandwidth requirements for transmitting a matrix over a network can be reduced if the dense matrix is first compressed into a sparse representation with corresponding metadata.

[0030] Figure 2 is an example block diagram of compressing a dense matrix 202 into an expressive sparse matrix with limited metadata, according to at least one embodiment. A given dense matrix (e.g., dense matrix 202) can be compressed into various sparse matrix representations with corresponding metadata depending on the selected compression format. The selected compression scheme determines the size of the metadata. Figure 2 A dense matrix 202 being compressed using a 4:8:4, F compression scheme is shown with four different formats: 204a, 204b, 204c, and 204d. For example, a dense matrix 202 compressed using a 4:8:4, F compression scheme with format 204a results in a sparse matrix 206a and metadata 208a. Four non-zero elements out of eight consecutive elements of dense matrix 202 are preserved in sparse matrix 206a. Each non-zero value of sparse matrix 206a can be located in one of four possible positions (because P = 4 in an N:M:P, F compression scheme). Metadata 208a indicates the index of each non-zero value in its corresponding four available positions. (See Figure 4A 、 Figure 4B 、 Figure 4C and / or Figure 4D some possible formats and examples of available positions for each non-zero value under that format.)

[0031] Compressing the dense matrix 202 using the 4:8:4, F compression scheme with format 204b results in a sparse matrix 206b and corresponding metadata 208b. Compressing the dense matrix 202 using the 4:8:4, F compression scheme with format 204c results in a sparse matrix 206c and corresponding metadata 208c. Compressing the dense matrix 202 using the 4:8:4, F compression scheme with format 204d results in a sparse matrix 206d and corresponding metadata 208d.

[0032] The selected format determines the resulting sparse matrix and metadata. For example, the sparse matrix 206a and the sparse matrix 206c share the same values, but their corresponding metadata values (e.g., metadata 208a and metadata 208c, respectively) are different due to the different formats. Similarly, the sparse matrix 206b and the sparse matrix 206d share the same values, but their corresponding metadata values (e.g., metadata 208b and metadata 208d, respectively) are different. While in this example, the sparse matrix 206a, the sparse matrix 206b, the sparse matrix 206c, and the sparse matrix 206d share the same values but in different orders, it should not be understood that this is always the case. In some cases, not all of the highest magnitudes can be represented in the compressed format, so different formats can result in different values of the dense matrix being included in the sparse matrix.

[0033] While the dense matrix 202 is shown as having 8 integers, it should be understood that the dense matrix 202 can include values (or elements) of any length. For example, the dense matrix 202 can include 8 32-bit floating point numbers. After compression, the corresponding sparse matrix (e.g., the sparse matrix 206a) can include 4 32-bit floating point numbers. The metadata corresponding to the sparse matrix (e.g., the metadata 208a) can include 4 2-bit values representing the indices of the 4 non-zero values in their corresponding 4 available positions.

[0034] As another example, the dense matrix can include 16 16-bit floating point numbers. If the dense matrix is compressed using the 6:16:8, F compression scheme, the resulting sparse matrix can include 6 16-bit floating point numbers, and the corresponding metadata can include 6 3-bit values (e.g., log2(8) = 3) representing the indices of the 6 non-zero values in their corresponding 8 available positions.

[0035] While the dense matrix 202 is shown as having one row of elements (e.g., a 1x8 matrix), it should be understood that the dense matrix 202 can include multiple rows of elements. For example, the dense matrix 202 can be an 8x8 matrix and can be compressed to an 8x4 sparse matrix having a corresponding 8x4 matrix of metadata values. In some embodiments, each row of the dense matrix 202 can be compressed using different P and / or format. In some embodiments, each row of the dense matrix 202 can be compressed using different compression schemes, e.g., a first row using N:M:P,F and a second row using N:M:Bb. In some embodiments, a subset of rows of the dense matrix 202 can be compressed separately. In some embodiments, the resulting sparse matrices can be joined (e.g., concatenated) together to form a single resulting sparse matrix. In some embodiments, the elements of multiple rows of the dense matrix 202 can be treated as contiguous elements for compression.

[0036] To perform compression using the N:M:P,F scheme, a dense matrix can be received and / or identified. The dense matrix can include at least M contiguous elements. The dense matrix can have K non-zero elements. N of the M elements can be preserved during compression and can be included in a sparse matrix. The N elements of the resulting sparse matrix can be a subset of the K non-zero elements of the dense matrix. In some embodiments, the elements of the dense matrix that are not preserved in the sparse matrix are all zero. In some embodiments, the elements of the dense matrix that are not preserved in the sparse matrix include at least one non-zero value.

[0037] The number of non-zero positions P and the format F of the N:M:P, F compression scheme determines how the dense matrix is expressed in the sparse matrix and corresponding metadata. The goal of the compression can be to maximize the overall importance value of the resulting sparse matrix (e.g., the sum of the importance values of each element). In some embodiments, all possible combinations of sparse matrices and metadata can be evaluated based on the available non-zero positions P in the determined format F, and the overall importance of each resulting sparse matrix can be determined. The sparse matrix and metadata that results in the highest overall importance of the resulting matrix can then be selected as the result of the dense matrix compression. For example, an 8-element dense matrix compressed using the 4:8:4, checkerboard compression scheme can be compressed into 66 unique sparse matrix and metadata pairs. Each of the 66 unique pairs can be evaluated to determine which pair results in the highest overall importance value. However, in some embodiments (e.g., in hardware implementations), evaluating all possible combinations can be difficult and / or infeasible. Moreover, since certain P, F compression schemes have overlapping non-zero positions, the order in which the non-zero values and corresponding metadata are selected / assigned during compression can affect the overall importance of the resulting sparse matrix. In such cases, a compression sequence can be used to determine the resulting sparse matrix and metadata pair. The compression sequence can include an importance comparison of the non-zero values and / or a sorting of the non-zero values.

[0038] In some embodiments, the first compression sequence can include determining the non-zero values and corresponding metadata values in a predefined order. For example, in the 4:8:4, checkerboard compression scheme, it can be advantageous to determine the first non-zero value and the fourth non-zero value and their corresponding metadata values before determining the second non-zero value and the third non-zero value and their corresponding metadata values. In other words, the importance values of all 8 elements of the dense matrix to be compressed (or only the importance values of all K non-zero elements of the dense matrix) can be determined. The first sparse matrix value (and its corresponding metadata value) can then be selected as the element with the highest importance value among elements 0-3 of the dense matrix (since the available position for the first non-zero value under the “checkerboard” format covers elements 0-3 of the dense matrix) (see Figure 4D In some embodiments, the first compression sequence can include determining the non-zero values and corresponding metadata values in a predefined order. For example, in the 4:8:4, checkerboard compression scheme, it can be advantageous to determine the first non-zero value and the fourth non-zero value and their corresponding metadata values before determining the second non-zero value and the third non-zero value and their corresponding metadata values. In other words, the importance values of all 8 elements of the dense matrix to be compressed (or only the importance values of all K non-zero elements of the dense matrix) can be determined. The first sparse matrix value (and its corresponding metadata value) can then be selected as the element with the highest importance value among elements 0-3 of the dense matrix (since the available position for the first non-zero value under the “checkerboard” format covers elements 0-3 of the dense matrix) (see

[0039] The fourth sparse matrix value (and its corresponding metadata value) can then be selected as the element with the highest importance value among elements 4-7 of the dense matrix (since the available position for the fourth non-zero value under the “checkerboard” format covers elements 4-7 of the dense matrix). The importance values of the selected first non-zero value and the selected fourth non-zero value can be cleared (e.g., set to 0 or some other minimum value).

[0040] The second sparse matrix value (and its corresponding metadata value) can then be selected as the element of the dense matrix with the largest importance value among elements 0, 2, 4, and 6, given the importance values that were cleared after assigning the first non-zero value and the fourth non-zero value. The third sparse matrix value (and its corresponding metadata value) can be selected as the element of the dense matrix with the largest importance value among elements 1, 3, 5, and 7, given the importance values that were cleared after assigning the first non-zero value and the fourth non-zero value. The resulting sparse matrix can include the 4 non-zero elements selected during their respective phases. In some embodiments, the overall importance value of the resulting sparse matrix determined according to this first compression sequence can not be maximized, but the compression can be performed in a hardware-efficient manner, enabling an acceptable tradeoff.

[0041] In some embodiments, selecting one or more sparse matrix values can be performed concurrently. For example, selecting the first sparse matrix value from elements 0-3 of the dense matrix can be performed concurrently (e.g., concurrently, simultaneously, etc.) with selecting the second sparse matrix value from elements 4-7 of the dense matrix. In some embodiments, selecting one or more sparse matrix values can be performed serially.

[0042] In some embodiments, the second compression sequence can include ordering the non-zero values by importance value and assigning the non-zero values in a predetermined order. As an example, to compress a dense matrix having 8 elements using a 4:8:4, checkerboard compression scheme according to the second compression sequence, the importance of all 8 elements of the dense matrix (or the K non-zero elements of the dense matrix) can be determined. The elements can be ordered according to their importance values. The most important element can be assigned to the first compatible non-zero value of the sparse matrix (e.g., the first non-zero value whose non-zero position covers the index of the element of the dense matrix). The next most important element can then be assigned to the next compatible non-zero value of the sparse matrix. This process can continue until all non-zero values of the sparse matrix have been assigned.

[0043] In some embodiments, the third compression sequence can include ordering the non-zero values by their importance values and selecting the non-zero values based on which non-zero values are subject to the most stringent constraints. Depending on the P, F compression scheme selected, more than one of the available non-zero positions in format F can be used to express a particular index of the dense matrix. For example, in a“4, checkerboard” compression scheme (see Figure 4D), a non-zero element in index 0 of the dense matrix can be expressed with a non-zero value 1 with a metadata value of 0, or a non-zero value 2 with a metadata value of 0. As another example, in the same “4, checkerboard” compression scheme, a non-zero element in index 3 of the dense matrix can be expressed with a non-zero value 1 with a metadata value of 3, or a non-zero value 3 with a metadata value of 1. Thus, the order in which non-zero elements are selected can affect the overall significance value of the resulting sparse matrix. It can be advantageous to select non-zero elements of a dense matrix based on the number of ways the index of the non-zero element can be expressed in a sparse matrix and the corresponding metadata.

[0044] As an example, to compress a dense matrix having 8 elements using a 4:8:4, checkerboard compression scheme according to a third compression sequence, the significance values of all 8 elements of the dense matrix (or the K non-zero elements of the dense matrix) can be determined. Then, a constraint value can be calculated for each element, which indicates how many non-zero elements can represent the index of that element. The element with the highest significance value and the lowest constraint value (e.g., the most significant and most constrained value) can be selected first and assigned to a constrained non-zero value of the sparse matrix. As a selection of other elements of the dense matrix, the assigned non-zero element can be removed, and the constraint values can be (optionally) recalculated. Second, the element with the second highest significance value and the second lowest constraint value can be selected and assigned to a constrained non-zero value. This process can continue until all non-zero elements of the sparse matrix have been assigned a value.

[0045] Figure 3 is an example block diagram of decompressing an expressive sparse matrix 302 having limited metadata into an uncompressed matrix, according to at least one embodiment. A given sparse matrix (e.g., sparse matrix 302) and corresponding metadata (e.g., metadata 304) can be decompressed into various uncompressed matrices according to the format selected during compression. Figure 3 Decompression of sparse matrix 302 and corresponding metadata 304 using a 4:8:4, F (de)compression scheme is shown, which has four different formats: 306a, 306b, 306c, and 306d. For example, decompression of sparse matrix 302 and corresponding metadata 304 using the 4:8:4, F scheme with format 306a results in uncompressed matrix 308a. The four non-zero elements of sparse matrix 302 are preserved in uncompressed matrix 308a. Based on metadata 304 and format 306a, the non-zero elements of sparse matrix 302 are assigned locations in uncompressed matrix 308a. In some embodiments, other locations of uncompressed matrix 308a (e.g., locations not corresponding to non-zero values in sparse matrix 302) can be assigned zero values.

[0046] The sparse matrix 302 and metadata 304 can be decompressed using format 306b to obtain an uncompressed matrix 308b. The sparse matrix 302 and metadata can be decompressed using format 306c to obtain an uncompressed matrix 308c. The sparse matrix 302 and metadata 304 can be decompressed using format 306d to obtain an uncompressed matrix 308d.

[0047] The format used during decompression determines the resulting uncompressed matrix. For example, the sparse matrix 302 and metadata 304 can be decompressed into at least 4 different matrices (e.g., uncompressed matrix 308a, uncompressed matrix 308b, uncompressed matrix 308c, uncompressed matrix 308d) based on the selected format. Using the same format during compression and decompression can result in an uncompressed matrix that is similar to the original dense matrix. In other words, if the sparse matrix is decompressed using the same compression scheme and format used to compress the dense matrix, the uncompressed matrix can have a subset of the values of the dense matrix in the same locations as the corresponding values in the dense matrix.

[0048] Figure 4A is an example block diagram of decompressing an expressive sparse matrix with limited metadata into an uncompressed matrix according to at least one embodiment. The sparse matrix 302 and corresponding metadata 304 can be decompressed according to format 306a to obtain an uncompressed matrix 308a. In some embodiments, format 306a can be referred to as a “replicated” format (e.g., N:M:P, replicated). Format 306a is represented with output indices 402a (0-7) and buckets corresponding to each possible location of each non-zero element of the sparse matrix 302. For example, since format 306a corresponds to a 4:8:4, F (de)compression scheme, there are 4 rows of non-zero locations (e.g., 404a, 406a, 408a, and 410a) that correspond to the 4 non-zero elements of the sparse matrix 302 (e.g., N=4 in the N:M:P, F scheme). Each row of non-zero locations has 4 buckets corresponding to the 4 available locations for each non-zero (e.g., P=4 in the N:M:P, F scheme).

[0049] The non-zero 1 locations 404a include 4 buckets indexed 0-3, each corresponding to an index of the uncompressed matrix 308a: NZ1:0 corresponds to index 0, NZ1:1 corresponds to index 1, NZ1:2 corresponds to index 2, and NZ1:3 corresponds to index 3. Thus, based on the first index value in the metadata 304, the first non-zero value (e.g., 8) of the sparse matrix 302 can end up at index 0, index 1, index 2, or index 3 of the uncompressed matrix 308a. Since the first index value of the metadata 304 is 0, the first non-zero value (e.g., 8) of the sparse matrix 302 is assigned to the bucket of the 0th index of the non-zero 1 locations 404a and ends up at index 0 of the uncompressed matrix 308a.

[0050] The non-zero 2 locations 406a include 4 buckets indexed 0-3, each corresponding to an index of the uncompressed matrix 308a: NZ2:0 corresponds to index 0, NZ2:1 corresponds to index 1, NZ2:2 corresponds to index 2, and NZ2:3 corresponds to index 3. Thus, based on the second index value in the metadata 304, the second non-zero value (e.g., 3) of the sparse matrix 302 can end up at index 0, index 1, index 2, or index 3 of the uncompressed matrix 308a. Since the second index value of the metadata 304 is 2, the second non-zero value (e.g., 3) of the sparse matrix 302 is assigned to the bucket of the 2nd index of the non-zero 2 locations 406a and ends up at index 2 of the uncompressed matrix 308a.

[0051] The non-zero 3 locations 408a include 4 buckets indexed 0-3, each corresponding to an index of the uncompressed matrix 308a: NZ3:0 corresponds to index 4, NZ3:1 corresponds to index 5, NZ3:2 corresponds to index 6, and NZ3:3 corresponds to index 7. Thus, based on the third index value in the metadata 304, the third non-zero value (e.g., 5) of the sparse matrix 302 can end up at index 4, index 5, index 6, or index 7 of the uncompressed matrix 308a. Since the third index value of the metadata 304 is 1, the third non-zero value (e.g., 5) of the sparse matrix 302 is assigned to the bucket of the 1st index of the non-zero 3 locations 408a and ends up at index 5 of the uncompressed matrix 308a.

[0052] The non-zero 4 position 410a includes 4 buckets indexed 0-3, each corresponding to one index of the uncompressed matrix 308a: NZ4:0 corresponds to index 4, NZ4:1 corresponds to index 5, NZ4:2 corresponds to index 6, and NZ4:3 corresponds to index 7. Thus, based on the fourth index value in the metadata 304, the fourth non-zero value (e.g., 2) of the sparse matrix 302 is ultimately likely to be located at index 4, index 5, index 6, or index 7 of the uncompressed matrix 308a. Since the fourth index value of the metadata 304 is 3, the fourth non-zero value (e.g., 2) of the sparse matrix 302 is assigned to the bucket of the 3rd index of the non-zero 4 position 410a and is ultimately located at index 7 of the uncompressed matrix 308a.

[0053] Figure 4B is an example block diagram of decompressing an expressive sparse matrix with limited metadata into an uncompressed matrix, according to at least one embodiment. The sparse matrix 302 and corresponding metadata 304 can be decompressed according to a format 306b to obtain an uncompressed matrix 308b. In some embodiments, the format 306b can be referred to as an “alternating” format (e.g., N:M:P, alternating). The format 306b is represented with output indices 402b 0-7 and buckets corresponding to each possible position for each non-zero element of the sparse matrix 302. For example, since the format 306b corresponds to a 4:8:4, F (de)compression scheme, there are 4 rows of non-zero positions (e.g., 404b, 406b, 408b, and 410b) that correspond to the 4 non-zero elements of the sparse matrix 302 (e.g., N=4 in the N:M:P, F scheme). Each row of non-zero positions has 4 buckets corresponding to the 4 available positions for each non-zero value (e.g., P=4 in the N:M:P, F scheme).

[0054] Since the first index value of the metadata 304 is 0, the first non-zero value (e.g., 8) of the sparse matrix 302 is assigned to the bucket of the 0th index of the non-zero 1 position 404b and is ultimately located at index 0 of the uncompressed matrix 308b. Since the second index value of the metadata 304 is 2, the second non-zero value (e.g., 3) of the sparse matrix 302 is assigned to the bucket of the 2nd index of the non-zero 2 position 406b and is ultimately located at index 6 of the uncompressed matrix 308b. Since the third index value of the metadata 304 is 1, the third non-zero value (e.g., 5) of the sparse matrix 302 is assigned to the bucket of the 1st index of the non-zero 3 position 408b and is ultimately located at index 1 of the uncompressed matrix 308b. Since the fourth index value of the metadata 304 is 3, the fourth non-zero value (e.g., 2) of the sparse matrix 302 is assigned to the bucket of the third index of the non-zero 4 position 410b and is ultimately located at index 7 of the uncompressed matrix 308b.

[0055] As Figure 4B shown, in some embodiments, the order of non-zero values in sparse matrix 302 can be different from the order of non-zero values in an uncompressed matrix (e.g., uncompressed matrix 308b), depending on the format used for decompression.

[0056] Figure 4C is an example block diagram of decompressing an expressive sparse matrix with finite element data into an uncompressed matrix, according to at least one embodiment. Sparse matrix 302 and corresponding metadata 304 can be decompressed according to format 306c to obtain uncompressed matrix 308c. In some embodiments, format 306c can be referred to as a “direct” format (e.g., N:M:P, direct). Format 306c is represented with output indices 402c 0-7 and buckets corresponding to each possible position of each non-zero element of sparse matrix 302. For example, since format 306c corresponds to a 4:8:4, F (de)compression scheme, there are 4 rows of non-zero positions (e.g., 404c, 406c, 408c, and 410c) that correspond to the 4 non-zero elements of sparse matrix 302 (e.g., N=4 in the N:M:P, F scheme). Each row of non-zero positions has 4 buckets that correspond to the 4 available positions for each non-zero value (e.g., P=4 in the N:M:P, F scheme).

[0057] Since the first index value of metadata 304 is 0, the first non-zero value of sparse matrix 302 (e.g., 8) is assigned to the bucket of the 0th index of non-zero 1 position 404c and ends up at index 0 of uncompressed matrix 308c. Since the second index value of metadata 304 is 2, the second non-zero value of sparse matrix 302 (e.g., 3) is assigned to the bucket of the 2nd index of non-zero 2 position 406c and ends up at index 3 of uncompressed matrix 308c. Since the third index value of metadata 304 is 1, the third non-zero value of sparse matrix 302 (e.g., 5) is assigned to the bucket of the 1st index of non-zero 3 position 408c and ends up at index 4 of uncompressed matrix 308c. Since the fourth index value of metadata 304 is 3, the fourth non-zero value of sparse matrix 302 (e.g., 2) is assigned to the bucket of the 3rd index of non-zero 4 position 410c and ends up at index 7 of uncompressed matrix 308c.

[0058] Figure 4Dis an example block diagram of decompressing an expressive sparse matrix with finite element data into an uncompressed matrix, according to at least one embodiment. Sparse matrix 302 and corresponding metadata 304 can be decompressed according to format 306d to obtain uncompressed matrix 308d. In some embodiments, format 306d can be referred to as a “checkerboard” format (e.g., N:M:P, checkerboard). Format 306d is represented with output indices 402d 0-7 and buckets corresponding to each possible location of each non-zero element of sparse matrix 302. For example, since format 306d corresponds to a 4:8:4, F (de)compression scheme, there are 4 rows of non-zero locations (e.g., 404d, 406d, 408d, and 410d) that correspond to 4 non-zero elements of sparse matrix 302 (e.g., N=4 in the N:M:P, F scheme). Each row of non-zero locations has 4 buckets that correspond to the 4 available locations of each non-zero (e.g., P=4 in the N:M:P, F scheme).

[0059] Since the first index value of metadata 304 is 0, the first non-zero value of sparse matrix 302 (e.g., 8) is assigned to the 0th index bucket of non-zero 1 location 404d and ends up at index 0 of uncompressed matrix 308d. Since the second index value of metadata 304 is 2, the second non-zero value of sparse matrix 302 (e.g., 3) is assigned to the 2nd index bucket of non-zero 2 location 406d and ends up at index 4 of uncompressed matrix 308d. Since the third index value of metadata 304 is 1, the third non-zero value of sparse matrix 302 (e.g., 5) is assigned to the 1st index bucket of non-zero 3 location 408d and ends up at index 3 of uncompressed matrix 308d. Since the fourth index value of metadata 304 is 3, the fourth non-zero value of sparse matrix 302 (e.g., 2) is assigned to the 3rd index bucket of non-zero 4 location 410d and ends up at index 7 of uncompressed matrix 308d.

[0060] Although 4 specific formats are illustrated here, it should be understood that other formats can be used during compression and / or decompression of dense / sparse matrices.

[0061] Figure 5A is an example block diagram of compressing a dense matrix 502 into a fully expressive sparse matrix 516 with finite metadata 518, according to at least one embodiment. To perform compression using an N:M:Bb scheme, a dense matrix can be received and / or identified. The dense matrix can include at least M consecutive elements. The dense matrix can have K non-zero elements. N of the M elements can be preserved during compression and can be included in the sparse matrix. The N elements of the resulting sparse matrix can be a subset of the K non-zero elements of the dense matrix.

[0062] In some embodiments, the N elements are selected from the K non-zero elements based on an importance value of each element. For example, an importance value can be assigned to each of the K non-zero elements based on an importance criterion (e.g., absolute value, etc.). The N elements with the highest importance values can be selected from the K non-zero elements and included in the sparse matrix 516.

[0063] The number of metadata bits per non-zero, B, in the N:M:Bb compression scheme used for compression determines how the dense matrix is represented in the sparse matrix and corresponding metadata. We will discuss compressing a dense matrix using a 4:8:2b scheme, but it should be understood that other N:M:Bb compression schemes can also utilize the principles discussed herein.

[0064] Compressing a dense matrix using a 4:8:2b scheme means that 4 non-zero values out of 8 consecutive elements of a dense matrix (e.g., dense matrix 502) can be retained in a sparse matrix (e.g., sparse matrix 516) and each non-zero value will have 2 bits of metadata that represents the index of that non-zero value in the dense matrix. In a given 4:8:2b compression scheme, each non-zero value can have 5 available positions (e.g., non-zero 1 position 508, non-zero 2 position 510, non-zero 3 position 512, non-zero 4 position 514), as represented by format 504. The first non-zero value and the second non-zero value can be combined into a first group and the third non-zero value and the fourth non-zero value can be combined into a second group. The first non-zero value and the second non-zero value can cover indices 0-5 of a total of 8 indices 506 of the dense matrix (or uncompressed matrix during decompression). The third non-zero value and the fourth non-zero value can cover indices 2-7 of the total of 8 indices 506 of the dense matrix (or uncompressed matrix during decompression). The discussion will focus on the first non-zero value and the second non-zero value, but these techniques can be applied in a similar manner to the third non-zero value and the fourth non-zero value.

[0065] Four of the 5 available positions for the first non-zero value and the second non-zero value can overlap and can cover indices 1-4 of the dense matrix. These 4 overlapping positions can share similar indices. For example, non-zero 1 position 508 includes 5 positions with indices 0-4: NZ1:4, NZ1:0, NZ1:1, NZ1:2, and NZ1:3. Non-zero 2 position 510 includes 5 positions with indices 0-4 as well: NZ2:0, NZ2:1, NZ2:2, NZ2:3, and NZ2:4. Indices 0-3 of non-zero 1 position 508 and non-zero 2 position 510 both correspond to indices 1-4 of the dense matrix. Index 4 of non-zero 1 position 508 can cover index 0 of the dense matrix, while index 4 of non-zero 2 position 510 can cover index 5 of the dense matrix.

[0066] The indices of the elements of dense matrix 502 having the highest importance values within indices 0-5 can be determined. For example, the element in index 0 (e.g., 8) and the element in index 1 (e.g., 7) can be selected. These positions correspond to metadata index (4, 0) using non-zero 1 positions 508 and non-zero 2 positions 510. Index 0 of dense matrix 502 corresponds to index 4 of non-zero 1 positions 508, and index 1 of dense matrix 502 corresponds to index 0 of non-zero 2 positions 510. The index value "4" cannot be expressed as a metadata index using only 2 bits of metadata, so the metadata will need to be "packed" as 2 bits.

[0067] The metadata can be packed using the following rules, which impose structure on the metadata. For a given metadata index pair (A, B), if A is 4 and B is less than or equal to 1, A can be assigned the same value as B. For example, in the example above, the metadata index is (4, 0). This value can be "packed" (e.g., transformed, compressed, etc.) as the value (0, 0) (because A = 4 and B <= 1, so A = B = 0).

[0068] For a given metadata index pair (A, B), if A is 4 and B is greater than or equal to 2, the value of A can be incremented by 1 modulo 5 (to limit the value to between 0 and 4, inclusive), the value of B can be decremented by 1 modulo 5, and A and B can be swapped. For example, if the metadata index is (4, 3), A can be incremented to 5 and "wrap around" (due to the modulo operator) to 0, B can be decremented to 2, and A and B can be swapped, resulting in the "packed" metadata index pair (2, 0).

[0069] For a given metadata index pair (A, B), if A is not 4 and B is 4, and A is greater than or equal to 2, B can be assigned the same value as A. For example, if the metadata index is (2, 4), B can be assigned the same value as A, resulting in the metadata index (2, 2).

[0070] For a given metadata index pair (A, B), if A is not 4 and B is 4, and A is less than or equal to 1, A can be incremented by 1 modulo 5, B can be decremented by 1 modulo 5, and A and B can be swapped. For example, if the metadata index is (1, 4), A can be incremented to 2, B can be decremented to 3, and A and B can be swapped, resulting in the "packed" metadata index pair (3, 2).

[0071] In the above scenario, the metadata index pairs can be "packed" before being stored with the sparse matrix. In any other scenario, the metadata index pairs (A, B) can be stored without modification. When accessing the sparse matrix and the corresponding metadata (e.g., during decompression or sparse matrix mathematical operations), the metadata can be "unpacked" if necessary. The "packing" scheme discussed above makes it easy to determine whether the metadata was initially packed. See below. Figure 5B An example of decompression.

[0072] The above metadata packaging rules can be expressed using the following pseudocode:

[0073] For a pair of indices (A, B)

[0074] If A = 4:

[0075] If B <= 1: Set A to B

[0076] If B >= 2: ++A mod 5, --B mod 5, swap (A, B)

[0077] Otherwise, if B = 4:

[0078] If A >= 2: Set B to A

[0079] If A <= 1: ++A mod 5, --B mod 5, swap (A, B)

[0080] exist Figure 5A In the example shown, dense matrix 502 is compressed using a 4:8:2b compression scheme. The four elements with the maximum values ​​(e.g., 8 at index 0, 7 at index 1, 5 at index 3, and 6 at index 7) are compressed into a sparse matrix 516 with corresponding metadata 518. As discussed above, the first two selected non-zero values ​​to be compressed (e.g., 8 and 7) have corresponding metadata indices 4 and 0, respectively (based on their corresponding available non-zero positions (e.g., non-zero position 1 508 and non-zero position 2 510)). The metadata pair (4,0) can be "packed" into the pair (0,0) for storage according to the rules described above, as shown by the first two elements of metadata 518.

[0081] The next two selected non-zero values ​​to be compressed (e.g., 5 and 6) have corresponding metadata indices 0 and 4 (based on their corresponding available non-zero positions (e.g., non-zero position 3 512 and non-zero position 4 514)). The metadata pair (0,4) can be "packed" into the pair (3,1) for storage according to the rules described above, as shown in the last two elements of metadata 518. Figure 5BThe depicted example will undergo decompression of the sparse matrix 516 and metadata 518 to obtain an uncompressed matrix (e.g., uncompressed matrix 520) corresponding to the dense matrix 502.

[0082] The above metadata unpacking rules can be expressed in the following pseudocode:

[0083] For a pair of indices (A, B)

[0084] If A < B: no modification

[0085] If A == B:

[0086] If A <= 1: set A to 4

[0087] If A >= 2: set B to 4

[0088] If A > B: swap (A, B), --A mod 5, ++B mod 5

[0089] In Figure 5B In the depicted example, the sparse matrix 516 and corresponding metadata 518 are decompressed using a 4:8:2b decompression scheme. The 4 non-zero elements of the sparse matrix 516 (e.g., 8, 7, 5, and 6) are decompressed to the uncompressed matrix 520. As noted above, the first two metadata index values (e.g., 0 and 0) can be evaluated as the pair (0, 0), which can be “unpacked” to the pair (4, 0) according to the above rules. Thus, the first non-zero value of the sparse matrix 516 (e.g., 8) can be assigned to index 4 of the non-zero 1 position 508, which corresponds to index 0 of the uncompressed matrix 520, and the second non-zero value of the sparse matrix 516 (e.g., 7) can be assigned to index 0 of the non-zero 2 position 510, which corresponds to index 1 of the uncompressed matrix 520.

[0090] The next two metadata index values (e.g., 3 and 1) can be evaluated as the pair (3, 1), which can be “unpacked” to the pair (0, 4) according to the above rules. Thus, the third non-zero value of the sparse matrix 516 (e.g., 5) can be assigned to index 0 of the non-zero 3 position 512, which corresponds to index 3 of the uncompressed matrix 520; and the fourth non-zero value of the sparse matrix 516 (e.g., 6) can be assigned to index 4 of the non-zero 4 position 514, which corresponds to index 7 of the uncompressed matrix 520.

[0091] In some embodiments, more than 2 non-zero elements and / or metadata index values can be evaluated at a time (e.g., in groups of 3, in groups of 4, etc.) during compression and / or decompression.

[0092] Figure 5Bis an example block diagram of decompressing a fully expressive sparse matrix 516 having finite element data 518 into an uncompressed matrix 520 according to at least one embodiment. The sparse matrix 516 and corresponding metadata 518 can be the result of compressing a dense matrix 502. To perform decompression using an N:M:Bb scheme, the sparse matrix and corresponding metadata can be received and / or identified. The sparse matrix can include N non-zero elements and can be decompressed into an M element matrix. Figure 5A

[0093] We will discuss decompressing a sparse matrix using a 4:8:2b scheme, but it should be understood that other N:M:Bb decompression schemes can also utilize the principles discussed herein.

[0094] The sparse matrix 516 and corresponding metadata 518 can be decompressed to obtain an uncompressed matrix 520. The uncompressed matrix 520 can include the N non-zero elements from the sparse matrix 516 as well as M-N other elements (e.g., zero value elements, random value elements, etc.).

[0095] In a given 4:8:2b decompression scheme, there can be 5 available positions for each non-zero value (e.g., non-zero 1 position 508, non-zero 2 position 510, non-zero 3 position 512, non-zero 4 position 514). The first non-zero value and the second non-zero value can be combined into a first group and the third non-zero value and the fourth non-zero value can be combined into a second group. The metadata values can correspond to indices within the available positions of the non-zero positions of the respective groups. As discussed above, some metadata values can be “packed” to express a non-zero value in index 4 of one of the non-zero positions. When metadata is packed according to the rules discussed above, the metadata values can be “unpacked” according to the following rules. Figure 5A

[0096] As discussed above, metadata indices can be evaluated in pairs. For a given metadata index pair (A, B), if A is less than B, the metadata can not need to be unpacked and the indices directly correspond to the respective non-zero position indices. For example, given a metadata index pair (1, 3) for the first 2 non-zero values of a sparse matrix, the first non-zero value of the sparse matrix can be assigned to index 2 of the uncompressed matrix because index 1 of the non-zero 1 position 508 corresponds to index 2 of the uncompressed matrix; and the second non-zero value of the sparse matrix can be assigned to index 4 of the uncompressed matrix because index 3 of the non-zero 2 position 510 corresponds to index 4 of the uncompressed matrix.

[0097] ​​For a given metadata index pair (A, B), if A is equal to B and A is less than or equal to 1, the A index can be set to 4. For example, the "packed" metadata index pair (1, 1) can be "unpacked" to (4, 1). This means that the first non-zero value of the sparse matrix can be assigned to index 0 of the uncompressed matrix because the index 4 of the non-zero 1 position 508 corresponds to index 0 of the uncompressed matrix, and the second non-zero value of the sparse matrix can be assigned to index 2 of the uncompressed matrix because the index 1 of the non-zero 2 position 510 corresponds to index 2 of the uncompressed matrix.

[0098] For a given metadata index pair (A, B), if A is equal to B and A is greater than or equal to 2, the B index can be set to 4. For example, the "packed" metadata index pair (3, 3) can be "unpacked" to (3, 4). This means that the first non-zero value of the sparse matrix can be assigned to index 4 of the uncompressed matrix because the index 3 of the non-zero 1 position 508 corresponds to index 4 of the uncompressed matrix, and the second non-zero value of the sparse matrix can be assigned to index 5 of the uncompressed matrix because the index 4 of the non-zero 2 position 510 corresponds to index 5 of the uncompressed matrix.

[0099] For a given metadata index pair (A, B), if A is greater than B, A and B can be swapped, the new A can be decremented by 1 modulo 5, and the new B can be incremented by 1 modulo 5. For example, the "packed" metadata index pair (2, 0) can be "unpacked" to (4, 3) by swapping 2 and 0, decrementing 0 by 1 and "wrapping around" to 4 (as a result of the modulo operator), and incrementing 2 to 3. This means that the first non-zero value of the sparse matrix can be assigned to index 0 of the uncompressed matrix because the index 4 of the non-zero 1 position 508 corresponds to index 0 of the uncompressed matrix, and the second non-zero value of the sparse matrix can be assigned to index 4 of the uncompressed matrix because the index 3 of the non-zero 2 position 510 corresponds to index 4 of the uncompressed matrix.

[0100] The same rule can be applied to the second pair of metadata values in metadata 518. Thus, if the A index in the metadata index pair (A, B) is greater than or equal to the B index, the metadata values can be unpacked, otherwise they need not be modified. These (un)packing rules can be based on the following principles: 1) two metadata indices with corresponding to the same dense matrix index (or uncompressed matrix index) are redundant; 2) metadata pair (0, 1) is equal to metadata pair (1, 0). Thus, for the first principle, it can be decided that metadata indices with the same value can be given a special meaning. And, for the second principle, it can be decided that the metadata index values arranged in ascending (or descending) order can be applied directly, while the metadata index values arranged in descending (or ascending) order can be given a special meaning. Thus, when the metadata indices are the same or A is greater than or equal to B (e.g., the metadata index values are arranged in descending order), the special meaning (as indicated by the rules above) can be applied.

[0101] Figure 6 is an example block diagram of computing a dot product between a sparse matrix 606 with corresponding metadata 608 and a matrix operand 616, according to at least one embodiment. The dense matrix 602 can be converted into the sparse matrix 606 and the corresponding metadata 608 using the compression 604 according to a given N:M:P, F compression scheme and / or a given N:M:Bb compression scheme. In some embodiments, the compression 604 is performed by the sparse matrix subsystem 106 of Figure 1 The sparse matrix 606 and the metadata 608 can include a plurality of elements. A subset of elements of the sparse matrix 606 and the metadata 608 (e.g., a sparse matrix row 610) can be provided to the tensor core 632. In some embodiments, the subset of elements from the sparse matrix 606 and the metadata 608 can be a column of elements of the sparse matrix 606 and the corresponding metadata 608 or some other subset of elements.

[0102] The tensor core 632 can perform one or more matrix mathematical operations on the sparse matrix row 610. For example, the tensor core 632 can receive a subset of elements from the operand 616 (e.g., an operand column 618). In some embodiments, the subset of elements from the operand 616 can be a row of elements of the operand 616 or some other subset of elements. In some embodiments, the tensor core 632 can be included in a central processing unit (CPU) or a graphics processing unit (GPU).

[0103] The tensor core 632 can include a control circuit 620 and one or more selection circuits (e.g., selector 622, selector 624). The control circuit 620 can receive the sparse matrix row metadata 612 and convert the index values of the sparse matrix row metadata 612 to control signals for the one or more selection circuits. In some embodiments, the selector 622 and / or the selector 624 can be multiplexers. For example, the selector 622 and the selector 624 can each be 4:2 multiplexers. In some embodiments, a single 8:4 multiplexer can be used. In some embodiments, 4 8: 1 multiplexers can be used, or any other combination that produces 4 output values. In some embodiments, 4 5: 1 multiplexers can be used to implement a 4:8:2b (de)compression scheme.

[0104] The selector 622 and the selector 624 can select a subset of the operand column 618 based on the sparse matrix row metadata 612 (e.g., based on control signals generated by the control circuit 620 based on the sparse matrix row metadata 612). For example, the sparse matrix row metadata 612 can include indices of the operand column 618 that should be retained. In some embodiments, the control circuit 620 can modify one or more values of the sparse matrix row metadata 612 (e.g., bit shifts, additions, subtractions, etc.) before providing these values to the selector 622 and / or the selector 624. For example, in some embodiments, one or more values of the sparse matrix row metadata 612 can be “unpacked” according to the rules discussed above in connection with the compression 604, and then provided to the selector 622 and / or the selector 624. Figure 5B The selector 622 and the selector 624 can select a subset of the operand column 618 based on the sparse matrix row metadata 612 (e.g., based on control signals generated by the control circuit 620 based on the sparse matrix row metadata 612). For example, the sparse matrix row metadata 612 can include indices of the operand column 618 that should be retained. In some embodiments, the control circuit 620 can modify one or more values of the sparse matrix row metadata 612 (e.g., bit shifts, additions, subtractions, etc.) before providing these values to the selector 622 and / or the selector 624. For example, in some embodiments, one or more values of the sparse matrix row metadata 612 can be “unpacked” according to the rules discussed above in connection with the compression 604, and then provided to the selector 622 and / or the selector 624.

[0105] In some embodiments, the selector 622 and / or the selector 624 can select a subset of elements from the operand column 618 according to more than one compression scheme format. For example, in some embodiments, a format indicator can be included in the metadata 608 during the compression 604. The selector 622 and / or the selector 624 can be configured to interpret index values of the metadata 608 differently based on the format indicator in the metadata 608.

[0106] Figure 7 is an example block diagram of an integrated circuit 702 for performing matrix mathematical operations using sparse matrices with corresponding metadata, according to at least one embodiment. The integrated circuit 702 can include a selection circuit 704, an arithmetic logic unit (ALU) 706, a memory 708, and a memory 710. In some embodiments, the selection circuit 704 can include one or more multiplexers for selecting a subset of input values based on a selector value. The selection circuit 704 can be coupled to the memory 710. In some embodiments, the integrated circuit 702 includes more than one ALU (e.g., two arithmetic logic units (ALUs)). In some embodiments, the operands can be stored in the memory 710. In some embodiments, the selection circuit 704 can include a control circuit that causes the one or more multiplexers to select elements of a matrix operand based on indices included in metadata also provided to the selection circuit 704. For example, the metadata can include indices for each of N non-zero elements in a dense matrix, and the selection circuit 704 can select a subset of elements from the operands based on these indices. The selection circuit 704 can also be coupled to the ALU 706. The ALU 706 can be capable of performing one or more mathematical operations on given inputs. For example, the ALU 706 can perform one or more matrix mathematical operations (e.g., a dot product). In some embodiments, the ALU 706 can be coupled to the memory 708, which can store one or more dense matrices, sparse matrices, and / or corresponding metadata.

[0107] In some embodiments, the integrated circuit 702 can be part of a tensor core circuit of a processor. In some embodiments, the processor is a central processing unit (CPU) or a graphics processing unit (GPU).

[0108] Figure 8 is a flowchart of an example method 800 for compressing a dense matrix into an expressive sparse matrix with limited metadata, according to at least one embodiment. Figure 9 is a flowchart of an example method 900 for decompressing an expressive sparse matrix with limited metadata into an uncompressed matrix, according to at least one embodiment. Figure 10 is a flowchart of an example method 1000 for computing a dot product using an expressive sparse matrix with limited metadata, according to at least one embodiment. Figure 11 is a flowchart of an example method 1100 for performing a matrix multiplication operation using an expressive sparse matrix with limited metadata, according to at least one embodiment.

[0109] Methods 800, 900, 1000, and / or 1100 may be executed using one or more processing units (e.g., CPU, GPU, accelerator, physical processing unit (PPU), data processing unit (DPU), etc.), which may include one or more memory devices (or communicate with them). In at least one embodiment, methods 800, 900, 1000, and / or 1100 may be executed using one or more processing devices. In at least one embodiment, methods 800 and / or 900 may use... Figure 1 The system 102's processing unit executes the method. In at least one embodiment, method 1000 and / or 1100 can be performed by... Figure 6 Tensor kernel 632 and / or Figure 7 The integrated circuit 702 is used to execute the method. In at least one embodiment, the processing unit executing any of methods 800, 900, 1000, and / or 1100 can execute instructions stored on a non-transitory computer-readable storage medium. In at least one embodiment, any of methods 800, 900, 1000, and / or 1100 can be executed using multiple processing threads (e.g., CPU threads and / or GPU threads), each thread executing one or more individual functions, routines, subroutines, or operations of the method. In at least one embodiment, the processing threads implementing any of methods 800, 900, 1000, and / or 1100 can be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, the processing threads implementing any of methods 800, 900, 1000, and / or 1100 can execute asynchronously with each other. The various operations of methods 800, 900, 1000, and / or 1100 can be performed in accordance with... Figure 8 , Figure 9 , Figure 10 and / or Figure 11 The different sequences shown are executed sequentially. Some operations of any of methods 800, 900, 1000, and / or 1100 can be executed concurrently with other operations. In at least one embodiment, Figure 8 , Figure 9 , Figure 10 and / or Figure 11 One or more operations shown may not always be performed.

[0110] Figure 8is a flowchart of an example method 800 for compressing a dense matrix into an expressively sparse matrix having limited element data, according to at least one embodiment. At block 802, a processing unit executing the method 800 can generate a sparse matrix having corresponding metadata based on a dense matrix. In some embodiments, to generate the sparse matrix having corresponding metadata, at block 804, the processing unit can identify a first number of elements (M) to be compressed, a second number of elements (N) to be preserved, a third number of positions (P), and a format. In some embodiments, P is less than M. At block 806, the processing unit can determine a metadata value for each of the N elements of the dense matrix based on the identified P and the identified format. The dense matrix can include at least M elements. At block 808, the processing unit can generate a sparse matrix including the N elements of the dense matrix. At block 810, the processing unit executing the method 800 can store the sparse matrix and the corresponding metadata.

[0111] The corresponding metadata can include the metadata value for each of the N elements of the dense matrix. In some embodiments, the metadata value for each of the N elements of the dense matrix indicates an index corresponding to one of the P positions associated with the identified format. In some embodiments, the sparse matrix is stored in memory with the corresponding metadata. In some embodiments, the sparse matrix and the corresponding metadata are stored in separate memory locations.

[0112] In some embodiments, the processing unit can also identify a fourth number of non-zero elements (K) of the dense matrix and identify the N elements of the dense matrix from the K elements based on an importance criterion. In some embodiments, to identify the N elements of the dense matrix based on the importance criterion, the processing unit can select the N elements of the dense matrix that maximize an importance value of the sparse matrix. In some embodiments, the processing unit can further decompress the sparse matrix to obtain an uncompressed sparse matrix having at least the N elements of the dense matrix. In some embodiments, to decompress the sparse matrix to obtain the uncompressed sparse matrix, the processing unit can initialize a matrix having M elements and assign values to the N elements of the matrix based on the sparse matrix and the corresponding metadata. The corresponding metadata can indicate, based on the identified P and the identified format, a position in the matrix at which each value of the sparse matrix occurs.

[0113] Figure 9is a flowchart of an example method 900 for decompressing an expressive sparse matrix having limited metadata into an uncompressed matrix, according to at least one embodiment. At block 902, a processing unit executing the method 900 can receive a sparse matrix and metadata corresponding to the sparse matrix. The sparse matrix can be a compressed representation of a dense matrix. The sparse matrix can include a first number (N) of elements to be preserved in the dense matrix, while the dense matrix can include at least a second number (M) of elements. The metadata corresponding to the sparse matrix can be based on a third number (P) of positions and a format determined during compression of the dense matrix. In some embodiments, P is less than M. At block 904, the processing unit can generate an uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix. In some embodiments, the metadata corresponding to the sparse matrix can include an index for each of the N elements to be preserved in the dense matrix. The index can correspond to one of the P positions associated with the format determined during compression of the dense matrix.

[0114] In some embodiments, the dense matrix can include a fourth number (K) of non-zero elements, and the N elements to be preserved in the dense matrix represent a subset of the K non-zero elements. In some embodiments, the uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix can include at least the N elements of the dense matrix. In some embodiments, the uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix can include the M elements. In some embodiments, the processing unit executing the method 900 can further perform one or more matrix multiplication operations on the uncompressed matrix based on the sparse matrix, the metadata corresponding to the sparse matrix, and a matrix operand.

[0115] Figure 10 is a flowchart of an example method 1000 for computing a dot product using an expressive sparse matrix having limited metadata, according to at least one embodiment. At block 1002, a processing unit executing the method 1000 can receive a sparse matrix, metadata corresponding to the sparse matrix, and a matrix operand. The sparse matrix can be a compressed representation of a dense matrix. The sparse matrix can include a first number (N) of elements to be preserved in the dense matrix, while the dense matrix can include at least a second number (M) of elements. The metadata corresponding to the sparse matrix can be based on a third number (P) of positions and a format determined during compression of the dense matrix. In some embodiments, P is less than M. At block 1004, the processing unit can compute at least a first dot product between a first element of the sparse matrix and a second element of the matrix operand. In some embodiments, the metadata corresponding to the sparse matrix includes an index for each of the N elements to be preserved in the dense matrix. In some embodiments, the index can correspond to one of the P positions associated with the format determined during compression of the dense matrix.

[0116] In some embodiments, to compute at least a first dot product between a first element of the sparse matrix and a second element of the matrix operand, the processing unit can select, based on an index of each element of the N elements of the dense matrix to be retained, a subset of elements of the matrix operand, the subset of elements of the matrix operand being the second element of the matrix operand. In some embodiments, the dense matrix can include a fourth number (K) of non-zero elements, and the N elements of the dense matrix to be retained can represent a subset of the K non-zero elements. In some embodiments, the N elements of the dense matrix to be retained are identified based on an importance value of each element of the M elements. In some embodiments, the sparse matrix and the metadata corresponding to the sparse matrix are stored in separate memory locations.

[0117] Figure 11 is a flowchart of an example method 1100 for performing matrix multiplication operations using an expressive sparse matrix with limited metadata, according to at least one embodiment. At block 1102, a processing unit performing the method 1100 can receive a sparse matrix, metadata corresponding to the sparse matrix, and a matrix operand. The sparse matrix can include a first number (N) of elements of a dense matrix to be retained, the dense matrix can include at least a second number (M) of elements. The metadata corresponding to the sparse matrix can be based on a third number (P) of locations and a format determined during compression of the dense matrix. In some embodiments, P is less than M. At block 1104, the processing unit can select, via one or more selection circuits, a subset of elements of the matrix operand based on the metadata corresponding to the sparse matrix. At block 1106, the processing unit can perform one or more matrix multiplication operations on the sparse matrix and the subset of elements of the matrix operand.

[0118] In some embodiments, the method 1100 can be performed by the integrated circuit 702 of Figure 7 In some embodiments, the metadata corresponding to the sparse matrix can include an index of each element of the N elements of the dense matrix to be retained. The index can correspond to one of the P locations associated with the format determined during compression of the dense matrix. In some embodiments, the dense matrix can include a fourth number (K) of non-zero elements, and the N elements of the dense matrix to be retained can represent a subset of the K non-zero elements. In some embodiments, the N elements of the dense matrix to be retained are identified based on an importance value of each element of the M elements. In some embodiments, the sparse matrix and the metadata corresponding to the sparse matrix are stored in separate memory locations.

[0119] Figure 12is a flowchart of an example method 1200 for compressing a dense matrix into a fully expressive sparse matrix with limited element data, according to at least one embodiment. At block 1202, a processing unit executing the method 1200 can generate a sparse matrix with corresponding metadata based on a dense matrix. In some embodiments, to generate the sparse matrix with corresponding metadata, at block 1204, the processing unit can identify a first number (M) of elements to be compressed, a second number (N) of elements to be preserved, and a third number (B) indicating a number of bits used by each metadata value. At block 1206, the processing unit can determine a metadata value for each of the N elements of the dense matrix. The dense matrix can include at least M elements. At block 1208, the processing unit can pack first metadata values having more than B bits into second metadata values having B bits. At block 1210, the processing unit can generate a sparse matrix including the N elements of the dense matrix. At block 1212, the processing unit executing the method 1200 can store the sparse matrix and the corresponding metadata.

[0120] The corresponding metadata can include the second metadata values having B bits. In some embodiments, the metadata value for each of the N elements of the dense matrix indicates an index corresponding to one of a plurality of positions available to each of the N elements of the dense matrix. In some embodiments, the sparse matrix is stored in a memory with the corresponding metadata. In some embodiments, the sparse matrix is stored in separate memory locations from the corresponding metadata.

[0121] In some embodiments, the processing unit can also identify a fourth number (K) of non-zero elements of the dense matrix and identify the N elements of the dense matrix from the K elements based on an importance criterion. In some embodiments, to identify the N elements of the dense matrix based on the importance criterion, the processing unit can select the N elements of the dense matrix that maximize an importance value of the sparse matrix. In some embodiments, the processing unit can further decompress the sparse matrix to obtain an uncompressed sparse matrix having at least the N elements of the dense matrix. In some embodiments, to decompress the sparse matrix to obtain the uncompressed sparse matrix, the processing unit can initialize a matrix having M elements, unpack the second metadata values having B bits into first metadata values having more than B bits, and assign values to first elements of the matrix based on the sparse matrix and the first metadata values having more than B bits.

[0122] Although in some embodiments, the M elements of the dense matrix to be compressed are described as consecutive elements, in other embodiments, the M elements of the dense matrix are not consecutive. For example, the M elements of the dense matrix to be compressed can be every other element, every third element, etc.

[0123] Although in some embodiments the N elements of the dense matrix to be preserved have been described as non-zero elements, in other embodiments the N elements of the dense matrix include at least one zero-valued element. For example, the N elements of the dense matrix to be preserved can all be zero. Similarly, although in some embodiments the P positions have been described as non-zero positions, in other embodiments the P positions are positions available for each of the N elements to be preserved, regardless of whether the particular element is non-zero.

[0124] Other variations are within the spirit of the present disclosure. Thus, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the disclosure to the specific form or forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the disclosure, as defined in the appended claims.

[0125] The use of the terms "a" and "an" and "the" and similar referents in the context of describing the disclosed embodiments (especially in the context of the following claims) are to be interpreted in the context to encompass both singular and plural, and can not be limited to a single referent unless otherwise indicated by the context. The terms "comprising," "having," "including," and "containing" are to be construed as open-ended terms (meaning including, but not limited to) unless otherwise noted. The term "connected" (as used in the context of the present disclosure) is to be construed as partly or fully encompassed, attached, or joined together, even if there are some intervening materials or structures, unless otherwise indicated by the context. Unless otherwise indicated herein, the use of numerical ranges in the description herein are intended to use as shorthand method of referring individually to each separate value included in the range, and each separate value is incorporated into the specification as if it were individually recited herein. In at least one embodiment, the use of the term "set" (e.g., "set of items") or "subset" is to be construed as a non-empty set of one or more members, unless otherwise indicated or contradicted by context. Further, the term "subset" of a corresponding set does not necessarily denote a proper subset of the corresponding set, but rather the subset and the corresponding set can be equal, unless otherwise indicated or contradicted by context.

[0126] Unless explicitly stated otherwise or apparent from context, a phrase such as "at least one of A, B, and C" or "at least one of A, B, or C" shall indicate that the group is inclusive of any of the items, elements, etc. individually or any combination of the items, elements, etc. For example, the phrases "at least one of A, B and C" and "at least one of A, B, or C" shall cover: (a) A alone, (b) B alone, (c) C alone, (d) at least one of A and B together, (e) at least one of A and C together, (f) at least one of B and C together, and (g) all of A, B, and C together. In other words, the phrase "at least one of A, B, and C" or "at least one of A, B, or C" shall mean that the group is an inclusive- or group. In addition, unless otherwise stated or apparent from context, the term "plurality" shall indicate a state of more than one (i.e., at least two). In at least one embodiment, a plurality of items can be at least two items, but if explicitly indicated or otherwise apparent from context, can be more than two items. Furthermore, unless otherwise stated or apparent from context, the phrase "based on" is intended to mean "based, at least in part, on" or "based on at least." For example, "based on the input" is intended to mean "based, at least in part, on the input."

[0127] The operations of a process described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. In at least one embodiment, a process, such as those described herein (or variations and / or combinations thereof) is performed under the control of one or more computer systems configured with executable instructions to perform the operations of the process, and the process is implemented as code (e.g., executable instructions, one or more computer programs or one or more applications) that is collectively executed by a hardware or combination of hardware and / or software. In at least one embodiment, the code is stored on a computer-readable storage medium, such as a computer program product, which is readable by a computer system including one or more processors. In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues). In at least one embodiment, the code (e.g., executable instructions or source code) is stored on a set of one or more non-transitory computer-readable storage media (or other memory for storing executable instructions) having stored thereon executable instructions that, as a result of being executed by one or more processors of a computer system (i.e., as a result of being executed), cause the computer system to perform operations described herein. In at least one embodiment, a set of non-transitory computer-readable storage media includes multiple non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media in the multiple non-transitory computer-readable storage media lack all of the code, with the multiple non-transitory computer-readable storage media collectively storing the entire code. In at least one embodiment, executable instructions are executed by different processors, e.g., a non-transitory computer-readable storage medium stores instructions and a main central processing unit (“CPU”) executes some instructions, while a graphics processing unit (“GPU”) executes other instructions. In at least one embodiment, different components of a computer system have separate processors and different processors execute different subsets of the instructions.

[0128] Accordingly, in at least one embodiment, a computer system is configured to implement one or more services that individually or collectively perform operations of processes described herein, and such a computer system is configured with applicable hardware and / or software to enable implementation of the operations. Moreover, a computer system implementing at least one embodiment of the present disclosure is a single device, and in another embodiment is a distributed computer system including multiple devices operating in different manners such that the distributed computer system performs operations described herein and such that a single device does not perform all of the operations.

[0129] The use of any and all examples, or exemplary language (e.g., "such as") provided herein, is intended merely to better illuminate embodiments of the disclosure and does not pose a limitation on the scope of the disclosure unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0130] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0131] In the description and claims, the terms "coupled" and "connected," along with derivatives thereof, can be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, "connected" or "coupled" can be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.

[0132] In some embodiments, unless specifically stated otherwise, it can be appreciated that throughout the specification terms such as "processing," "computing," "calculating," "determining," or the like, refer to the action and / or processes of a computer or computing system or similar electronic computing device, that manipulate and / or transform data represented as physical, such as electronic, quantities within the computing system's registers and / or memories into other data similarly represented as physical quantities within the computing system's memories, registers or other such information storage, transmission or display devices.

[0133] In a similar manner, the term "processor" can refer to any device or portion of a device that processes electronic data, from registers and / or memory to transform that electronic data into other electronic data that can be stored in registers and / or memory. As a non-limiting example, a "processor" can be a CPU or GPU. A "computing platform" can include one or more processors. As used herein, a "software" process can include, for example, software and / or hardware entities such as tasks, threads, and intelligent agents that perform work over time. Likewise, each process can refer to multiple processes to sequentially or concurrently execute instructions, either continuously or intermittently. In at least one embodiment, the terms "system" and "method" can be used interchangeably herein, as long as the system can embody one or more methods, and the method can be considered a system.

[0134] In this document, obtaining, acquiring, receiving, or importing analog or digital data into a subsystem, computer system, or computer-implemented machine can be referenced. In at least one embodiment, the process of obtaining, acquiring, receiving, or importing analog and digital data can be accomplished in a variety of ways such as, for example, by receiving the data as a parameter to a function call or call to an application programming interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or importing analog or digital data can be accomplished by transferring the data via a serial or parallel interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or importing analog or digital data can be accomplished by transferring the data from a providing entity to an acquiring entity via a computer network. In at least one embodiment, providing, outputting, transferring, sending, or presenting analog or digital data can also be referenced. In various examples, the process of providing, outputting, transferring, sending, or presenting analog or digital data can be accomplished by transferring the data as an input or output parameter to a function call, a parameter to an application programming interface, or an interprocess communication mechanism.

[0135] Although the description herein sets forth example implementations of the described technology, other architectures can be used to implement the described functionality, and are intended to fall within the scope of the present disclosure. Moreover, although specific distributions of responsibilities are defined above for the components, for purposes of description only, in other implementations, the various functions and responsibilities described herein can be distributed among components in a different manner.

[0136] Further, although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

1. A method comprising: receiving a sparse matrix and metadata corresponding to the sparse matrix, wherein: the sparse matrix is a compressed representation of a dense matrix; the sparse matrix contains a first number N of elements of the dense matrix to be preserved, the dense matrix comprising at least a second number M of elements; and the metadata corresponding to the sparse matrix is a third number P that is position-based and a format determined during compression of the dense matrix; and generating an uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix.

2. The method of claim 1, wherein P is less than M.

3. The method of claim 1, wherein the metadata corresponding to the sparse matrix comprises an index of each of the N elements of the dense matrix to be preserved, wherein the index corresponds to one of P positions associated with the format determined during compression of the dense matrix.

4. The method of claim 1, wherein the dense matrix contains a fourth number K of non-zero elements, and the N elements of the dense matrix to be preserved represent a subset of the K non-zero elements.

5. The method of claim 1, wherein the uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix comprises at least the N elements of the dense matrix.

6. The method of claim 1, wherein the uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix comprises M elements.

7. The method of claim 1, further comprising: performing one or more matrix multiplication operations on the uncompressed matrix based on the sparse matrix, the metadata corresponding to the sparse matrix, and a matrix operand.

8. A method comprising: receiving a sparse matrix, metadata corresponding to the sparse matrix, and a matrix operand, wherein: the sparse matrix is a compressed representation of a dense matrix; the sparse matrix contains a first number N of elements of the dense matrix to be preserved, the dense matrix comprising at least a second number M of elements; and the metadata corresponding to the sparse matrix is a third number P that is position-based and a format determined during compression of the dense matrix; and computing at least a first dot product between a first element of the sparse matrix and a second element of the matrix operand.

9. The method of claim 8, wherein P is less than M.

10. The method of claim 8, wherein the metadata corresponding to the sparse matrix comprises an index of each of the N elements of the dense matrix to be preserved, wherein the index corresponds to one of P positions associated with the format determined during compression of the dense matrix.

11. The method of claim 10, wherein computing at least the first dot product between the first element of the sparse matrix and the second element of the matrix operand comprises: selecting a subset of elements of the matrix operand based on the index of each of the N elements of the dense matrix to be preserved, the subset of elements of the matrix operand being the second element of the matrix operand.

12. The method of claim 8, wherein the dense matrix includes a fourth number K of non-zero elements, and the N elements of the dense matrix to be preserved represent a subset of the K non-zero elements.

13. The method of claim 8, wherein the N elements of the dense matrix to be preserved are identified based on an importance value of each of the M elements.

14. The method of claim 8, wherein the sparse matrix and the metadata corresponding to the sparse matrix are stored in separate memory locations.

15. A system comprising: a memory; and a processor coupled to the memory for performing operations comprising: receiving a sparse matrix and metadata corresponding to the sparse matrix, wherein: the sparse matrix is a compressed representation of a dense matrix; the sparse matrix includes a first number N of elements of the dense matrix to be preserved, the dense matrix including at least a second number M of elements; and the metadata corresponding to the sparse matrix is based on a third number P of locations and a format determined during compression of the dense matrix; and generating an uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix.

16. The system of claim 15, wherein P is less than M.

17. The system of claim 15, wherein the metadata corresponding to the sparse matrix includes an index of each of the N elements of the dense matrix to be preserved, wherein the index corresponds to one of the P locations associated with the format determined during compression of the dense matrix.

18. The system of claim 15, wherein the dense matrix includes a fourth number K of non-zero elements, and the N elements of the dense matrix to be preserved represent a subset of the K non-zero elements.

19. The system of claim 15, wherein the uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix includes at least the N elements of the dense matrix.

20. The system of claim 15, wherein the uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix includes M elements. ​

Images