Apparatus and method for matrix operations

Abstract

A data processing apparatus is provided that performs an operation on a first matrix and a second matrix. The data processing apparatus includes a receiver circuit that receives elements of the first matrix, elements of the second matrix, and correspondence data indicating where the elements of the first matrix are located within the first matrix. A decision circuit uses the correspondence data to make a decision as to whether, for a given element of the first matrix in column i of the first matrix, a given element of the second matrix occurs in row i of the second matrix. The aggregation circuit includes a function circuit that calculates an aggregation between a given row of the first matrix and a given column of the second matrix, and, in response to the decision, performs a function on the given element of the first matrix and the given element of the second matrix to generate a partial result.

Description

[Technical Field]

[0001] This disclosure relates to data processing. [Background technology]

[0002] When performing matrix operations such as matrix multiplication, numerous substeps may be executed. In many cases, these substeps have little impact on the overall matrix operation. For example, when multiplying two sparse matrices (those containing a relatively large proportion of entries with zero values), numerous multiplications by zero are performed. This is particularly true in convolutional neural networks (CNNs), often used for image classification. While multiplication by zero can be wasteful as it results in a zero outcome, the process of actually performing the multiplication can still consume circuit time and power. Furthermore, storing such matrices can use a significant amount of memory space. Therefore, it is desirable to perform such operations using fewer substeps. It is also desirable that such matrices be stored using less memory space, thereby reducing resource consumption, as well as power and potentially circuit size. [Overview of the project]

[0003] From the first exemplary configuration, the first matrix and the second matrix against A data processing device is provided to perform calculations. The data processing device includes a receiver circuit that receives corresponding data indicating the elements of a first matrix, the elements of a second matrix, and the location of the elements of the first matrix within the first matrix. Using the corresponding data, the device determines that for a given element of the first matrix at row i, column ji, one of the given elements of the second matrix is ​​at row j, column i of the second matrix. existence The system includes a determination circuit that performs a determination of whether or not to perform an action, and an aggregation circuit that calculates an aggregation between a given row i of a first matrix and a given column i of a second matrix, and which includes a function circuit that performs a function on a given element of the first matrix and a given element of the second matrix in accordance with the determination to generate a partial result.

[0004] From the second exemplary configuration, the first matrix and the second matrix against A data processing method is provided for performing an operation, the method comprising: receiving an element of a first matrix, an element of a second matrix, and corresponding data indicating the location of the element of the first matrix within the first matrix; and using the corresponding data, determining that for a given element of the first matrix in column i of the first matrix, a given element of the second matrix in row i of the second matrix existence The process includes the steps of: performing a determination of whether or not to perform an action; and calculating an aggregation between a given row of a first matrix and a given column of a second matrix, wherein, depending on the determination, a partial result is generated by performing a function on a given element of the first matrix and a given element of the second matrix.

[0005] In a third exemplary configuration, a data processing device is provided, the data processing device comprising a decode circuit that receives one or more instructions including matrix operation instructions, and references matrix operation instruction reference elements of a first matrix, elements of a second matrix, and corresponding data indicating the location of the elements of the first matrix within the first matrix, and in response to receiving a matrix operation instruction, the decode circuit outputs the first matrix and the second matrix to one or more execution units against Generate one or more control signals to perform matrix operations. [Brief explanation of the drawing]

[0006] The present invention will be further described, merely as an example, with reference to the embodiments shown in the attached drawings. [Figure 1] Several exemplary data processing devices according to various embodiments are schematically shown. [Figure 2] This demonstrates matrix multiplication and summation operations and shows how the location of a specific element in a matrix can be represented. [Figure 3] A portion of the matrix in the compressed form, Figure 2, is shown along with corresponding data indicating how the elements of the compressed matrix are arranged within the (original) matrix. [Figure 4] Figure 3 shows how the compression matrix can be stored in a register. [Figure 5] The following describes circuits, according to several embodiments, that can perform matrix operations within the previously illustrated compressed matrix. [Figure 6] We will demonstrate a matrix multiplication operation where the second matrix is ​​uncompressible. [Figure 7] Figure 6 shows how the uncompressed second matrix is ​​stored in the register. [Figure 8] The following describes circuits, according to several embodiments, that can perform matrix operations within the previously illustrated compressed matrix. [Figure 9] The following diagrams show circuits for decoding matrix operation instructions according to several embodiments. [Figure 10] A flowchart illustrating data processing methods according to several embodiments is shown. [Modes for carrying out the invention]

[0007] Before examining embodiments with reference to the attached drawings, the following embodiments will be described.

[0008] According to some embodiments, the first matrix and the second matrix against A data processing device is provided to perform calculations, the data processing device includes a receiver circuit that receives corresponding data indicating the elements of a first matrix, the elements of a second matrix, and the location of the elements of the first matrix within the first matrix, and using the corresponding data, for a given element of the first matrix in column i of the first matrix, a given element of the second matrix in row i of the second matrix existence The system comprises a determination circuit that performs a determination of whether or not to perform an action, and an aggregation circuit that calculates an aggregation between a given row of a first matrix and a given column of a second matrix, and includes a function circuit that, in response to the determination, performs a function on a given element of the first matrix and a given element of the second matrix to generate a partial result.

[0009] Some or all of the elements of the first and second matrices are received along with corresponding data. The corresponding data is used to indicate where not all elements are provided and where provided elements are located in the first and / or second matrices. The determination circuit provides an indicator of whether two of the provided elements (one from the first matrix and one from the second matrix) will "interact" with each other so that when a specific operation (such as multiplication or addition) is performed between the first and second matrices, the two elements will interact with each other. For example, it indicates whether one of the given elements of the second matrix exists in row i for one of the given elements in column i of the first matrix. Based on this determination, the function circuit performs a function on the given elements of the first and second matrices. For example, if the two elements interact, the function is performed to produce a partial result; if the two elements do not interact, the function is not performed (or simply returns "0" as a partial result). The partial results are then aggregated together. Aggregation is performed with respect to a given row of the first matrix and a given column of the second matrix. In this way, it is possible to perform operations between the first matrix and the second matrix without providing all the elements of the first and second matrices. In particular, the interaction between the first matrix and the second matrix is existence By determining where to perform operations, it is possible to limit the total number of calculations that must be performed. Furthermore, since only the elements of the first matrix, the elements of the second matrix, and the corresponding data rather than the entire first and second matrices need to be stored, memory requirements can be reduced.

[0010] In some embodiments, the elements of the first matrix are the elements of a compressed form of the first matrix, which is generated by removing at least some zero entries from the first matrix and at least some columns or rows where each entry is zero. By removing some of the entries that have zeros and at least some of the columns or rows where each entry in that column or row is zero, it is possible to reduce the overall size of the matrix and the number of entries that need to be provided to the matrix. In some embodiments, the removal of non-zero values ​​may be used. However, it should be understood that in the case of a sparse matrix (where a high proportion of values ​​are zero), many operations may be performed where one of the operands is zero. Such operations can be time-consuming, even if the results are predictable or even irrelevant. By removing such values ​​from the matrix, the number of operations can be reduced.

[0011] In some embodiments, the correspondence data indicates the row and column numbers for each element of the first matrix, where the row and column numbers for a given element within the matrix correspond to the row and column in which the given element is located within the first matrix. Thus, the correspondence data indicates a method for transforming the elements of the first matrix into the first matrix. This can be used to compress the first matrix by omitting certain known values. Compression can then be achieved by providing only some of the elements of the matrix along with the locations where these elements actually lie, instead of providing all of the elements of the matrix.

[0012] In some embodiments, the receiver circuit is adapted to receive further correspondence data indicating the location of the elements of the second matrix within the second matrix, and the decision circuit is adapted to perform a decision using the further correspondence data in addition to the correspondence data. Similar to the correspondence data that provides location information for the elements of the first matrix, the further correspondence data provides location information for the elements of the second matrix.

[0013] In some embodiments, additional correspondence data indicates, for each element of the second matrix, a row number and a column number, and the row number and column number for a given element within an element correspond to the row and column in which the given element is located within the second matrix. Note that the correspondence data and the additional correspondence data are not required to have the same format or even exactly the same information.

[0014] In some embodiments, the elements of the second matrix are elements of a compressed form of the second matrix, and the compressed form of the second matrix is generated by removing at least some entries that are zero from the second matrix and removing at least some columns or rows in which each entry is zero. By providing both the first matrix and the second matrix in a compressed form, the amount of storage required to perform the two-matrix against operations can be reduced, and the number of calculations performed when performing the operations can also be reduced.

[0015] In some embodiments, the operation includes a multiplication operation on the first matrix and the second matrix, the aggregation circuit includes, as an aggregation, a dot product circuit that calculates the dot product of a given row and a given column, the functional circuit includes, as a function, a multiplication circuit that performs multiplication, and the dot product circuit includes an addition circuit that adds each partial result generated for a given row and a given column. Performing multiplication between two matrices can be time-consuming because it involves a large number of calculations being performed. However, in many cases, only some of the calculations are relevant. For example, the calculations of interest may be those in which one of the values is non-zero. Therefore, by compressing one or both of the matrices and using the correspondence data, it is possible to determine existence where the relevant interactions (e.g., between non-zero elements) will occur and perform those specific calculations. This can reduce the number of calculations performed.

[0016] In some embodiments, a receiver circuit is adapted to receive elements of a third matrix, and the operation involves adding the third matrix with the results of multiplication operations of the first and second matrices, and the adder circuit is adapted to add each sub-result generated for a given row and a given column together with one of the elements of the third matrix. In a multiply-accumulate operation, two operands are multiplied together, and the result is added to an "accumulator" that multiplies the results of a number of such multiplications.

[0017] In some embodiments, the multiplication circuit is such that any element of the second matrix is ​​within row i of the second matrix existence It is adapted to output the value "0" in response to the determination that it has not been done. None of the elements of the second matrix are within row i of the second matrix existence If not, there may be no relevant interactions with those elements. For example, any such interaction is with elements of the matrix that are not provided. This is, for example, when all elements are zero. existence It is possible. As a result of the calculation not being performed, a value of zero is output directly, thereby eliminating the need to perform a more complex calculation.

[0018] In some embodiments, the determination circuit determines if a given element of the second matrix is ​​located at a location other than row j and column i of the second matrix. existence The system is adapted to provide the value "0" as input to the multiplication circuit when the determination circuit makes such a determination. One way to cause the multiplication circuit to output the value "0" is to provide "0" as input to the multiplication circuit itself.

[0019] In some embodiments, the data processing device includes a storage circuit that stores elements of a first matrix, the storage circuit containing corresponding data. In such embodiments, the elements of the first matrix are stored together with the corresponding data, the corresponding data indicating where those elements appear in the first matrix.

[0020] In some embodiments, the memory circuit is a vector register. A vector register may consist of multiple regular registers for the purpose of storing large amounts of data. Vector registers are typically wired or organized together so that all data within the vector register can be accessed or provided simultaneously. In some cases, the vector register can be scalable so that its amount of storage can be varied, and therefore, in such embodiments, it is possible to limit the size of the vector register so that it is sufficient to store the matrix elements and corresponding data. In this way, it is possible to support a variety of different matrix sizes (or number of elements) without requiring a large vector register to handle worst-case scenarios.

[0021] In some embodiments, a memory circuit is provided for storing the elements of a second matrix, and the memory circuit includes further corresponding data. In a similar manner to the memory circuit used for storing the elements of the first matrix, the second matrix can also be stored in a memory circuit and stored together with further corresponding data. It should be noted that the memory circuits used for the first and second matrices may be the same or different.

[0022] In some embodiments, the memory circuit is a vector register. The vector register used to store the second matrix may be the same vector register used to store the first matrix, or it may be a different vector register.

[0023] In some embodiments, the width of the memory circuit is b bits, each element of the first matrix has a width of c bits, and the number of elements in the first matrix is ​​at most b / c. Therefore, the number of elements can be varied depending on the width of the elements and the size of the storage. For example, if each element is only 1 bit and the memory circuit is 32 bits, the number of elements in the first matrix can be up to 32.

[0024] In some embodiments, the width of the correspondence data is x bits for each element of the first matrix, and the number of elements in the first matrix is ​​at most b / (c+x). Correspondence data may be provided for each element to indicate that the element is in the first matrix. Such correspondence data can use a number of bits depending on the size of the first matrix. For example, if correspondence data is provided for each row (including m columns), the size of the correspondence data may be log2(m) bits for each element of the first matrix. Similarly, if correspondence data is provided for each column (including n rows), the size of the correspondence data may be log2(n) for each element of the first matrix. If the correspondence data is not provided on a row or column basis, and therefore each element of the first matrix must have a specified row and column, the correspondence data may be log2(m)+log2(n) bits.

[0025] In some embodiments, the width of the memory circuit is b bits, each element of the second matrix has a width of c bits, and the number of elements in the second matrix is ​​at most b / c. Similarly, the number of elements in the second matrix can be varied depending on the size of the memory circuit storing the elements of the second matrix, as well as the number of bits used to represent each element.

[0026] In some embodiments, the width of the further corresponding data is x bits for each element of the second matrix, and the number of elements in the second matrix is ​​at most b / (c+x). As above, the further corresponding data has a width that depends on the size of the second matrix.

[0027] According to some embodiments, a data processing device is provided, the data processing device comprising a decoding circuit that receives one or more instructions including a matrix operation instruction, the matrix operation instruction refers to corresponding data indicating the elements of a first matrix, the elements of a second matrix, and the location of the elements of the first matrix within the first matrix, and in response to receiving a matrix operation instruction, the decoding circuit outputs to one or more execution units the first matrix and the second matrix against Generate one or more control signals to perform matrix operations.

[0028] In such an embodiment, the first matrix and the second matrix against To perform the operation, it is possible to provide instructions that also refer to the elements of the first matrix and the elements of the second matrix, as well as corresponding data indicating how the elements of the first matrix relate to the first matrix (e.g., where the elements are located). Compression can be achieved by providing some (but not all) of the elements of the first matrix.

[0029] Here, a specific embodiment will be described with reference to the drawings.

[0030] Figure 1 schematically shows a data processing device 100 according to several embodiments. The device includes a receiver circuit 110 that receives correspondence data indicating some elements of a first matrix, at least some elements of a second matrix, and the locations of the contributing elements of the first matrix within the first matrix. This information is passed from the receiver circuit to a determination circuit 120. The determination circuit uses the correspondence data to determine which contributing elements of the first matrix interact with the elements of the second matrix when performing a matrix operation. Whether two elements will interact depends on their locations in the first and second matrices and the operation being performed. For example, in matrix multiplication, when the first and second matrices are multiplied together, the elements in the first row and first column of the first matrix do not interact with the elements in the second row and second column of the second matrix. Since the correspondence data indicates the locations of the contributing elements of the first matrix within the first matrix, this information can be used to determine which pairs of interacting elements and their locations.

[0031] existenceInformation regarding the interactions is passed to the aggregation circuit 130. The aggregation circuit 130 is used to calculate aggregations across a large number of interacting pairs. For example, the aggregation circuit can calculate aggregations of pairs that interact with a given row / column when performing matrix multiplication. The aggregation circuit 130 includes a function circuit 140 that performs matrix operations. In particular, for a specific pair of elements (one from the first matrix and one from the second matrix) that the determination circuit 120 has determined to interact, the function circuit calculates the interaction. existence The determined operation is then executed. This causes the functional circuit to generate a partial result of a single interacting pair, which is then aggregated by the aggregation circuit 130.

[0032] In some embodiments, the receiver circuit 110 also receives further corresponding data so that the elements of the second matrix are within the second matrix. existence This indicates where to perform the action. This could be, for example, when only some of the elements of the second matrix are provided. Further correspondence data is passed to a determination circuit 120 that uses this further correspondence data to determine which pairs are interacting. In other embodiments, the further correspondence data can be ignored. This could be, for example, when the elements of the second matrix include all of the elements of the second matrix.

[0033] Figure 2 illustrates a method by which the locations of elements in the first and second matrices can be indicated within the first and second matrices, respectively, using corresponding data and optional additional corresponding data. In Figure 2, an example of a multiply-accumulate operation is shown in which the result of multiplying matrices a and b is added to the multiplier matrix c in order to provide the result matrix d. In this example, the multiplier matrix c is a 2x2 matrix, the first matrix a is an 8x2 matrix, the second matrix is ​​a 2x8 matrix, and the result matrix d is a 2x2 matrix. The nature of matrix multiplication means that the result of multiplying matrix a by matrix b is a 2x2 matrix. Naturally, it will be understood that other sizes of matrices can be used, and in fact, matrix multiplication can be performed without multiplication using the same process.

[0034] In this embodiment, the elements provided by the first matrix a are non-zero elements. Similarly, the elements provided by the second matrix b are non-zero values. For the first matrix a, corresponding data is provided for each row, and each non-zero element provides an index of the location within that row. For example, in the first row of matrix a, non-zero elements exist at indices 0, 5, and 7. In the second row, non-zero values ​​exist at indices 1, 4, and 6. Therefore, the corresponding data for the first row of the first matrix a represents the values ​​0, 5, and 7, and the corresponding data for the second row represents the values ​​1, 4, and 6. For the second matrix b, further corresponding data is provided for each column. In this embodiment, the first column of matrix b contains non-zero values ​​at indices 0, 3, and 7, and therefore, the further corresponding data for the first column includes indices 0, 3, and 7. In the second column, non-zero values ​​are found at indices 0, 3, and 6. Therefore, further corresponding data would indicate that elements of the second matrix b are located at positions 0, 3, and 6.

[0035] Using the corresponding data and further corresponding data, the first and second matrices can be reconstructed from the elements provided by the first matrix and the elements provided by the second matrix, respectively.

[0036] Figure 3 illustrates how matrices a and b can be compressed using corresponding data to form compressed matrices a' and b', respectively. In this case, the compressed matrices consist of the non-zero elements of the original matrices a and b. Thus, the first matrix is ​​reduced from 8×2 to 3×2, and the second matrix b is reduced from 2×8 to 2×3. Consequently, the memory space required to store or represent the matrices is reduced compared to storing or representing the matrices in their complete form, even though the corresponding data (and optionally further corresponding data) is also stored. In this embodiment, the first matrix a and the second matrix b are compressed to corresponding sizes suitable for matrix multiplication operations. However, for other matrix operations, the compressed matrices a' and b' can be compressed so that their sizes do not correspond, depending on the operation to be performed.

[0037] Figure 3 also shows how the corresponding data and further corresponding data are represented. In particular, for the first matrix a, the corresponding data is provided for each row of the compressed matrix a', while for the second matrix b, the corresponding data is provided for each column of the compressed matrix b'.

[0038] Figure 4 shows an embodiment of a method in which a compressed matrix can be stored using vector registers 410, 420, and 430. In this embodiment, two vector registers VR a’ 410 and VR b’ 420 is used to store the compressed matrices a' and b', and the third vector register VR c 430 is used to represent the multiplication matrix c. Each of the vector registers 410, 420, and 430 has a width of b bits.

[0039] For the first compressed matrix a', each element of the second row is stored, followed by the corresponding data for that second row. This is followed by the elements of the first row, and then the corresponding data for that first row. Each element is given a width of c bits. Therefore, the maximum number of elements that can be stored in one of the vector registers 410, 420, or 430 without corresponding data is b / c. In this embodiment, x bits are used to store each corresponding data for each element. Since each row of the first compressed matrix a' has three elements, 3x bits of corresponding data are provided for each row, i.e., a total of 6x bits. Therefore, the maximum number of elements that can be stored in each of the vector registers 410, 420, or 430 is equal to (b-6x) / c. In other words, if x bits of corresponding data are provided for each element, the maximum number of elements that can be stored in the vector registers 410, 420, or 430 is b / (c+x).

[0040] In this example, x is 5. It should be understood that this is more than sufficient to represent the element index. In fact, in this example, the original matrix has 8 elements per row (for matrix a) or column (for matrix b), so only log2(8) = 3 bits are needed per element.

[0041] In this embodiment, the first corresponding data 440 corresponding to elements 4, 5, and 6 is equal to 6<<10|4<<5|1. In other words, value 1 is obtained by concatenating it with value 4 and shifting it 5 times to the left, and then concatenating it with value 6 and shifting it 10 times to the left. This represents the fact that value 6 belongs to the first index, value 5 belongs to the fourth index, and value 4 belongs to the sixth index. This is shown in the second row of matrix a in Figure 2. The shifts are performed to prevent the indices from competing with each other in storage. Similarly, the second corresponding data 450 corresponding to elements 1, 2, and 5 is equal to 7<<10|5<<5|0. In other words, value 0 is obtained by concatenating it with value 5 and shifting it 5 times to the left, and then concatenating it with value 7 and shifting it 10 times to the left. This represents the fact that value 3 is at index 0. existence And the value 2 is at index 5. existence And the value 1 is at index 7. existence This represents the fact that... This is shown in the first row of matrix a in Figure 2. Note that since the corresponding data are grouped together row by row, it is possible to know the location of a given value based on the index that forms part of the corresponding data.

[0042] Similarly, the second vector register VR b’The data stored within contains two further corresponding data items, 460 and 470. The first corresponding data 460 corresponds to elements B, C, and D, and is equal to 6 << 10 | 3 << 5 | 0. In other words, value 0 is obtained by concatenating with value 3 and shifting left 5 times, and by concatenating with value 6 and shifting left 10 times. This represents the fact that value D is stored at index 0, value C is stored at index 3, and value B is stored at index 6. This is shown in the second column of matrix b in Figure 2. The second further corresponding data 470 corresponds to values ​​8, 9, and A, and is equal to 7 << 10 | 3 << 5 | 0. In other words, value 0 is obtained by concatenating with value 3 and shifting left 5 times, and by concatenating with value 7 and shifting left 10 times. This represents the fact that value A is stored at index 0. existence And the value 9 is at index 3. existence And the value 8 is at index 7. existence This demonstrates that, again, this is shown as the first column of matrix b in Figure 2. Note that since the further corresponding data are grouped together by column, it is possible to determine the location of a given value based on the index that forms part of the corresponding data.

[0043] Note that since the multiplication matrix is ​​not compressed in the same way as matrices a' and b', corresponding data is not required for the data elements of the multiplication matrix c. Therefore, the third vector register VR c 430 simply contains the four values ​​of the multiplication matrix c. Since there is no corresponding data and only four values ​​exist, more bits can be allocated to the elements of the multiplication matrix c. This is also appropriate because the size of the elements of the multiplication matrix can be much larger, especially if they represent the sum of many multiplications.

[0044] Figure 5 shows a circuit used to perform multiplication or multiply-accumulate operations between two compression matrices a' and b' and an accumulation matrix c'. This process involves performing a series of dot product operations. The circuit 500 is constructed from four units 530a, 530b, 530c, and 530d, each of which performs a dot product. Each of the units 530a, 530b, 530c, and 530d receives different combinations of elements from the first compression matrix a' and the second compression matrix b' according to the matrix multiplication operation. In addition, if a multiply-accumulate operation is performed, the accumulation elements are passed to each of the four units 530a, 530b, 530c, and 530d. For example, the dot product of the first column of the first compression matrix a' and the first row of the second compression matrix b' produces an element at (1,1) in the resulting matrix. If a multiply-accumulate operation is to be performed, the element at (1,1) of the multiplication matrix must also be added to the result of the dot product to generate the result matrix. Similarly, the dot product of the first column of the first compression matrix a' and the second row of the second compression matrix b' generates an element at (2,1) (second column, first row) of the result matrix. If a multiply-accumulate operation is performed, the element at (2,1) of the multiplication matrix is ​​also added to generate the result matrix. Regardless of whether multiplication or multiply-accumulate operations are performed, the output of each of the four units 530a, 530b, 530c, and 530d is one of the elements of the result matrix d.

[0045] Figure 5 also shows the structure of one of the units 530a. The structures of the other units 530b, 530c, and 530d are similar except for the elements of the received compression matrices a' and b', and the elements of the integration matrix c.

[0046] In the illustrated embodiment, the elements of the first row of the compression matrix a' and the elements of the first column of the compression matrix b' are received. Unit 530a includes three decision circuits 520aa, 520ab, and 520ac, so that the circuit can support a compression matrix b' having up to three elements per column (for the first compression matrix b'). Each of the decision circuits 520aa, 520ab, and 520ac receives each of the three elements of the compression matrix b'. Furthermore, each of the decision circuits receives the value z(0) as an additional input. In this embodiment, each of the decision circuits 520aa, 520ab, and 520ac takes the form of a multiplexer. The selection signal S of the multiplexer consists of three components s2, s1, and s0. Each of these elements is a single bit, and the overall selection signal S is one-hot.

[0047] As shown in Figure 5, in the case of the first determination circuit 520aa, s0=(a0id==b0id), s1=(a0id==b1id), and s2=(a0id==b2id). In other words, The first determination circuit 520aa has a first selection bit s0 equal to 1 if the index of the first contributing element of the first compression matrix a' is equal to the index of the first contributing element of the second compression matrix b'. Otherwise, the selection bit s0 is equal to 0.

[0048] The first determination circuit 520aa has a second selection bit s1 equal to 1 if the index of the first contributing element of the first compression matrix a' is equal to the index of the second contributing element of the second compression matrix b'. Otherwise, the selection bit s1 is equal to 0.

[0049] The first determination circuit 520aa has a third selection bit s2 equal to 1 if the index of the first contributing element of the first compression matrix a' is equal to the index of the third contributing element of the second compression matrix b'. Otherwise, the selection bit s2 is equal to 0.

[0050] This index is determined according to the corresponding data and further corresponding data. Therefore, the selection signal indicates to the first determination circuit 520aa whether the index of any of the first providing elements of the first compression matrix a' is equal to the index of any of the three elements of the second compression matrix b'. As can be understood, in the multiplication operation between the first matrix and the second matrix, the first element of the first matrix interacts with (is multiplied by) the second element of the second matrix if its index matches, that is, if the column number of the first element is the same as the row number of the second element. Therefore, the selection signal indicates whether any such match existence This indicates where to do it, allowing the system to select matching elements from the second matrix.

[0051] Therefore, the output of the determination circuit 520aa is equal to 0 when there is no interaction, or when there is an interaction between the first element of the first compression matrix a' and the elements of the second compression matrix. existence In this case, it is equal to the value of one of the elements of the second compression matrix b'. This value is then passed to the first functional circuit 540aa in the form of a first multiplier circuit, which also receives the first element of the first compression matrix a' as input. The first multiplier circuit 520aa then performs the multiplication. Its output is passed to the integrator circuit 550a, which is in the form of an adder, which performs the addition portion of the dot product operation.

[0052] The second determination circuit 520ab determines if any interaction occurs between the three contributing elements of the second compression matrix b' and the second contributing element of the first compression matrix a'. existence It is provided to determine whether or not to do so. Again, the selection signal S consists of three bits s0, s1, and s2. However, in the case of the second decision circuit 520ab, s0=(a1id==b0id), s1=(a1id==b1id), and s2=(a1id==b2id). The output of the second decision circuit 520ab is one of the elements of the second compression matrix b', which is an element that is considered to interact with the second element of the first compression matrix a', or does not interact existenceIf not, z is 0. The result is passed to the second multiplier circuit 540ab. The second multiplier circuit multiplies the value passed along with the second element of the first compression matrix a'. The result of the multiplication is then passed to the integrator circuit 550a, which is in the form of an adder that performs the addition part of the dot product operation.

[0053] Similarly, the third decision circuit 520ac determines whether any of the three elements of the second compression matrix b' interacts with the third element of the first compression matrix a'. Here, the selection signal bits are s0=(a2id==b0id), s1=(a2id==b1id), and s2=(a2id==b2id). Therefore, the output of the second decision circuit 520ac is either zero, or one of the three contributing elements of the second compression matrix b' that is considered to interact with the second element of the first compression matrix a'. The output is passed to the third multiplier circuit 540ac, where the result is multiplied by the third contributing value of the first compression matrix a', and this result is passed to the integrator 550a.

[0054] The integrator circuit 550a adds together the results of the multiplier circuits 540aa, 540ab, and 540ac. The integrator also adds the provided elements from the integrator matrix c (if provided). This result produces one of the values ​​of the result matrix d, as shown in Figure 5.

[0055] The other units 530b, 530c, and 530d behave in a similar manner. Each of the decision circuits 520 compares the indices of three contributing elements of the second compression matrix b' with the index of one of the three contributing elements of the first compression matrix a'. The three decision circuits within a single unit compare each of the indices of the contributing elements of the second compression matrix b' with each of the indices of the contributing elements of the first compression matrix a' together.

[0056] Therefore, by using corresponding data and further corresponding data in the form of an index, the decision circuit determines the interaction between the elements of the first matrix a' and the elements of the second matrix b'. existenceIt can be seen that it is possible to determine where to do it. existence If it is deemed that the elements are equal, an operation (e.g., multiplication) is performed between them. Otherwise, the multiplication performed is multiplication by zero.

[0057] In this embodiment, only three elements each of the first compression matrix a' and the second compression matrix b' are considered. However, it should be understood that this can be scaled by providing a further multiplication circuit 540, thereby providing further input to the decision circuit 520. Furthermore, the output result provided is a 2x2 matrix, and therefore 4 (2x2) units 530 are provided. However, if the output matrix d is larger, a larger array of units may be provided. For example, if the output matrix is ​​a 3x3 matrix, then 9 such units 530 will be provided. Of course, in such a case, the size of the compression matrix is ​​likely to be larger as well, and therefore the size of the individual units 530 is also likely to be expanded as described above.

[0058] As a result of the device 500, it will be understood that it is not necessary to perform the complete set of operations that may be required on uncompressed matrices a and b. For example, considering the number of elements in uncompressed matrices a and b, a total of 32 multiplications would be required, but due to compression, in this embodiment only 12 multiplications are required. Furthermore, a large number of multiplications can be performed in parallel across multiple units 530. existence As a result of both of these reasons, this matrix operation can be performed more quickly than in the case of uncompressed matrices. Furthermore, it should be understood that compressed matrices a' and b' are significantly smaller than uncompressed matrices a and b, and therefore can be stored using less memory space.

[0059] In this embodiment, both matrices are compressed. However, it should be understood that one of the two matrices can be in an uncompressed form. When this occurs, since the required index can be directly determined from the matrix itself, no corresponding data or further corresponding data is required.

[0060] In other embodiments, operations other than multiplication (or sum of products) can be performed. For example, when adding two matrices together, the addition of one or two zero operands is essentially a redundant operation. Therefore, if two values "interact" with each other in an addition, the actual addition operation can be ignored by simply returning the non-zero (or zero if all operands are zero) operands.

[0061] FIG. 6 considers an embodiment in which "tiling" can be used to perform an operation where the first matrix and the second matrix have non-matching sizes. In particular, matrix a can be compressed by being a sparse matrix with many non-zero entries. Thus, it can be compressed from a 2×8 matrix a to a 2×3 matrix a' as shown previously. In contrast, matrix b is dense, has no non-zero entries, and thus remains an 8×1 matrix. Normally, the matrix multiplication between these matrices produces a 2×1 matrix. However, by tiling, the 8×1 matrix can be treated as a 4×2 matrix with two columns 610, 620. Thus, this output is a 2×2 matrix. As a result, elements d 00 and d 10 are generated using the first half (bits 0 to 3) 610 of matrix b. On the other hand, elements d 01 and d 11 are generated using the second half (bits 4 to 7) 620 of the second matrix b.

[0062] FIG. 7 shows how the storage of the embodiment of FIG. 6 is achieved. In particular, the storage for the first compressed matrix a' and the accumulation matrix c occurs in vector registers 710, 730 in the same way as shown with respect to FIG. 4. In the case of the second matrix b, (compression is existenceSince no further corresponding data is needed (and therefore the space is freed for each element of the second matrix b that needs to be stored).

[0063] Figure 8 shows a circuit 800 that may be used to perform a matrix operation in which one of two matrices has a non-conforming size. In this embodiment, matrix b remains in its uncompressed form and is therefore technically an 8x1 matrix, and the circuit in Figure 8 instead processes it as a 4x2 matrix using tiling.

[0064] Circuit 800 operates similarly to circuit 500 shown in Figure 5. One important difference is that the uncompressed matrix b is processed to have four rows, so each of the four units 830a, 830b, 830c, and 830d receives four inputs in relation to the second uncompressed matrix b.

[0065] Each of the decision circuits 820aa, 820ab, and 820ac within the four units 830a, 830b, 830c, and 830d receives four inputs along with a constant 0(z). Furthermore, the selection signal S is composed of four components s0, s1, s2, and s3. For example, in the first unit 820aa, s0=(a0id==b0id), s1=(a0id==b1id), s2=(a0id==b2id), and s3=(a0id==b3id). Again, the components of the selection signal S interact with each of the given elements of the first compression matrix a' and the presented elements of the second matrix b. existence This indicates whether or not compression is performed. In this case, since no compression is performed, further corresponding data is not available for the elements of the second matrix b. However, since the location of each element of the second matrix b is known, further corresponding data is not needed.

[0066] The circuit shown in Figure 8 can also be used with a pair of compressed matrices, such as the one for b'. This can be achieved simply by using the index provided as part of the (further) corresponding data. Similarly, the circuit in Figure 8 can be used for multiplication of matrices of non-corresponding size (in this case, a compressed 2x3 matrix is ​​multiplied by an 8x1 matrix and treated as a 4x2 matrix), The same circuit can also be used for smaller matrices by ignoring any extra input signals to the decision circuit 820 and setting the corresponding selection bits of the selection signal S for those inputs to 0 so that they are not selected. For example, by ignoring the fourth input to the decision circuit 820 and setting the selection value by s3 to 0, the same circuit can be used to multiply a 2x3 matrix by another 2x3 matrix (as shown in Figure 5). If the first matrix is ​​smaller (for example, if the compressed matrix a' is 2x2), the extra decision unit 820 (and its associated multiplication unit 840) are ignored. For example, if the compressed matrix a' is a 2x2 matrix, the third decision circuit 820ac and its associated multiplication unit 840ac are simply ignored.

[0067] Figure 9 shows an apparatus 900 according to several embodiments. The apparatus includes a number of components 930 in the form of a pipeline. Within the pipeline 930, a fetch circuit 910 receives an instruction (MatrixOp) for performing a matrix operation. The instruction identifies an element of a first matrix (elemA), an element of a second matrix (elemB), corresponding data for the element of the first matrix (correspondenceA), optional further corresponding data for the element of the second matrix (correspondenceB), and an optional element of a third matrix integrator (elemC). In each case, a reference to the matrix element may be provided in the form of a register(s) or a location in memory where the element can be found. The fetch circuit 910 passes the fetch instruction to a decode circuit 920. The decode circuit decodes the instruction, which provides one or more control signals. These control signals are then passed to a matrix operation circuit 500, such as the one shown with respect to Figure 5 or Figure 8.

[0068] Figure 10 shows a data processing method 1000 according to several embodiments. In step 1010, the elements of the first matrix are the elements of the second matrix, and the elements of the first matrix are within the first matrix existence It is received along with corresponding data indicating the location. In step 1020, according to matrix operations, it is determined which elements of the first matrix and the second matrix interact with each other. In step 1030, the aggregation is performed on the elements of the first matrix and the second matrix that are deemed to interact with each other by the determination step 1020. against This is done by performing matrix operations.

[0069] In this application, the term "configured to..." is used to mean that an element of the device has a configuration that enables it to perform a defined operation. In this context, "configuration" means the configuration or interconnection of hardware or software. For example, the device may have dedicated hardware to provide the defined operation, or a processor or other processing device may be programmed to perform the function. "Configured" does not mean that the device element needs to be modified in any way to provide the defined operation.

[0070] While exemplary embodiments of the present invention have been described in detail herein with reference to the accompanying drawings, it will be understood that the present invention is not limited to those exact embodiments, and that various changes, additions, and modifications can be made by those skilled in the art without departing from the scope and spirit of the invention as defined by the appended claims. For example, various combinations of the features of the dependent claims and the features of the independent claims can be made without departing from the scope of the present invention.

Claims

1. The system includes a decoding circuit that receives one or more instructions, including a matrix operation instruction, wherein the matrix operation instruction refers to a non-zero element of a first matrix, a non-zero element of a second matrix, and corresponding data indicating the location of the element of the first matrix within the first matrix, and the elements of the first matrix and the corresponding data are stored in a vector register. Upon receiving the matrix operation instruction, the decoding circuit is configured to generate one or more control signals to cause one or more execution units to perform matrix operations on the first matrix and the second matrix. The one or more execution units described above are: A receiver circuit that receives the corresponding data indicating the location in which the elements of the first matrix, the elements of the second matrix, and the elements of the first matrix are located within the first matrix, A determination circuit that uses the aforementioned corresponding data to determine whether a given element of the second matrix exists in row i of the second matrix for a given element of the first matrix in column i of the first matrix, An aggregation circuit for calculating the aggregation between a given row of the first matrix and a given column of the second matrix, A data processing device comprising: an aggregation circuit, which includes a functional circuit that performs functions on the given elements of the first matrix and the given elements of the second matrix in accordance with the determination, to generate partial results.

2. The first matrix is ​​generated by removing at least some zero entries from the first matrix in its uncompressed form, and removing at least some columns or rows where each entry is zero. The data processing device according to claim 1.

3. The corresponding data indicates the row number or column number for each of the elements of the first matrix, The row number and column number of the given element correspond to the row and column in which the given element is located within the first matrix. The data processing device according to claim 1.

4. The receiver circuit is configured to receive further corresponding data and to indicate the location of the element of the second matrix within the second matrix. The determination circuit is adapted to perform the determination using the additional corresponding data in addition to the corresponding data. The data processing device according to claim 1.

5. The further corresponding data indicates the row number or column number for each of the elements of the second matrix, The row number and column number for a given element of the aforementioned element correspond to the row and column in which the given element is located within the second matrix. The data processing device according to claim 4.

6. The second matrix is ​​generated by removing at least some of the zero entries from the uncompressed second matrix, and removing at least some of the columns or rows where each entry is zero. The data processing device according to claim 4.

7. The matrix operation includes a multiplication operation on the first matrix and the second matrix, The aggregation circuit includes a dot product circuit that calculates the dot product of the given row and the given column as the aggregation, The aforementioned functional circuit includes a multiplication circuit that performs multiplication as its function, The dot product circuit comprises an adder circuit that adds up each of the partial results generated for the given row and the given column. The data processing device according to claim 1.

8. The receiver circuit is adapted to receive the elements of the third matrix, The operation includes an addition operation of the third matrix using the results of the multiplication operation of the first matrix and the second matrix, The addition circuit is adapted to add each partial result generated for a given row and a given column together with one of the elements of the third matrix. The data processing device according to claim 7.

9. The multiplication circuit is adapted to output the value "0" in response to the determination that none of the elements of the second matrix exist in row i of the second matrix. The data processing device according to claim 7.

10. The determination circuit is adapted to provide the value "0" as input to the multiplication circuit when it determines that the given element of the second matrix exists at a location other than row j and column i of the second matrix. The data processing device according to claim 9.

11. The system comprises a memory circuit for storing the elements of the second matrix, The memory circuit includes the further corresponding data, The data processing device according to claim 5.

12. The memory circuit is the vector register or another vector register used to store the elements of the first matrix. The data processing device according to claim 11.

13. The width of the memory circuit is b bits, Each of the elements of the first matrix has a width of c bits, The number of elements in the first matrix is ​​at most b / c. The data processing device according to claim 1.

14. The width of the corresponding data is x bits for each of the elements of the first matrix, The number of elements in the first matrix is ​​at most b / (c+x), The data processing device according to claim 13.

15. The width of the memory circuit is b bits, Each of the elements of the second matrix has a width of c bits, The number of elements in the second matrix is ​​at most b / c. The data processing device according to claim 11.

16. The width of the further corresponding data is x bits for each of the elements of the second matrix, The number of elements in the second matrix is ​​at most b / (c+x), The data processing device according to claim 15.

17. A data processing method that performs operations on a first matrix and a second matrix, Receiving a matrix operation instruction that references a non-zero element of the first matrix, a non-zero element of the second matrix, and corresponding data indicating the location of the element of the first matrix within the first matrix, In response to receiving a matrix operation instruction, the decoding circuit includes generating one or more control signals to cause one or more execution units to perform matrix operations on a first matrix and a second matrix, The aforementioned data processing method further includes: Receiving the corresponding data indicating the location where the elements of the first matrix, the elements of the second matrix, and the elements of the first matrix are located within the first matrix, Using the aforementioned corresponding data, a determination is made for a given element of the first matrix in column i of the first matrix, to determine whether a given element of the second matrix exists in row i of the second matrix. The calculation involves calculating an aggregation between a given row of the first matrix and a given column of the second matrix, wherein the calculation includes, in accordance with the determination, performing functions on the given elements of the first matrix and the given elements of the second matrix to generate a partial result. A data processing method that includes this.

Images