Matrix calculation method, chip and related equipment

Abstract

The invention provides a matrix calculation method, a chip and related equipment. The chip comprises a first buffer, a second buffer, a scheduling module and a processing element, the first buffer is used for caching a first vector, and the second buffer is used for caching a second vector; the scheduling module generates a gating signal according to the bitmap of the first vector, and the gating signal can enable the processing element to obtain a group of non-zero elements in the first vector from the first buffer and enable the processing element to obtain a group of elements in the second vector from the second buffer; and then operation of the first vector and the second vector is carried out according to the set of non-zero elements in the first vector and the set of elements in the second vector, and the bitmap of the first vector is used for indicating the non-zero elements in the first vector. When the chip provided by the invention is used for carrying out the operation of two vectors, the element with the value of 0 in one vector does not participate in the calculation, so that the calculation amount can be reduced, and the calculation efficiency can be improved.

Description

[0001] This application is a divisional application. The application number of the original application is 202210460849.1, and the original application date is April 28, 2022. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a matrix calculation method, chip and related equipment. Background Art

[0003] In current fields such as artificial intelligence (AI) and high performance computing (HPC), a large number of matrix multiplication operations are involved, such as AI model training and AI reasoning. In order to speed up the calculation and improve the calculation efficiency, the matrix is ​​usually sparsed through pruning technology, and then the sparse matrix is ​​used for calculation, thereby reducing the amount of calculation in the calculation process. For example, after the AI ​​training is completed, the trained weight matrix is ​​structured pruned to complete the sparsification of the weight matrix, and the weight matrix after structured pruning is used for reasoning during AI reasoning. However, the above method only supports pruning a certain matrix and using the pruned matrix for calculation, and does not support scenarios where the sparsity of the matrix changes dynamically during the calculation process. Summary of the invention

[0004] The present application provides a matrix calculation method, chip and related equipment, which can support the calculation of matrices with dynamically changing sparsity, reduce the amount of calculation in the calculation, improve the calculation efficiency, and do not reduce the calculation accuracy.

[0005] In a first aspect, the present application provides a chip, comprising: a first cache, a second cache, a first scheduling module and a first processing element, wherein the first cache is used to cache a first vector; the second cache is used to cache a second vector; the first scheduling module is used to generate a first selection signal according to a bitmap of the first vector, and the first selection signal enables the first processing element to obtain a first group of non-zero elements in the first vector from the first cache, and enables the first processing element to obtain a second group of elements in the second vector from the second cache; the first processing element is used to implement operations on the first vector and the second vector according to the first group of non-zero elements in the first vector and the second group of elements in the second vector, wherein the bitmap of the first vector indicates the non-zero elements in the first vector.

[0006] Each bit in the bitmap of the first vector corresponds to an element in the first vector, and the value of each bit indicates whether the corresponding element in the first vector is 0. For example, if the value of a bit is 0, it means that the value of the corresponding element in the first vector is 0. If the value of a bit is 1, it means that the value of the corresponding element in the first vector is not 0. The first scheduling module can determine which elements in the first vector are non-zero elements according to the bitmap of the first vector, so that the first processing element can only obtain the non-zero elements in the first vector, and then obtain the elements at the corresponding position from the second vector, and perform the operation between the first vector and the second vector, thereby reducing the amount of calculation and improving the calculation efficiency. In addition, in the process of performing the dot product of the vectors, the multiplication value of the element with the value of 0 in one vector and the non-zero element in another vector is 0, which will not affect the result of the dot product of the two vectors. Therefore, only the non-zero elements in the first vector are obtained to perform the operation of the first vector and the second vector, which will not reduce the calculation accuracy.

[0007] In a possible implementation, the chip further includes a first multiplexer and a second multiplexer; the first multiplexer is used to obtain a first group of non-zero elements in the first vector from a first buffer according to a first selection signal, and input them into a first processing element; the second multiplexer is used to obtain a second group of elements in the second vector from a second buffer according to the first selection signal, and input them into the first processing element.

[0008] In a possible implementation, the first multiplexer and the second multiplexer each include K multiplexers; the first buffer and the second buffer each include W rows and K columns of data units, each data unit is used to cache an element in a vector or a matrix; each multiplexer in the first multiplexer is connected to multiple data units in the first buffer, and each data unit is connected to at least one multiplexer; the connection relationship between the i-th multiplexer in the second multiplexer and the data unit in the second buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

[0009] In the operation of the first vector and the second vector, the rth element of the first vector and the rth element of the second vector need to be multiplied. When storing each element of the first vector in the first buffer and storing each element of the second vector in the second buffer, the relative position of the rth element of the first vector stored in the first buffer is the same as the relative position of the rth element of the second vector stored in the second buffer. For example, the rth element of the first vector is stored in the rth data unit of the first row of the first buffer, and the rth element of the second vector is stored in the rth data unit of the first row of the second buffer. At the same time, the i-th multiplexer in the first multiplexer connects multiple data units in the first buffer. The relative positions of these multiple data units in the first buffer are the same as the relative positions of multiple data units in the second buffer connected to the i-th multiplexer in the second multiplexer in the second buffer. For example, if the i-th multiplexer in the first multiplexer connects the second data unit of the first row and the fifth data unit of the second row in the first buffer, then the i-th multiplexer in the second multiplexer connects the second data unit of the first row and the fifth data unit of the second row in the second buffer. In this way, the i-th multiplexer of the first multiplexer and the i-th multiplexer of the second multiplexer can read the data in the data units with the same relative position in the first buffer and the second buffer according to the same selection signal, so that the first multiplexer and the second multiplexer can read the elements that need to be multiplied in the first vector and the second vector according to the same selection signal.

[0010] In a possible implementation, the first scheduling module is specifically used to: determine, according to the bitmap of the first vector, that in the data unit connected to the j-th multiplexer of the first multiplexer, the element stored in the k-th data unit is a non-zero element, then the first scheduling module generates a selection signal for the j-th multiplexer of the first multiplexer, and sends the selection signal of the j-th multiplexer of the first multiplexer to the j-th multiplexer of the first multiplexer and the j-th multiplexer of the second multiplexer, and the first selection signal includes the selection signal of the j-th multiplexer of the first multiplexer.

[0011] The first multiplexer includes K multiplexers, and the first scheduling module generates a strobe signal corresponding to each of the K multiplexers in one cycle, that is, the first strobe signal includes the strobe signal corresponding to each of the K multiplexers, so that each multiplexer in the first multiplexer and the second multiplexer reads one data from the connected data unit according to the received strobe signal;

[0012] The multiple data units connected to the above-mentioned multiplexer have different priorities. Before generating a selection signal for a multiplexer, the first scheduling module first determines whether the element of the first vector stored in the data unit with the highest priority among the data units connected to the multiplexer is 0 according to the bitmap of the first vector. If the element stored in the data unit with the highest priority is not 0, a selection signal corresponding to the data unit with the first priority is generated; if the element stored in the data unit with the highest priority is 0, then determine whether the element stored in the data unit with the second priority is 0 according to the bitmap of the first vector. If the element stored in the data unit with the second priority is not 0, a selection signal corresponding to the data unit with the second priority is generated. If the element stored in the data unit with the second priority is 0, then determine whether the element stored in the data unit with the third priority is 0 according to the bitmap of the first vector, and so on, until the selection signal of the multiplexer is generated. By setting the priority, the elements in the multiple data units connected to each multiplexer can be read in order.

[0013] It should be noted that after the first scheduling module determines that the element stored in a data unit is not 0 and generates a selection signal corresponding to the data unit, it needs to set the bit position corresponding to the element stored in the data unit in the bitmap to 0 to prevent the element in the data unit from being read again, resulting in calculation errors.

[0014] In a possible implementation, the first multiplexer is specifically used to: according to the selection signal of the j-th multiplexer of the first multiplexer, obtain the first element in the k-th data unit in the data unit connected to the j-th multiplexer of the first multiplexer through the j-th multiplexer of the first multiplexer, and input the first element to the first processing element, and the first element is one of the above-mentioned first group of non-zero elements; the second multiplexer is specifically used to: according to the selection signal of the j-th multiplexer of the first multiplexer, obtain the second element in the k-th data unit in the data unit connected to the j-th multiplexer of the second multiplexer through the j-th multiplexer of the second multiplexer, and input the second element to the first processing element, and the second element is one of the above-mentioned second group of elements.

[0015] In one cycle, the first scheduling module sequentially generates the selection signals corresponding to the K multiplexers in the first multiplexer, so that the first multiplexer and the second multiplexer respectively take out K elements from the connected buffers, and then realize the dot product of the first vector and the second vector based on the K pairs of data taken out. It should be understood that the first multiplexer may not be able to take out all the non-zero elements in the first buffer in one cycle, but the first multiplexer can take out K non-zero elements at a time, and the first buffer stores W rows and K columns of elements. Therefore, after at most W cycles, all the non-zero elements in the first buffer can be taken out to complete the operation between the first vector and the second vector.

[0016] In one possible implementation, after the first processing element completes the operation of the first vector and the second vector, the first scheduling module generates an erase signal, which is used to instruct the first cache and the second cache to erase the currently cached data to cache the data required for the next calculation.

[0017] In a possible implementation, the first vector may be part of a vector, and the second vector may be part of another vector; or the first vector is a row vector, and the second vector is a column in a matrix; or the first vector belongs to any row in the first matrix, and the second vector belongs to any column in the second matrix. Then, the chip can realize vector-to-vector operations, vector-to-matrix operations, or matrix-to-matrix operations through multiple calculation processes.

[0018] In a possible implementation, the chip further includes a third buffer and a second processing element, the third buffer being used to cache a third vector, the third vector belonging to a column of the second matrix other than the column where the second vector is located; the first selection signal is also used to enable the second processing element to obtain a third group of elements in the third vector from the third buffer; the second processing element is used to perform operations on the first vector and the third vector based on the first group of non-zero elements and the third group of elements.

[0019] By adding a third buffer and a second processing element, the operation of the first vector and the third vector can be completed by the second processing element, and the operation of the first vector and the second vector can be completed by the first processing element, and the second vector and the third vector both belong to the second matrix, thereby enabling the chip to realize vector and matrix operations.

[0020] In a possible implementation, the chip further includes a third multiplexer, and the third multiplexer is used to obtain a third group of elements in the third vector from a third buffer according to the first selection signal, and input the third group of elements into the second processing element.

[0021] In one possible implementation, the third multiplexer includes K multiplexers, the third buffer includes W rows and K columns of data units, and each data unit is used to cache an element; the connection relationship between the i-th multiplexer in the third multiplexer and the data unit in the third buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

[0022] The third multiplexer is used to obtain the elements of the third vector from the third buffer and input them into the second processing element so that the second processing element realizes the operation of the first vector and the third vector. In the operation of the first vector and the third vector, the rth element of the first vector and the rth element of the third vector need to be multiplied. When storing each element of the first vector in the first buffer and storing each element of the third vector in the third buffer, the relative position of the rth element of the first vector stored in the first buffer is the same as the relative position of the rth element of the third vector stored in the third buffer. For example, the rth element of the first vector is stored in the rth data unit of the first row of the first buffer, and the rth element of the third vector is stored in the rth data unit of the first row of the third buffer. At the same time, the i-th multiplexer in the first multiplexer connects multiple data units in the first buffer. The relative positions of these multiple data units in the first buffer are the same as the relative positions of multiple data units in the third buffer connected to the i-th multiplexer in the third multiplexer in the third buffer. For example, the i-th multiplexer in the first multiplexer connects the second data unit of the first row and the fifth data unit of the second row in the first buffer, and the i-th multiplexer in the third multiplexer connects the second data unit of the first row and the fifth data unit of the second row in the third buffer. In this way, the i-th multiplexer of the first multiplexer and the i-th multiplexer of the third multiplexer can read the data in the data units with the same relative position in the first buffer and the third buffer according to the same selection signal, so that the first multiplexer and the third multiplexer can read the elements that need to be multiplied in the first vector and the third vector according to the same selection signal.

[0023] In a possible implementation, the third multiplexer is specifically used to: obtain, according to a selection signal of the jth multiplexer of the first multiplexer, through the jth multiplexer of the third multiplexer, the third element in the kth data unit in the data unit connected to the jth multiplexer of the third multiplexer, and input the third element into the second processing element, where the third element is one of the third group of elements.

[0024] In a possible implementation, the chip further includes a fourth buffer, a second scheduling module, and a third processing element; wherein the fourth buffer is used to cache a fourth vector, and the fourth vector belongs to a row of the first matrix other than the row where the first vector is located;

[0025] The second scheduling module is used to generate a second selection signal according to the bitmap of the fourth vector, the second selection signal is used to enable the third processing element to obtain a fourth group of non-zero elements in the fourth vector from the fourth buffer; and to enable the third processing element to obtain a fifth group of elements in the second vector from the second buffer; wherein the bitmap of the fourth vector indicates the non-zero elements in the fourth vector;

[0026] The third processing element is configured to implement an operation between the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

[0027] After adding the third buffer and the second processing element, the fourth buffer, the second scheduling module and the third processing element are added again; the third processing element can complete the operation of the fourth vector in the first matrix and the second vector in the second matrix, the second processing element can complete the operation of the first vector in the first matrix and the third vector in the second matrix, and the second element can complete the operation of the first vector in the first matrix and the second vector in the second matrix, so that the chip can realize matrix-matrix operations.

[0028] In a possible implementation, the above-mentioned chip also includes a fourth multiplexer and a fifth multiplexer. The fourth multiplexer is used to obtain a fourth group of non-zero elements in a fourth vector from a fourth buffer according to a second selection signal, and input them into a third processing element; the fifth multiplexer is used to obtain a fifth group of elements in the second vector from the second buffer according to the second selection signal, and input them into the third processing element.

[0029] In a possible implementation, the fourth multiplexer includes K multiplexers, and the fifth multiplexer includes K multiplexers; the fourth buffer includes W rows and K columns of data units, each data unit is used to cache one element;

[0030] Each multiplexer in the fourth multiplexer is connected to a plurality of data units in the fourth buffer, and each data unit is connected to at least one multiplexer; the connection relationship between the i-th multiplexer in the fourth multiplexer and the data units in the fourth buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data units in the first buffer; the connection relationship between the i-th multiplexer in the fifth multiplexer and the data units in the second buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data units in the first buffer.

[0031] It should be noted that the fifth multiplexer can also be connected to a fifth buffer, but since the fifth multiplexer is used to obtain elements of the second vector, and the second vector is cached in the second buffer, the fifth multiplexer can share the second multiplexer with the second multiplexer, thereby reducing the complexity of the chip, reducing the chip size and reducing the cost.

[0032] The fourth multiplexer is used to obtain the elements of the third vector from the fourth buffer and input them into the third processing element so that the third processing element realizes the operation of the fourth vector and the second vector. In the operation of the fourth vector and the second vector, the rth element of the fourth vector and the rth element of the second vector need to be multiplied. When storing each element of the fourth vector in the fourth buffer and storing each element of the second vector in the second buffer, the relative position of the fourth buffer where the rth element of the fourth vector is stored is the same as the relative position of the second vector where the rth element is stored in the second buffer. For example, the rth element of the second vector is stored in the rth data unit of the first row of the second buffer, and the rth element of the fourth vector is stored in the rth data unit of the first row of the fourth buffer. At the same time, the i-th multiplexer in the fourth multiplexer connects multiple data units in the fourth buffer. The relative positions of these multiple data units in the fourth buffer are the same as the relative positions of the multiple data units in the second buffer connected to the i-th multiplexer in the fifth multiplexer in the second buffer. In this way, the i-th multiplexer of the fourth multiplexer and the i-th multiplexer of the fifth multiplexer can read the data in the data units with the same relative position in the fourth buffer and the second buffer according to the same selection signal, so that the fourth multiplexer and the fifth multiplexer can read the elements that need to be multiplied in the fourth vector and the second vector according to the same selection signal.

[0033] In a possible implementation, the second scheduling module is specifically used to: determine, according to the bitmap of the fourth vector, that in the data unit connected to the j-th multiplexer of the fourth multiplexer, the element stored in the m-th data unit is a non-zero element, then the first scheduling module generates a selection signal for the j-th multiplexer, and sends the selection signal of the j-th multiplexer to the j-th multiplexer of the fourth multiplexer and the j-th multiplexer of the fifth multiplexer, and the second selection signal includes the selection signal of the j-th multiplexer of the fourth multiplexer.

[0034] In a possible implementation, the fourth multiplexer is specifically used to: according to the selection signal of the j-th multiplexer of the fourth multiplexer, obtain the third element in the m-th data unit in the data unit connected to the j-th multiplexer of the fourth multiplexer through the j-th multiplexer of the fourth multiplexer, and input the third element to the third processing element, and the third element is one of the fourth group of non-zero elements; the fifth multiplexer is specifically used to: according to the selection signal of the j-th multiplexer of the fourth multiplexer, obtain the fourth element in the m-th data unit in the data unit connected to the j-th multiplexer of the fifth multiplexer through the j-th multiplexer of the fifth multiplexer, and input the fourth element to the third processing element, and the fourth element is one of the fifth group of elements.

[0035] In one cycle, the second scheduling module sequentially generates the selection signals corresponding to the K multiplexers in the fourth multiplexer, so that the fourth multiplexer and the fifth multiplexer respectively take out K elements from the connected buffers, and then realize the dot product of the fourth vector and the second vector based on the K pairs of data taken out. It should be understood that the second multiplexer may not be able to take out all the non-zero elements in the fourth buffer in one cycle, but the fourth multiplexer can take out K non-zero elements at a time, and the fourth buffer stores W rows and K columns of elements. Therefore, after at most W cycles, all the non-zero elements in the fourth buffer can be taken out to complete the operation between the fourth vector and the second vector.

[0036] In a second aspect, the present application provides a matrix calculation method, which is applied to a chip, wherein the chip includes a first cache, a second cache, a first scheduling module and a first processing element, and the method includes: the chip caches a first vector through the first cache, and caches a second vector through the second cache; after the chip caches the first vector and the second vector, the first scheduling module of the chip generates a first selection signal according to a bitmap of the first vector, and the first selection signal enables the first processing element to obtain a first group of non-zero elements in the first vector from the first cache, and enables the first processing element to obtain a second group of elements in the second vector from the second cache, wherein the bitmap of the first vector indicates the non-zero elements in the first vector; then the chip implements operations between the first vector and the second vector according to the first group of non-zero elements and the second group of elements through the first processing element.

[0037] In a possible implementation, the above-mentioned chip also includes a first multiplexer and a second multiplexer; the first multiplexer of the chip obtains a first group of non-zero elements in the first vector from the first buffer according to the first selection signal, and inputs them into the first processing element; the second multiplexer of the chip obtains a second group of elements in the second vector from the second buffer according to the first selection signal, and inputs them into the first processing element.

[0038] In one possible implementation, the first multiplexer and the second multiplexer each include K multiplexers; the first buffer and the second buffer each include W rows and K columns of data units, each data unit is used to cache an element; each multiplexer in the first multiplexer is connected to multiple data units in the first buffer, and each data unit is connected to at least one multiplexer; the connection relationship between the i-th multiplexer in the second multiplexer and the data unit in the second buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

[0039] In a possible implementation, the first scheduling module of the above-mentioned chip generates a first selection signal according to the bitmap of the first vector, including: the chip determines according to the bitmap of the first vector that the element stored in the kth data unit in the data unit connected to the jth multiplexer of the first multiplexer is a non-zero element, the first scheduling module generates a selection signal for the jth multiplexer, and sends the selection signal of the jth multiplexer to the jth multiplexer of the first multiplexer and the jth multiplexer of the second multiplexer, wherein the first selection signal includes the selection signal of the jth multiplexer of the first multiplexer.

[0040] In a possible implementation, the first multiplexer of the above-mentioned chip obtains a first group of non-zero elements in a first vector from a first buffer according to a first selection signal, and inputs the first processing element, and the second multiplexer of the chip obtains a second group of elements in a second vector from a second buffer according to the first selection signal, and inputs the second group of elements into the first processing element, including: the first multiplexer of the chip obtains the first element in a kth data unit in the data unit connected to the jth multiplexer of the first multiplexer through the jth multiplexer of the first multiplexer according to the selection signal of the jth multiplexer of the first multiplexer, and inputs the first element into the first processing element; the second multiplexer of the chip obtains the second element in a kth data unit in the data unit connected to the jth multiplexer of the second multiplexer through the jth multiplexer of the second multiplexer according to the selection signal of the jth multiplexer of the first multiplexer, and inputs the second element into the first processing element, wherein the first element is one of the first group of non-zero elements, and the second element is one of the second group of elements.

[0041] In a possible implementation, the first vector belongs to any row in the first matrix, and the second vector belongs to any column in the second matrix. That is, the first vector may be part or all of the elements in any row in one matrix, and the second vector may be part or all of the elements in any column in another matrix.

[0042] In a possible implementation, the chip further includes a third buffer and a second processing element; the method further includes: the chip caches a third vector through the third buffer, the third vector belongs to a column in the second matrix other than the column where the second vector is located, and the first selection signal can also enable the second processing element to obtain a third group of elements in the third vector from the third buffer; the second processing element of the chip implements operations on the first vector and the third vector based on the first group of non-zero elements and the third group of elements.

[0043] In a possible implementation, the chip further includes a third multiplexer, and the method further includes: the third multiplexer of the chip obtains a third group of elements in the third vector from a third buffer according to the first selection signal, and inputs the third group of elements into the second processing element.

[0044] In one possible implementation, the third multiplexer includes K multiplexers, the third buffer includes W rows and K columns of data units, each data unit is used to cache an element; the connection relationship between the i-th multiplexer in the third multiplexer and the data unit in the third buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

[0045] In a possible implementation, the third multiplexer of the chip obtains the third group of elements in the third vector from the third buffer according to the first selection signal and inputs the third group of elements into the second processing element, including:

[0046] The third multiplexer of the above-mentioned chip obtains the third element in the kth data unit in the data unit connected to the jth multiplexer of the third multiplexer through the jth multiplexer of the third multiplexer according to the selection signal of the jth multiplexer of the first multiplexer, and inputs the third element to the second processing element, wherein the third element is one of the above-mentioned third group of elements.

[0047] In a possible implementation, the chip further includes a fourth buffer, a second scheduling module and a third processing element; the method further includes: the chip caches a fourth vector through the fourth buffer, and the fourth vector belongs to a row in the first matrix other than the row where the first vector is located; the second scheduling module of the chip generates a second selection signal according to the bitmap of the fourth vector, and the second selection signal is used to enable the third processing element to obtain a fourth group of non-zero elements in the fourth vector from the fourth buffer; the third processing element obtains a fifth group of elements in the second vector from the second buffer, wherein the bitmap of the fourth vector indicates the non-zero elements in the fourth vector; the third processing element of the chip implements operations on the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

[0048] In a possible implementation, the above-mentioned chip also includes a fourth multiplexer and a fifth multiplexer, and the method also includes: the fourth multiplexer of the chip obtains the fourth group of non-zero elements in the fourth vector from the fourth buffer according to the above-mentioned second selection signal, and inputs them into the third processing element; the fifth multiplexer of the chip obtains the fifth group of elements in the second vector from the second buffer according to the second selection signal, and inputs them into the third processing element.

[0049] In a possible implementation, the fourth multiplexer and the fifth multiplexer each include K multiplexers; the fourth buffer includes W rows and K columns of data units, each data unit is used to cache one element;

[0050] Each multiplexer in the fourth multiplexer is connected to a plurality of data units in the fourth buffer, and each data unit is connected to at least one multiplexer; the connection relationship between the i-th multiplexer in the fourth multiplexer and the data units in the fourth buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data units in the first buffer; the connection relationship between the i-th multiplexer in the fifth multiplexer and the data units in the fifth buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data units in the first buffer.

[0051] In a possible implementation, the second scheduling module of the above-mentioned chip generates a second selection signal according to the bitmap of the fourth vector, including: determining according to the bitmap of the fourth vector that the element stored in the m-th data unit in the data unit connected to the j-th multiplexer of the fourth multiplexer is a non-zero element, then the second scheduling module generates the selection signal of the j-th multiplexer, and sends the selection signal of the j-th multiplexer to the j-th multiplexer of the fourth multiplexer and the j-th multiplexer of the fifth multiplexer, and the above-mentioned second selection signal includes the selection signal of the j-th multiplexer of the fourth multiplexer.

[0052] In a possible implementation, the fourth multiplexer of the chip obtains the fourth group of non-zero elements in the fourth vector from the fourth buffer according to the second selection signal, and inputs them into the third processing element; the fifth multiplexer of the chip obtains the fifth group of elements in the second vector from the second buffer according to the second selection signal, and inputs them into the third processing element, including: the fourth multiplexer of the chip obtains the fourth element in the mth data unit in the data unit connected to the jth multiplexer of the fourth multiplexer through the jth multiplexer of the fourth multiplexer according to the selection signal of the jth multiplexer of the fourth multiplexer, and inputs the fourth element to the third processing element; the fifth multiplexer of the chip obtains the fifth element in the mth data unit in the data unit connected to the jth multiplexer of the fifth multiplexer through the jth multiplexer of the fifth multiplexer according to the selection signal of the jth multiplexer of the fourth multiplexer, and inputs the fifth element to the third processing element, wherein the fourth element is one of the fourth group of non-zero elements, and the fifth element is one of the fifth group of elements.

[0053] In a third aspect, the present application provides a matrix computing device, comprising a first scheduling unit and a first processing unit, wherein the first scheduling unit is used to generate a first selection signal according to a bitmap of a first vector, the first selection signal enabling the first processing unit to obtain a first group of non-zero elements in the first vector from a first buffer, enabling the first processing unit to obtain a second group of elements in the second vector from a second buffer, and the first processing unit is used to implement operations on the first vector and the second vector according to the first group of non-zero elements in the first vector and the second group of elements in the second vector, wherein the bitmap of the first vector indicates the non-zero elements in the first vector.

[0054] In a possible implementation, after the first processing unit implements the operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements, it also includes: the first scheduling unit generates an erase signal, which instructs the first cache and the second cache to erase the data currently cached by each cache.

[0055] In a possible implementation manner, the first vector belongs to part or all of the elements in any row of the first matrix, and the second vector belongs to part or all of the elements in any column of the second matrix.

[0056] In a possible implementation, the matrix calculation device further includes a second processing unit, and the first selection signal can also enable the second processing unit to obtain a third group of elements in a third vector; the second processing unit is used to implement operations on the first vector and the third vector based on the first group of non-zero elements and the third group of elements; wherein the third vector belongs to a column in the second matrix other than the column where the second vector is located.

[0057] In a possible implementation, the matrix calculation device further includes a second scheduling unit and a third processing unit, the second scheduling unit being used to generate a second selection signal according to a bitmap of a fourth vector, the second selection signal being used to enable a third processing element to obtain a fourth group of non-zero elements in a fourth vector; and to enable the third processing element to obtain a fifth group of elements in a second vector, wherein the bitmap of the fourth vector indicates non-zero elements in the fourth vector, and the fourth vector belongs to a row in the first matrix other than a row where the first vector is located; and the third processing unit being used to implement operations on the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

[0058] In a fourth aspect, the present application provides a computing device, comprising a chip and a memory, the memory being used to store code, the chip executing the code to implement the method as described in the second aspect and any possible implementation of the second aspect.

[0059] In a fifth aspect, the present application provides a computer-readable storage medium, in which instructions are stored on a computing device. When the instructions are executed on a computing device, the computing device executes the method described in the second aspect and any possible implementation of the second aspect.

[0060] In a sixth aspect, a computer program product comprising instructions is provided, including a computer program or instructions, which, when executed on a computing device, enables the computing device to execute the method described in the second aspect and any possible implementation of the second aspect.

[0061] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 A schematic diagram of the structure of a chip provided in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of a processing element provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of a buffer and a multiplexer provided in an embodiment of the present application;

[0066] Figure 4A schematic diagram of a multiplexer and a buffer connected in accordance with an embodiment of the present application;

[0067] Figure 5 A schematic diagram of a data unit priority provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of data in a buffer and a corresponding bitmap provided in an embodiment of the present application;

[0069] Figure 7 A schematic diagram of implementing vector-to-vector operations provided in an embodiment of the present application;

[0070] Figure 8 A schematic diagram of the structure of another chip provided in an embodiment of the present application;

[0071] Fig. 9 A schematic diagram of implementing vector and matrix operations provided in an embodiment of the present application;

[0072] Fig.10 is a schematic diagram of the structure of another chip provided in an embodiment of the present application;

[0073] Fig.11 A schematic diagram of implementing matrix-matrix operations provided in an embodiment of the present application;

[0074] Fig.12 It is a flowchart of a matrix calculation method provided in an embodiment of the present application;

[0075] Fig.13 is a schematic diagram of a matrix computing device provided in an embodiment of the present application;

[0076] Fig.14 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0078] At present, in scenarios involving matrix multiplication operations, in order to improve computational efficiency, pruning technology is usually used to sparse the matrix to reduce the amount of calculation and improve computational efficiency. For example, after AI training is completed, the trained weight matrix is ​​structured pruned to complete the sparsification of the weight matrix, and the weight matrix after structured pruning is used for reasoning during AI reasoning. However, the above method is only suitable for pruning the matrix after determining the matrix involved in the calculation, such as AI reasoning scenarios, but is not suitable for other scenarios where the sparsity of the matrix changes dynamically. For example, during AI training, the weight matrix, gradient matrix or activation matrix are dynamically changing, and the sparsity of the matrix is ​​also dynamically changing. If the sparsification of the matrix is ​​to be achieved during the AI ​​training process, a large time cost is required, which will offset the acceleration effect brought about by the sparsification of the matrix through pruning. Therefore, how to achieve the acceleration of matrix calculation under the dynamic change of matrix sparsity is a technical problem that needs to be solved urgently.

[0079] The present application provides a chip that can realize the operation of matrices with dynamically changing sparsity. Figure 1 As shown, Figure 1 1 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip 100 includes a processing element (Processing Elements, PE) 110, a first buffer 120, a first multiplexer (Multiplexer, MUX) 130, a second buffer 140, a second multiplexer 150 and a scheduling module 160. Among them, the input end of the first multiplexer 130 is connected to the first buffer 120, and the output end of the first multiplexer 130 is connected to the processing element 110; the input end of the second multiplexer 150 is connected to the second buffer 140, and the output end of the second multiplexer 150 is connected to the processing element 110; the control ends of the first multiplexer 130 and the second multiplexer 150 are respectively connected to the scheduling module 160.

[0080] When the operation of the first vector and the second vector is required, the first buffer 120 is used to cache all or part of the elements in the first vector; the second buffer 140 is used to cache all or part of the elements in the second vector. The scheduling module 160 is used to generate a selection signal according to the data in the first buffer 120, and send the selection signal to the first multiplexer 130 and the second multiplexer 150, and the selection signal is used to make the first multiplexer 130 obtain a group of non-zero elements from the connected first buffer 120, and make the second multiplexer obtain a group of elements from the connected second buffer 140; the first multiplexer 130 and the second multiplexer 150 input a group of data obtained from the corresponding buffer to the processing element 110, so that the processing element 110 performs the operation between the first vector and the second vector.

[0081] The following is a detailed introduction of each component of the chip 100 in conjunction with the accompanying drawings. Figure 2 As shown, Figure 2 1 is a schematic diagram of a processing element provided in an embodiment of the present application. The processing element 110 is a dot product processing module with a length of K, including K multiply accumulate (MAC) units and K-1 adders (or adder trees); one processing element 110 can obtain K elements from the first buffer 120 in one cycle, and can obtain K elements from the second buffer 140, that is, one processing element 110 can obtain K pairs of values ​​in one cycle, perform K multiplications and K-1 addition operations, and realize a1b1+a2b2+…+a K b K Operation. Among them, a1, a2, a3, ..., a K are the elements in the first vector, b1, b2, b3, …, b K are the elements of the second vector.

[0082] like Figure 3 As shown, Figure 3 Schematic diagram of a buffer and a multiplexer provided in an embodiment of the present application. Each buffer includes W rows and K columns of data units, each data unit is used to cache the value of an element, Figure 3 In the example, W is equal to 4 and K is equal to 8. Each buffer corresponds to K read ports, each read port is implemented as a multiplexer, and the multiplexer corresponding to a buffer includes K multiplexers, that is, Figure 2 The first multiplexer 130 and the second multiplexer 150 in the embodiment include K multiplexers respectively. Each multiplexer is connected to a plurality of data units in the buffer respectively, and each data unit is connected to at least one multiplexer. Each multiplexer can be used to read the data cached in the plurality of data units connected to the multiplexer. Each multiplexer can only read the data in one of the plurality of data units connected at a time according to the selection signal of the scheduling module 160.

[0083] like Figure 4 As shown, Figure 4 A schematic diagram of a multiplexer connected to a buffer provided in an embodiment of the present application. A multiplexer includes multiple multiplexers, each of which is connected to multiple data units in the buffer, and the multiple data units connected to a multiplexer have different priorities. In the embodiment of the present application, the data unit in each buffer is represented as (r, c), such as Figure 4As shown in , the data unit in the 1st row and 3rd column is represented as (1, 3), the data unit in the 4th row and 5th column is represented as (4, 5), and so on, then r is an integer of [1, W], and c is an integer of [1, K]. Figure 4 Take the connection relationship between the multiplexer and the buffer as an example, that is, each multiplexer is connected to 8 data units of the buffer, and the priorities of the 8 data units of the i-th multiplexed connection are from high to low: (1, i), (2, i), (3, i), (4, i), (2, i+1), (2, mod (i-1, 8)), (3, mod (i-2, 8)), (4, mod (i-3, 8)). Among them, mod is the modulus operation.

[0084] For example, Figure 5 , when i is equal to 1 (i.e., the first multiplexer) and i is equal to 4 (i.e., the fourth multiplexer), the priority order of the 8 data units connected to each of the two multiplexers is shown. Figure 5 The numbers 1 to 8 in the data unit represent the priority, and the smaller the number, the higher the priority; the priorities of the 8 data units connected to the first multiplexer are from high to low: (1, 1), (2, 1), (3, 1), (4, 1), (2, 2), (2, 8), (3, 7), (4, 6); the priorities of the 8 data units connected to the fourth multiplexer are from high to low: (1, 4), (2, 4), (3, 4), (4, 4), (2, 5), (2, 3), (3, 2), (4, 1).

[0085] It should be noted that the above Figure 4 The fact that each multiplexer is connected to 8 data units is only an example and should not be construed as a specific limitation. Each multiplexer can also be connected to more or fewer data units, for example, each multiplexer can be connected to 7 data units, or 10, 15 or 16 data units, etc. However, each data unit only needs to be connected to at least one multiplexer. Figure 4The connection relationship between each multiplexer and multiple data units is only an example and cannot be understood as a specific limitation. The present application does not make specific limitations on the connection relationship between each multiplexer and multiple data units. However, in the same chip, the data units included in each buffer should be W rows and K columns. A buffer and a multiplexer connected to the buffer are called a cache module, then the connection relationship between the multiplexer and the data unit in the cache in any two cache modules in the same chip is the same, that is, if a chip includes multiple cache modules, the connection relationship between the multiplexer and the data unit in the cache in each cache module is the same. For example, the relative positions of the multiple data units connected to the first multiplexer in the first multiplexer 130 in the first cache 120 are the same as the relative positions of the multiple data units connected to the first multiplexer in the second multiplexer 150 in the second cache 140. Exemplarily, the i-th multiplexer in the first multiplexer 130 is connected to the eight data units (1, i), (2, i), (3, i), (4, i), (2, i+1), (2, mod(i-1, 8)), (3, mod(i-2, 8)), and (4, mod(i-3, 8)) in the first buffer 120, then the i-th multiplexer in the second multiplexer is also connected to the eight data units (1, i), (2, i), (3, i), (4, i), (2, i+1), (2, mod(i-1, 8)), (3, mod(i-2, 8)), and (4, mod(i-3, 8)) in the second buffer. In this way, the i-th multiplexer of the first multiplexer and the i-th multiplexer of the second multiplexer can read the data in the data units with the same relative position in the first buffer 120 and the second buffer 140 according to the same selection signal, and the first multiplexer and the second multiplexer can read the elements to be multiplied in the first vector and the second vector according to the same selection signal.

[0086] In the embodiment of the present application, each multiplexer reads a non-zero element from the multiple data units connected to it each time. When determining which data unit a multiplexer reads data from, the scheduling module 160 determines whether the element in the data unit with a priority of 1 is 0, starting from the data unit with the highest priority, according to the priority level. If the element in the data unit with a priority of 1 is not 0, the scheduling module 160 generates a selection signal for the data unit with a priority of 1, so that the multiplexer reads the element in the data unit with a priority of 1; if the element in the data unit with a priority of 1 is 0, the scheduling module 160 determines whether the element in the data unit with a priority of 2 is 0. If the element in the data unit with a priority of 2 is not 0, the scheduling module 160 generates a selection signal for the data unit with a priority of 2, so that the multiplexer reads the element in the data unit with a priority of 2; if the element in the data unit with a priority of 2 is 0, the scheduling module 160 determines whether the element in the data unit with a priority of 3 is 0. By analogy, the scheduling module 160 finds a data unit whose stored elements are not 0 in the order of priority, and then generates a selection signal corresponding to the data unit, so that the multiplexer reads the elements in the data unit and sends them to the processing element 110, so that the processing element 110 performs a dot product operation.

[0087] The above buffer is connected to K multiplexers. The scheduling module 160 needs to generate K scheduling signals DS for the K multiplexers corresponding to one buffer in one cycle. j , each multiplexer corresponds to a selection signal. Wherein, j=1, 2, ..., 8, j is a positive integer. That is to say, in each cycle, each multiplexer needs to read an element from the buffer and send it to the processing element 110 according to the selection signal of the scheduling module 160. In one cycle, the multiplexer can obtain K elements from the connected buffer through K multiplexers. It should be noted that if the elements in multiple data units connected to a multiplexer are all 0, the multiplexer sends element 0 to the processing element 110.

[0088] In the embodiment of the present application, the scheduling module 160 determines whether the element in each data unit is 0 according to the bitmap corresponding to the data stored in the buffer. Figure 6 As shown, Figure 6It is a schematic diagram of the data in the buffer and the corresponding bitmap. After storing the data, each buffer will generate a bitmap according to the data cached in the data unit, and each bit in the bitmap corresponds to a data unit in the buffer. The value of each bit in the bitmap is 0 or 1, which is used to indicate whether the element in the corresponding data unit is 0. For example, when a bit in the bitmap is 0, it indicates that the value of the element in the data unit corresponding to the bit is 0; when a bit in the bitmap is 1, it indicates that the value in the data unit corresponding to the bit is not 0; or when a bit in the bitmap is 0, it indicates that the value in the data unit corresponding to the bit is not 0; when a bit in the bitmap is 1, it indicates that the value in the data unit corresponding to the bit is 0. In the embodiment of the present application, taking the example that a bit in the bitmap is 0 to indicate that the value in the data unit corresponding to the bit is 0, and when a bit in the bitmap is 1 to indicate that the value in the data unit corresponding to the bit is not 0, the embodiment of the present application is introduced.

[0089] When the scheduling module 160 needs to generate a selection signal for a multiplexer, the scheduling module 160 first determines whether the value of the bit corresponding to the data unit with a priority of 1 in the data unit connected to the multiplexer in the bitmap is 0. If the value of the bit corresponding to the data unit with a priority of 1 in the bitmap is 1, it means that the element in the data unit with a priority of 1 is not 0, and the scheduling module 160 generates a selection signal corresponding to the data unit with a priority of 1 and sends it to the multiplexer. If the value of the bit corresponding to the data unit with a priority of 1 in the bitmap is 0, it means that the element in the data unit with a priority of 1 is 0, and the scheduling module 160 then determines whether the value of the bit corresponding to the data unit with a priority of 2 in the data unit connected to the multiplexer in the bitmap is 0. If the value of the bit corresponding to the data unit with a priority of 2 in the bitmap is 1, it means that the element in the data unit with a priority of 2 is not 0, and the scheduling module generates a selection signal corresponding to the data unit with a priority of 2 and sends it to the multiplexer. If the value of the bit corresponding to the data unit with a priority of 2 in the bitmap is 0, it means that the element in the data unit with a priority of 2 is 0, and the scheduling module 160 then determines whether the value of the bit corresponding to the data unit with a priority of 3 in the data unit connected to the multiplexer in the bitmap is 0. And so on, which will not be repeated here.

[0090] It should be noted that after the scheduling module 160 controls a multiplexer to obtain a non-zero element from a data unit, the scheduling module 160 needs to store the element of the data unit in the corresponding position 0 in the bitmap to prevent the elements in the data unit from being read repeatedly, so as to avoid calculation errors caused by the non-zero elements in the data units with lower priority than the data unit not being read and participating in the calculation.

[0091] Above Figures 2 to 6 Introduced Figure 1The working principles of the various parts included in the chip 100 shown are described below in conjunction with the accompanying drawings to introduce a method of using the above chip 100 to implement vector-to-vector operations, vector-to-matrix operations, and matrix-to-matrix operations.

[0092] like Figure 7 As shown, Figure 7 Schematic diagram of a method for implementing vector-to-vector operations provided by an embodiment of the present application. Figure 1 Taking the chip 100 in the embodiment of the present invention as an example to implement the dot product operation of vector C and vector D, the first buffer 120 and the second buffer 140 in the chip 100 both include W rows and K columns of data units, and the first multiplexer 130 and the second multiplexer 150 both include K multiplexers. Vector C is a 1*WK vector, and vector D is a WK*1 vector, that is, vector C includes 1 row and W*K columns of elements, and vector D includes W*K rows and 1 column of elements.

[0093] When the chip 100 performs a dot product operation, the chip 100 loads the elements included in the vector C into the data units of the first buffer 120 , and loads the elements included in the vector D into the data units of the second buffer 140 . When the data of vector C is loaded into the first buffer 120 and the data of vector D is loaded into the second buffer 140, the data from the 1st to the Kth columns of vector C are sequentially stored in the K data units of the first row of the first buffer 120, and the data from the 1st to the Kth rows of vector D are sequentially stored in the K data units of the first row of the second buffer 140; the data from the K+1th to the 2Kth columns of vector C are sequentially stored in the K data units of the second row of the first buffer 120, and the data from the K+1th to the 2Kth rows of vector D are sequentially stored in the K data units of the second row of the second buffer 140; and so on, until the data from the (W-1)Kth to the WKth columns of vector C are sequentially stored in the K data units of the Wth row of the first buffer 120, and the data from the (W-1)Kth to the WKth rows of vector D are sequentially stored in the K data units of the Wth row of the second buffer 140.

[0094] After the W*K data of the vector C are stored in the first buffer 120, the first buffer 120 generates a corresponding bitmap according to the value of the element stored in each data unit. If the value of an element in a data unit is not 0, the corresponding bit of the data unit in the bitmap is set to 1; if the value of an element in a data unit is 0, the corresponding bit of the data unit in the bitmap is set to 0. Figure 6 As shown, Figure 6 The bitmap shown in is a bitmap corresponding to the data cached in the first buffer 120 . Figure 6 Taking W equal to 4 and K equal to 8 as an example, the connection relationship between each multiplexer in the first multiplexer 130 and the data unit in the first buffer 120 is as follows: Figure 4 It should be understood that the second buffer 140 may also generate a corresponding bitmap according to the value in each data unit, which is not specifically limited in the embodiment of the present application.

[0095] After generating the bitmap, the first buffer 120 sends the bitmap to the scheduling module 160. After receiving the bitmap, the scheduling module 160 needs to first generate a selection signal DS1 for the first multiplexer in the first multiplexer 130. Specifically, the scheduling module 160 first determines whether the value of the bit corresponding to the data unit with a priority of 1 among the data units connected to the first multiplexer in the first multiplexer 130 is 0 in the bitmap according to the priorities of the multiple data units connected to the first multiplexer in the first multiplexer 130. If the value of the corresponding bit in the bitmap of the data unit with a priority of 1 is not 0, the scheduling module 160 generates a selection signal 000 and sends the selection signal 000 to the first multiplexer of the first multiplexer 130 and the first multiplexer of the second multiplexer 150. The selection signal 000 is used to enable the first multiplexer of the first multiplexer 130 to read the elements in the data unit with a priority of 1 and send them to the processing element 110, and to enable the first multiplexer of the second multiplexer 150 to read the elements in the data unit with a priority of 1 and send them to the processing element 110.

[0096] If the scheduling module 160 determines that the value of the bit corresponding to the data unit with a priority of 1 in the data unit connected to the first multiplexer in the first multiplexer 130 in the bitmap is 0, the scheduling module 160 then determines whether the value of the bit corresponding to the data unit with a priority of 2 in the data unit connected to the first multiplexer in the first multiplexer 130 in the bitmap is 0. If the value of the bit corresponding to the data unit with a priority of 2 in the bitmap is not 0, the scheduling module 160 generates a selection signal 001, and sends the selection signal 001 to the first multiplexer of the first multiplexer 130 and the first multiplexer of the second multiplexer 150. The selection signal 001 is used to enable the first multiplexer of the first multiplexer 130 to read the element in the data unit with a priority of 2 and send it to the processing element 110, and to enable the first multiplexer of the second multiplexer 150 to read the element in the data unit with a priority of 2 and send it to the processing element 110.

[0097] If the scheduling module 160 determines that the value of the bit corresponding to the data unit with a priority of 2 in the data unit connected to the first multiplexer in the first multiplexer 130 is 0, the scheduling module 160 then determines whether the value of the bit corresponding to the data unit with a priority of 3 in the data unit connected to the first multiplexer in the first multiplexer 130 is 0. This continues in a similar manner until the first multiplexer in the first multiplexer 130 reads a data c1 from the first buffer 120 and sends it to the processing element 110, and the first multiplexer in the second multiplexer 150 reads a data d1 from the second buffer 140 and sends it to the processing element 110, so that the processing element 110 performs the operation of c1*d1. It should be noted that if there are non-zero data in the elements cached by the multiple data units connected to the first multiplexer in the first multiplexer 130, the value of c1 is not 0, and d1 may be 0 or not 0; if the data cached by the multiple data units connected to the first multiplexer in the first multiplexer 130 are all 0, the value of c1 is 0, and d1 may be 0 or not 0.

[0098] For the second to the Kth multiplexers of the first multiplexer 130, the scheduling module 160 sequentially generates corresponding selection signals DS2 to DS K , so that each of the first multiplexer 130 and the second multiplexer 150 outputs a data to the processing element 110. In the first cycle, each of the first multiplexer 130 and the second multiplexer 150 outputs a data to the processing element 110, so that the processing element 110 completes K times of product operations and K-1 times of addition operations. The above K times of product operations and K-1 times of addition operations are: e1 = c1d1 + c2d2 + ... + c t d t +…+c K d K , where c t represents the data output by the t-th multiplexer in the first multiplexer 130, d t represents the data output by the t-th multiplexer in the second multiplexer 150, where t is a positive integer greater than 0 and less than or equal to K.

[0099] It should be noted that, after the scheduling module 160 generates a selection signal and sends it to the first multiplexer 130 and the first multiplexer 130 reads a data from a data unit, the scheduling module 160 sets the corresponding position of the data unit in the bitmap to 0.

[0100] In the second cycle and each subsequent cycle, the scheduling module 160 continues to execute the operations executed in the first cycle, so that each multiplexer in the first multiplexer 130 and the second multiplexer 150 outputs a data to the processing element 110, and the processing element 110 completes K times of multiplication operations and K-1 times of addition operations. Until the values ​​of all bits in the bitmap are 0. Finally, the values ​​obtained after the processing element 110 completes K times of multiplication operations and K-1 times of addition operations in each cycle are added together, which is the dot product of vector C and vector D.

[0101] Since a buffer includes W*K data units, and a cycle multiplexer can read the data in K data units to participate in the calculation, the dot product operation of vector C and vector D can be completed after W cycles at most. If vector C has a certain sparsity, that is, there are some elements with values ​​of 0 in vector C, through the above-mentioned chip, even if the sparsity of vector C input into the buffer is changed each time, the elements with values ​​of 0 in vector C can be prevented from participating in the calculation, thereby reducing the amount of calculation and improving the calculation efficiency without reducing the calculation accuracy.

[0102] It should be noted that after the scheduling module 160 determines that the values ​​in the bitmap are all 0, the scheduling module 160 generates an erase signal and sends it to the first cache 120 and the second cache 140, so that the first cache 120 and the second cache 140 erase the currently cached data for caching the next batch of data.

[0103] It should be understood that the above-mentioned vector C can be a part of vector X, and vector D can be a part of vector Y. For example, vector X is a 1*Z vector, and vector Y is a Z*1 vector, where Z is greater than W*K. Since a buffer in chip 100 can only store W*K data at a time, vector X and vector Y are divided, and at most W*K elements are stored in the buffer of chip 100 for calculation each time. The above-mentioned vector C can be a row vector with 1 row and W*K columns, and vector D can be any column in a matrix with W*K rows and T columns. The operation result of vector C and the matrix is ​​a vector with 1 row and T columns. In the process of performing the operation of vector C and the matrix, one column in the matrix is ​​cached in the second buffer 140 each time to obtain an element in the operation result.

[0104] like Figure 8 As shown, Figure 8 2 is a schematic diagram of another chip provided in an embodiment of the present application. The chip 200 includes N processing elements PE1 to PE N , N+1 buffers B0~B N , N+1 multiplexers M0~M Nand a scheduling module 210. Where N is an integer greater than or equal to 2. Buffer B0 is connected to the scheduling module 210, and the scheduling module 210 is connected to each multiplexer. The N+1 buffers correspond to the N+1 multiplexers one by one, and each multiplexer is connected to a buffer. Multiplexer M0 is connected to each of the N processing elements, and each data read by multiplexer M0 is synchronously sent to the N processing elements. The connection relationship between each multiplexer and the corresponding buffer can refer to the above Figure 3 and Figure 4 The corresponding description will not be repeated here.

[0105] In the embodiment of the present application, the chip 200 can implement vector and matrix multiplication operations. Figure 8 Take the chip 200 shown as an example to implement the multiplication operation of vector C and matrix B, wherein the buffers in chip 200 all include data units with W rows and K columns, and the multiplexers all include K multiplexers. Vector C is a 1*WK vector, and matrix B is a WK*N matrix, that is, vector C includes 1 row and W*K column elements, and matrix B includes W*K row and N column elements. It should be understood that matrix B is equivalent to N WK*1 vectors, that is, matrix B is equivalent to N of the above-mentioned vectors D; the multiplication operation of vector C and matrix B is equivalent to performing a vector-to-vector dot product operation on vector C and each column of matrix B, so matrix B can be regarded as including vectors D1~D N Each vector corresponds to a column in the matrix B.

[0106] like Fig. 9 As shown, Fig. 9 is a schematic diagram of implementing vector and matrix operations provided by an embodiment of the present application. Fig. 9 The black squares in vector C and matrix B represent non-zero elements. When performing a multiplication operation of vector C and matrix B through chip 200, chip 200 loads the elements included in vector C into the data unit of buffer B0, loads the elements of vector D1 corresponding to the first column of matrix B into the data unit of buffer B1, loads the elements of vector D2 corresponding to the second column of matrix B into the data unit of buffer B2, loads the elements included in vector D3 corresponding to the third column of matrix B into the data unit of buffer B3, and so on, and loads the elements included in vector D corresponding to the Nth column of matrix B into the data unit of buffer B3. N The elements are loaded into buffer B N The method of loading vector C into buffer B0 can refer to the above Figure 7 In the corresponding embodiment, the method for loading the vector C into the first buffer 120 is to load the vectors D1 to D N The method of loading them into the corresponding buffers can refer to the above Figure 7The method of loading the vector D into the second buffer 140 in the corresponding embodiment will not be described in detail here.

[0107] After the WK data of the vector C are stored in the buffer B0, the buffer B0 generates a corresponding bitmap. The method for generating the corresponding bitmap by the buffer B0 can refer to the method for generating the bitmap by the first buffer 120, which will not be described in detail here. After generating the bitmap, the buffer B0 sends the bitmap to the scheduling module 210. After receiving the bitmap, the scheduling module 210 first generates a selection signal DS1 for the first multiplexer in the multiplexer M0. The method for generating the selection signal DS1 by the scheduling module 210 can refer to the method for generating the selection signal DS1 by the scheduling module 160, which will not be described in detail here.

[0108] In the embodiment of the present application, after generating the selection signal DS1, the scheduling module 210 sends the selection signal DS1 to the multiplexers M0 to M1. N The first multiplexer in the multiplexer M0 reads a data according to the selection signal DS1 and sends the data to the N processing elements PE0~PE N . Multiplexer M1~M N The first multiplexer in reads a data according to the selection signal DS1 and sends it to the respectively connected processing elements.

[0109] For multiplexers M0~M N The scheduling module 160 generates corresponding selection signals DS2 to DS3 in sequence by the same method as above. K , so that multiplexers M0~M N Each multiplexer in the outputs a data to the processing element 110. The scheduling module 210 generates the selection signals DS1 to DS2 in sequence within one cycle. K After that, each processing element will obtain K pairs of data, complete K multiplication operations and K-1 addition operations.

[0110] It should be noted that, after the scheduling module 210 generates a selection signal and sends it to the multiplexer M0, and the multiplexer M0 reads a data from a data unit, the scheduling module 210 places the data unit at the corresponding position 0 in the bitmap.

[0111] In the second cycle and each subsequent cycle, the scheduling module 210 continues to perform the operations performed in the first cycle, so that the multiplexers M0-M N Each multiplexer in outputs a data to the processing element 110, and the processing element 110 performs K multiplication operations and K-1 addition operations until the values ​​of all bits in the bitmap are 0. For any processing element PE h, add the values ​​obtained after completing K multiplication operations and K-1 addition operations in each cycle, which is vector C and vector D h Wherein, h is a positive integer greater than or equal to 1 and less than or equal to N.

[0112] The result of the multiplication of vector C and matrix B is a 1*N vector H, where the processing element PE h Complete vector C and vector D h The value output after the dot product operation is the value of the hth element in vector H.

[0113] It should be understood that the above-mentioned vector C can be a part of the vector X, or part or all of the elements of a row of a matrix E; the matrix B can be a part of the matrix F, for example, the vector X is a 1*Z vector, and the matrix F is a Z*N vector, where Z is greater than W*K. Since a buffer in the chip 200 can only store W*K data at a time, the vector X and the matrix F are divided, and the W*K elements of the vector X are stored in the buffer B0 of the chip 200 each time, and the W*K elements of the hth column in the matrix F are stored in the buffer B of the chip 200. h That is, the W*K elements from the 1st column to the Nth column in the matrix F are distributed and stored in the buffers B0 to B0 of the chip 200. N middle.

[0114] like Fig.10 As shown, Fig.10 300 is a schematic diagram of another chip provided in an embodiment of the present application. The chip 300 includes M*N processing elements PE, M+N buffers, M*(N+1) multiplexers and M scheduling modules S1 to S M The connection relationship of the various parts included in the chip 300 is as follows: Fig.10 As shown in , M*N processing elements are distributed in a matrix, and the M*N processing elements are distributed in M ​​rows, and each row includes N processing elements. Fig.10 Each row in the structure of chip 300 is equivalent to Figure 8 The structure of the chip 200 shown in FIG. 3 is equivalent to the chip 300 including M Figure 8 Chip 200 is shown.

[0115] In the embodiment of the present application, the chip 300 can implement matrix-matrix multiplication operations. Fig.10Take the chip 300 shown as an example to implement the multiplication operation of matrix A and matrix B. Each buffer in chip 300 includes W rows and K columns of data units, and each multiplexer includes K multiplexers. Matrix A is an M*WK matrix, and matrix B is a WK*N matrix, that is, matrix A includes M rows and W*K columns of elements, and matrix B includes W*K rows and N columns of elements. It should be understood that matrix A is equivalent to M 1*WK vectors, that is, matrix A is equivalent to M of the above-mentioned vectors C; therefore, the multiplication operation of matrix A and matrix B is equivalent to performing a vector-matrix multiplication operation on each row of matrix A and matrix B respectively. Therefore, matrix A can be regarded as including vectors C1~C M There are M vectors, each of which corresponds to a row in the matrix A.

[0116] Above Fig. 9 The chip 200 shown can perform the multiplication operation of vector C and matrix B, and the matrix A is equivalent to M vectors C, and the chip 300 is equivalent to M Figure 8 or Fig. 9 The structure of the chip 200 is shown. Fig.11 As shown, Fig.11 A schematic diagram of implementing matrix-matrix operations provided in an embodiment of the present application. Chip 300 can calculate the multiplication of M vectors and matrix B at the same time according to the method of chip 200 for calculating the multiplication of vector C and matrix B. That is, chip 300 can calculate the M vectors (vectors C1 to C2) included in matrix A. M ) and matrix B, and then obtain the multiplication result of matrix A and matrix B. Among them, matrix A is an M*WK matrix, matrix B is a WK*N matrix, and the multiplication result of matrix A and matrix B is an M*N matrix Q. The N processing elements in the g-th row of chip 300 complete vector C g The 1*N vector output after the multiplication operation of the matrix B is the element of the g-th row in the matrix Q, where g is a positive integer greater than or equal to 1 and less than or equal to M. The process of implementing vector and matrix multiplication by the buffer, multiplexer, scheduling module and processing element included in each row of the chip 300 can refer to the above Fig. 9 The chip 200 shown implements the process of multiplying the vector C and the matrix B, which will not be described in detail here.

[0117] It should be noted that the buffer B in the chip 300 10 , B 20 ,…B g0 ,…B M0 After each buffer in B obtains the elements in matrix A, it will generate a bitmap corresponding to the data in its own buffer and send it to the scheduling module connected to it. 10 Generate Bits Figure 1 Sent to scheduling modules S1 and B20 Generate Bits Figure 2 Sent to scheduling module S2, B g0 Generate Bits Figure 1 Sent to the scheduling module S g etc.

[0118] It should be understood that the above matrix A can be a part of the matrix G; the matrix B can be a part of the matrix F. For example, the matrix A is an M*Z vector, and the matrix F is a Z*N vector, where Z is greater than W*K. Since a buffer in the chip 300 can only store W*K data at a time, the matrix G and the matrix F are divided, and the W*K elements of the g-th row of the matrix G are stored in the buffer B of the chip 200 each time. g0 In the example, the W*K elements of the hth column in the matrix F are stored in the buffer B of the chip 200. 1h middle.

[0119] Through the above chip 300, in the process of performing matrix-matrix multiplication operations, elements with element values ​​of 0 in the matrix can be excluded from the calculation, thereby reducing the amount of calculation and improving the calculation efficiency without reducing the calculation accuracy.

[0120] The following is combined with the above Figures 1 to 11 , introduces a matrix calculation method provided in an embodiment of the present application, which is applied to a chip capable of implementing matrix calculation, such as the above Figure 1 In the chip 100 shown in FIG. 1 , the relevant structure of the chip 100 can refer to the above Figures 1 to 5 The relevant description will not be repeated here. Fig.12 As shown, Fig.12 It is a flowchart of a matrix calculation method provided in an embodiment of the present application, and the matrix calculation method includes the following S121 to S123.

[0121] S121. The chip caches the first vector and the second vector.

[0122] The chip may be the above Figure 1 The chip 100 in the embodiment. The chip 100 caches the first vector in the first cache and caches the second vector in the second cache. The method for the chip 100 to cache the first vector in the first cache can refer to the method for the chip 100 to cache the vector C in the first cache 120, and the method for caching the second vector in the second cache can refer to the method for the chip 100 to cache the vector D in the second cache 140, which will not be repeated here.

[0123] It should be noted that the first vector can be the above Figure 7 In the embodiment shown, the second vector may be the vector C as described above. Figure 7Vector D in the illustrated embodiment. The first vector may be a part of vector X, and the second vector may be a part of vector Y, for example, vector X is a 1*Z vector, and vector Y is a Z*1 vector, where Z is greater than W*K. Since a buffer in chip 100 can only store W*K data at a time, vector X and vector Y are divided, and at most W*K elements are stored in the buffer of chip 100 for calculation each time. The first vector may also be a row in the first matrix or a part of a row in the first matrix, and the second vector may also be a column in the second matrix or a part of a column in the second matrix, for example, the above-mentioned vector X is a row in the first matrix, and vector Y is a column in the second matrix.

[0124] S122. The first scheduling module of the chip generates a first selection signal according to the bitmap of the first vector.

[0125] The bitmap of the first vector indicates the non-zero elements in the first vector. The bitmap of the vector can refer to the above Figure 6 The corresponding related description is not repeated here. The above-mentioned first selection signal can enable the first processing element to obtain the first group of non-zero elements in the first vector from the first buffer, and enable the first processing element to obtain the second group of elements in the second vector from the second buffer. The second group of elements obtained from the second vector may include non-zero elements or all-zero elements.

[0126] In the embodiment of the present application, the chip 100 further includes a first multiplexer and a second multiplexer, and the first processing element obtains a first group of non-zero elements of the first vector from the first buffer through the first multiplexer, and obtains a second group of non-zero elements of the second vector from the second buffer through the second multiplexer. Figure 7 In the scheduling module 160, the first multiplexer may be Figure 7 The first multiplexer 130 in the embodiment of the present invention, the second multiplexer may be Figure 7 The second multiplexer 150 in the embodiment of the present invention is as follows: Figure 7 The corresponding strobe signals DS1 to DS K The process of the first scheduling module generating the first selection signal according to the bitmap of the first vector can refer to the above Figure 7 In the corresponding embodiment, the scheduling module 160 generates the selection signals DS1-DS according to the bitmap of the vector C. K The process will not be described in detail here.

[0127] S123. The first processing element of the chip implements the operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements.

[0128] The first processing element may be Figure 7In the processing element 110, the first group of non-zero elements are K data obtained from the first buffer by the K multiplexers of the first multiplexer according to the received selection signal, and the second group of elements are K data obtained from the second buffer by the K multiplexers of the second multiplexer according to the received selection signal. The method by which the first multiplexer and the second multiplexer obtain the data in the buffer according to the received selection signal can refer to the first multiplexer 130 and the second multiplexer 150 mentioned above, and the method by which the first processing element can realize the operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements can refer to the method by which the processing element 110 realizes the operation between the vector C and the vector D, which will not be repeated here.

[0129] In the embodiment of the present application, the chip may further include a third buffer and a second processing element, the third buffer is used to buffer the third vector, and the first selection signal may also enable the second processing element to obtain the third group of elements in the third vector from the third buffer, and enable the second processing element to obtain the first group of non-zero elements in the first vector from the first buffer. The second processing element implements the operation of the first vector and the third vector according to the first group of non-zero elements in the first vector and the third group of elements in the third vector. The third vector belongs to a column of the second matrix other than the column where the second vector is located.

[0130] In a possible implementation, the first buffer may be Figure 8 The second buffer can be Figure 8 The first multiplexer can be Figure 8 The second multiplexer can be M0. Figure 8 In the multiplexer M1, the first processing element can be Figure 8 The third buffer may be Figure 8 Buffers B2 to B N Any one or more of the second processing element is a processing element connected to the third buffer, and the third multiplexer is a multiplexer connected to the third buffer. That is, when the chip includes the third ring buffer and the second processing element, the chip can be the above Figure 8 The chip 200 shown in FIG. 200 can be referred to in the above description for the method of implementing the operation of the first vector and the second vector, and the operation of the first vector and the third vector by the chip 200. Figure 8 and Fig. 9 The relevant description in the corresponding embodiment will not be repeated here.

[0131] In the embodiment of the present application, the chip may further include a fourth buffer, a second scheduling module and a third processing element on the basis of including a first buffer, a second buffer, a first scheduling module, a first processing element, a third buffer and a second processing element. The fourth buffer is used to cache a fourth vector, wherein the fourth vector belongs to a row in the first matrix other than the row where the first vector is located. The second scheduling module generates a second selection signal according to the bitmap of the fourth vector, and the second selection signal enables the third processing element to obtain the fourth group of non-zero elements in the fourth vector from the fourth buffer; enables the third processing element to obtain the fifth group of elements in the second vector from the second buffer, and the second processing element can implement the operation of the fourth vector and the second vector according to the fourth group of non-zero elements in the fourth vector and the fifth group of elements in the second vector. Wherein, the bitmap of the fourth vector indicates the non-zero elements in the fourth vector.

[0132] In a possible implementation, the first buffer may be Fig.10 Middle Buffer B 10 , the second buffer can be Fig.10 Buffer B in 11 , the first multiplexer can be Fig.10 The multiplexer M 10 , the second multiplexer can be Fig.10 The multiplexer M in 11 , the first processing element may be Fig.10 Processing element PE 11 The third buffer can be Fig.10 Buffer B in 12 ~B 1N Any one or more of the following, the second processing element is a processing element connected to the third buffer, and the third multiplexer is a multiplexer connected to the third buffer; the second scheduling module may be Fig.10 Middle scheduling module S2~S M Any one or more of, the fourth buffer is a buffer connected to the second scheduling module, the third processing element is Fig.10 One or more processing elements in the same row as the second scheduling module. That is, when the chip includes the first buffer, the second buffer, the first scheduling module, the first processing element, the third buffer and the second processing element, and further includes the fourth buffer, the second scheduling module and the third processing element, the chip may be the above-mentioned Fig.10 The chip 300 shown in FIG. 1 can refer to the above method for implementing the operation of the first vector and the second vector, the first vector and the third vector, and the fourth vector and the second vector. Fig.10 and Fig.11 The relevant description in the corresponding embodiment will not be repeated here.

[0133] For the above method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0134] Other reasonable step combinations that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of the present invention. Secondly, those skilled in the art should also be familiar with the fact that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0135] Combined with the above Figures 1 to 12 The chip provided in the embodiment of the present application and the method for performing matrix calculation based on the chip are described in detail. Fig.13 and Fig.14 , introduces the matrix computing device and apparatus provided in the embodiments of the present application. Fig.13 As shown, Fig.13 It is a schematic diagram of a matrix computing device provided by an embodiment of the present application. The matrix computing device may be the above-mentioned chip 100, chip 200 or chip 300. The matrix computing device 131 includes a first scheduling unit 132 and a first processing unit 133, wherein the first scheduling unit 132 is used to generate a first selection signal according to the bitmap of the first vector, and the first selection signal enables the first processing unit 133 to obtain a first group of non-zero elements in the first vector from the first buffer, and enables the first processing unit 133 to obtain a second group of elements in the second vector from the second buffer, and the first processing unit 133 is used to implement operations on the first vector and the second vector according to the first group of non-zero elements in the first vector and the second group of elements in the second vector, wherein the bitmap of the first vector indicates the non-zero elements in the first vector. Specifically, the matrix computing device 131 may be the above-mentioned Figure 7 In the chip 100 shown, the first scheduling unit 132 can be used to implement the functions implemented by the scheduling module 160, and the first processing unit 133 can be used to implement the functions implemented by the processing element 110, which will not be described in detail here.

[0136] In a possible implementation, after the first processing unit 133 implements the operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements, it also includes: the first scheduling unit 132 generates an erase signal, which instructs the first cache and the second cache to erase the currently cached data.

[0137] In a possible implementation manner, the first vector belongs to part or all of the elements in any row of the first matrix, and the second vector belongs to part or all of the elements in any column of the second matrix.

[0138] In a possible implementation, the matrix calculation device further includes a second processing unit 134, and the first selection signal can also enable the second processing unit 134 to obtain a third group of elements in the third vector, and enable the second processing unit 134 to obtain a first group of non-zero elements in the first vector; the second processing unit 134 implements the operation of the first vector and the third vector according to the first group of non-zero elements and the third group of elements; wherein the third vector belongs to a column of the second matrix other than the column where the second vector is located. Specifically, when the matrix calculation device includes the second processing unit 134, the matrix calculation device 131 can be the above Figure 8 or Fig. 9 The chip 200 shown in FIG. 1 is a first scheduling unit 132 that can be used to implement the above Fig. 9 The first processing unit 133 can be used to implement the functions implemented by the scheduling module 210. Fig. 9 The second processing unit 134 is used to implement the functions implemented by the processing element PE1. Fig. 9 The functions implemented by the processing elements other than the processing element PE1 are not described in detail here.

[0139] In one possible implementation, the matrix calculation device further includes a second scheduling unit 135 and a third processing unit 136, the second scheduling unit 135 generates a second selection signal according to the bitmap of the fourth vector, and the second selection signal is used to enable the third processing unit 136 to obtain a fourth group of non-zero elements in the fourth vector; to enable the third processing unit 136 to obtain a fifth group of elements in the second vector, wherein the bitmap of the fourth vector indicates the non-zero elements in the fourth vector; and the third processing unit 136 is used to implement operations on the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

[0140] Specifically, when the matrix calculation device further includes a second processing unit 134, a second scheduling unit 135 and a third processing unit 136, the matrix calculation device 131 may be the above-mentioned Fig.10 or Fig.11 The chip 300 shown in FIG. 1 is a first scheduling unit 132 that can be used to implement the above Fig.11 The first processing unit 133 can be used to implement the functions implemented by the scheduling module S1. Fig.11 Middle processing element PE 11 The second processing unit 134 is used to implement the above functions. Fig.11 The first row except the processing element PE 11The first scheduling unit 132 can be used to implement the functions implemented by other processing elements. Fig.11 The functions implemented by other scheduling modules except scheduling module S1 are not described here.

[0141] Fig.14 1 is a schematic diagram of a computing device provided in an embodiment of the present application, wherein the computing device 141 includes the above chip 100, chip 200 or chip 300. The computing device 141 also includes a processor 142, a chip 143, a memory 144 and a communication interface 145, wherein the processor 142, the memory 144 and the communication interface 145 communicate via a bus 146.

[0142] Chip 143 may be any one of the above-mentioned chips 100 , 200 or 300 , and may assist computing device 141 in implementing various functions implemented by the above-mentioned chips 100 , 200 or 300 .

[0143] Chip 143 can implement the above under the scheduling of processor 142 Figures 7 to 12 The operations in the corresponding embodiments. The processor 142 can have a variety of specific implementation forms, for example, the processor 142 can be a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processing unit (NPU) or a tensor processing unit (TPU), and the processor 142 can also be a single-core processor or a multi-core processor. The processor 142 can be a combination of a CPU and a hardware chip. The above-mentioned hardware chip can be an ASIC, a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. The processor 142 can also be implemented using a logic device with built-in processing logic alone, such as an FPGA or a digital signal processor (DSP).

[0144] The memory 144 may be a non-volatile memory, such as a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The memory 144 may also be a volatile memory, which may be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0145] The memory 144 can be used to store program codes and data, such as caching the above-mentioned vectors or matrices, so that the chip 143 can call the program codes stored in the memory 144 to execute. Figures 1 to 12 The operating steps in the corresponding embodiment.

[0146] The communication interface 145 is a wired interface (such as an Ethernet interface), an internal interface (such as a high-speed serial computer expansion bus (Peripheral Component Interconnect express, PCIE) bus interface), a wired interface (such as an Ethernet interface) or a wireless interface (such as a cellular network interface or a wireless local area network interface) for communicating with other computing devices or modules.

[0147] The bus 146 is a Peripheral Component Interconnect Express (PCIE) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus 146 includes an out-of-band bus and a high-speed bus, etc. For the sake of clarity, various buses are labeled as bus 146 in the figure.

[0148] Need to explain, Fig.14 This is only one possible implementation of the embodiment of the present application. In actual applications, the computing device 141 also includes more or fewer components, which is not limited here.

[0149] The present application provides a computer-readable storage medium, including: the computer-readable storage medium stores computer instructions; when the computer instructions are executed on a computing device, the computing device executes the above Figures 1 to 12 The operations in the corresponding embodiments.

[0150] The present application embodiment provides a computer program product including instructions, including a computer program or instructions, which, when executed on a computer, causes the computing device to execute the above-mentioned Figures 1 to 12 The operations in the corresponding embodiments.

[0151] The above embodiments are implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments are implemented in whole or in part in the form of a computer program product. The computer program product includes at least one computer instruction. When the computer program instruction is loaded or executed on a computer, the process or function according to the embodiment of the present invention is generated in whole or in part. The computer is a general-purpose computer, a special-purpose computer, a computer network, or other programming device. The computer instructions are stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions are transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium is any medium that can be accessed by a computer or a data storage node such as a server, data center, etc. that contains at least one medium set. The medium is a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a high-density digital video disc (DVD), or a semiconductor medium.

[0152] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent repairs or replacements within the technical scope disclosed by the present invention, and these repairs or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A chip, It is characterized in that comprising a first buffer, a second buffer, a first scheduling module and a first processing element, The first buffer is used to cache the first vector; The second buffer is used to cache the second vector; The first scheduling module is configured to generate a first gating signal according to a bitmap of the first vector, the bitmap of the first vector indicating non-zero elements in the first vector; The first processing element is used to obtain a first group of non-zero elements in the first vector from the first buffer based on the first selection signal, obtain a second group of elements in the second vector from the second buffer, and implement operations between the first vector and the second vector according to the first group of non-zero elements and the second group of elements.

2. The chip according to claim 1, It is characterized in that The chip also includes a first multiplexer and a second multiplexer; The first multiplexer is used to obtain the first group of non-zero elements in the first vector from the first buffer according to the first selection signal, and input the first processing element; The second multiplexer is used to obtain the second group of elements in the second vector from the second buffer according to the first selection signal, and input the second group of elements into the first processing element.

3. The chip according to claim 2, It is characterized in that The first multiplexer includes K multiplexers, and the second multiplexer includes K multiplexers; the first buffer and the second buffer each include W rows and K columns of data units, each data unit is used to cache one element; Each multiplexer in the first multiplexer is connected to a plurality of data units in the first buffer, and each data unit is connected to at least one multiplexer; The connection relationship between the i-th multiplexer in the second multiplexer and the data unit in the second buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

4. The chip according to claim 3, It is characterized in that The first scheduling module is specifically used for: According to the bitmap of the first vector, it is determined that among the data units connected to the j-th multiplexer of the first multiplexer, the element stored in the k-th data unit is a non-zero element, the first scheduling module generates a selection signal for the j-th multiplexer, and sends the selection signal of the j-th multiplexer to the j-th multiplexer of the first multiplexer and the j-th multiplexer of the second multiplexer, and the first selection signal includes the selection signal of the j-th multiplexer of the first multiplexer.

5. The chip according to claim 4, It is characterized in that The first multiplexer is specifically used to: obtain, through the jth multiplexer of the first multiplexer, according to a selection signal of the jth multiplexer of the first multiplexer, a first element in a kth data unit in the data units connected to the jth multiplexer of the first multiplexer, and input the first element to the first processing element, wherein the first element is one of the first group of non-zero elements; The second multiplexer is specifically used to: according to the selection signal of the jth multiplexer of the first multiplexer, obtain the second element in the kth data unit in the data unit connected to the jth multiplexer of the second multiplexer through the jth multiplexer of the second multiplexer, and input the second element into the first processing element, the second element being one of the second group of elements.

6. The chip according to any one of claims 1 to 5, It is characterized in that The first scheduling module is further used to generate an erase signal after implementing the operation between the first vector and the second vector, and the erase signal is used to instruct the first buffer and the second buffer to erase currently cached data.

7. The chip according to claim 4 or 5, It is characterized in that The first vector belongs to any row in the first matrix, and the second vector belongs to any column in the second matrix.

8. The chip according to claim 7, It is characterized in that The chip also includes a third buffer and a second processing element; wherein, The third buffer is used to buffer a third vector, and the third vector belongs to a column of the second matrix other than the column where the second vector is located; the first selection signal is also used to enable the second processing element to obtain a third group of elements in the third vector from the third buffer; The second processing element is used to implement an operation between the first vector and the third vector according to the first group of non-zero elements and the third group of elements.

9. The chip according to claim 8, It is characterized in that The chip also includes a third multiplexer, configured to obtain the third group of elements in the third vector from a third buffer according to the first selection signal, and input the third group of elements into the second processing element.

10. The chip according to claim 9, It is characterized in that The third multiplexer includes K multiplexers, the third buffer includes W rows and K columns of data units, each data unit is used to cache one element; The connection relationship between the i-th multiplexer in the third multiplexer and the data unit in the third buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

11. The chip according to claim 10, It is characterized in that The third multiplexer is specifically used to: according to the selection signal of the jth multiplexer of the first multiplexer, obtain the third element in the kth data unit in the data unit connected to the jth multiplexer of the third multiplexer through the jth multiplexer of the third multiplexer, and input the third element into the second processing element, the third element being one of the third group of elements.

12. The chip according to any one of claims 7 to 11, It is characterized in that The chip further includes a fourth buffer, a second scheduling module and a third processing element; wherein, The fourth buffer is used to cache a fourth vector, where the fourth vector belongs to a row of the first matrix other than the row where the first vector is located; The second scheduling module is used to generate a second selection signal according to the bitmap of the fourth vector, the second selection signal is used to enable the third processing element to obtain a fourth group of non-zero elements in the fourth vector from the fourth buffer; to enable the third processing element to obtain a fifth group of elements in the second vector from the second buffer, the bitmap of the fourth vector indicating the non-zero elements in the fourth vector; The third processing element is used to implement an operation between the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

13. The chip according to claim 12, It is characterized in that The chip also includes a fourth multiplexer and a fifth multiplexer, The fourth multiplexer is used for acquiring the fourth group of non-zero elements in the fourth vector from the fourth buffer according to the second selection signal, and inputting the fourth group of non-zero elements in the fourth vector into the third processing element; The fifth multiplexer is used to obtain the fifth group of elements in the second vector from the second buffer according to the second selection signal, and input the fifth group of elements into the third processing element.

14. The chip according to claim 13, It is characterized in that The fourth multiplexer includes K multiplexers, and the fifth multiplexer includes K multiplexers; the fourth buffer includes W rows and K columns of data units, each data unit is used to cache one element; Each multiplexer in the fourth multiplexer is connected to a plurality of data units in the fourth buffer, and each data unit is connected to at least one multiplexer; the connection relationship between the i-th multiplexer in the fourth multiplexer and the data unit in the fourth buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer; The connection relationship between the i-th multiplexer in the fifth multiplexer and the data unit in the second buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

15. The chip according to claim 14, It is characterized in that The second scheduling module is specifically used for: According to the bitmap of the fourth vector, it is determined that among the data units connected to the j-th multiplexer of the fourth multiplexer, the element stored in the m-th data unit is a non-zero element, the second scheduling module generates a selection signal for the j-th multiplexer of the fourth multiplexer, and sends the selection signal of the j-th multiplexer of the fourth multiplexer to the j-th multiplexer of the fourth multiplexer and the j-th multiplexer of the fifth multiplexer, and the second selection signal includes the selection signal of the j-th multiplexer of the fourth multiplexer.

16. The chip according to claim 15, It is characterized in that The fourth multiplexer is specifically used to: obtain, through the jth multiplexer of the fourth multiplexer, according to the selection signal of the jth multiplexer of the fourth multiplexer, a fourth element in the mth data unit in the data units connected to the jth multiplexer of the fourth multiplexer, and input the fourth element to the third processing element, wherein the fourth element is one of the fourth group of non-zero elements; The fifth multiplexer is specifically used to: according to the selection signal of the j-th multiplexer of the fourth multiplexer, obtain the fifth element in the m-th data unit in the data unit connected to the j-th multiplexer of the fifth multiplexer through the j-th multiplexer of the fifth multiplexer, and input the fifth element into the third processing element, the fifth element being one of the fifth group of elements.

17. A matrix calculation method, It is characterized in that Applied to chips, including: The chip caches a first vector and a second vector, wherein the first vector is cached in a first cache of the chip, and the second vector is cached in a second cache of the chip; A first scheduling module of the chip generates a first gating signal according to a bitmap of a first vector, the bitmap of the first vector indicating non-zero elements in the first vector; The first processing element of the chip obtains a first group of non-zero elements in the first vector from the first buffer based on the first selection signal, obtains a second group of elements in the second vector from the second buffer, and implements operations between the first vector and the second vector according to the first group of non-zero elements and the second group of elements.

18. The method according to claim 17, It is characterized in that The chip further includes a first multiplexer and a second multiplexer; and the method further includes: The first multiplexer of the chip obtains the first group of non-zero elements in the first vector from the first buffer according to the first selection signal, and inputs the first processing element; The second multiplexer of the chip obtains the second group of elements in the second vector from the second buffer according to the first selection signal, and inputs the second group of elements into the first processing element.

19. The method according to claim 18, It is characterized in that The first multiplexer includes K multiplexers, and the second multiplexer includes K multiplexers; the first buffer and the second buffer each include W rows and K columns of data units, each data unit is used to cache one element; Each multiplexer in the first multiplexer is connected to a plurality of data units in the first buffer, and each data unit is connected to at least one multiplexer; The connection relationship between the i-th multiplexer in the second multiplexer and the data unit in the second buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

20. The method according to claim 19, It is characterized in that The first scheduling module of the chip generates a first selection signal according to the bitmap of the first vector, including: The first scheduling module of the chip determines, according to the bitmap of the first vector, that the element stored in the kth data unit in the data unit connected to the jth multiplexer of the first multiplexer is a non-zero element, generates a selection signal for the jth multiplexer, and sends the selection signal of the jth multiplexer to the jth multiplexer of the first multiplexer and the jth multiplexer of the second multiplexer, wherein the first selection signal includes the selection signal of the jth multiplexer of the first multiplexer.

21. The method according to claim 20, It is characterized in that The first multiplexer of the chip obtains the first group of non-zero elements in the first vector from the first buffer according to the first selection signal and inputs the first processing element, and the second multiplexer of the chip obtains the second group of elements in the second vector from the second buffer according to the first selection signal and inputs the first processing element, including: The first multiplexer of the chip obtains a first element in a kth data unit among the data units connected to the jth multiplexer of the first multiplexer through the jth multiplexer of the first multiplexer according to a selection signal of the jth multiplexer of the first multiplexer, and inputs the first element to the first processing element, wherein the first element is one of the first group of non-zero elements; The second multiplexer of the chip obtains the second element of the kth data unit in the data units connected to the jth multiplexer of the second multiplexer through the jth multiplexer of the second multiplexer according to the selection signal of the jth multiplexer of the first multiplexer, and inputs the second element into the first processing element, the second element being one of the second group of elements.

22. The method according to any one of claims 17 to 21, It is characterized in that After the first processing element of the chip implements the operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements, the chip further includes: The first scheduling module of the chip generates an erase signal, and the erase signal instructs the first buffer and the second buffer to erase currently cached data.

23. The method according to claim 20 or 21, It is characterized in that The first vector belongs to any row in the first matrix, and the second vector belongs to any column in the second matrix.

24. The method according to claim 23, It is characterized in that The chip further includes a third buffer and a second processing element; and the method further includes: The chip caches a third vector, the third vector is cached in the third cache, the third vector belongs to a column of the second matrix other than the column where the second vector is located, and the first selection signal is further used to enable the second processing element to obtain a third group of elements in the third vector from the third cache; The second processing element of the chip implements an operation between the first vector and the third vector according to the first group of non-zero elements and the third group of elements.

25. The method according to claim 24, It is characterized in that The chip further includes a third multiplexer, and the method further includes: The third multiplexer of the chip obtains the third group of elements in the third vector from a third buffer according to the first selection signal, and inputs the third group of elements into the second processing element.

26. The method according to claim 25, It is characterized in that The third multiplexer includes K multiplexers, the third buffer includes W rows and K columns of data units, each data unit is used to cache one element; The connection relationship between the i-th multiplexer in the third multiplexer and the data unit in the third buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

27. The method according to claim 26, It is characterized in that The third multiplexer of the chip obtains the third group of elements in the third vector from the third buffer according to the first selection signal, and inputs the third group of elements into the second processing element, including: The third multiplexer of the chip obtains the third element of the kth data unit in the data units connected to the jth multiplexer of the third multiplexer through the jth multiplexer of the third multiplexer according to the selection signal of the jth multiplexer of the first multiplexer, and inputs the third element into the second processing element, and the third element is one of the third group of elements.

28. The method according to any one of claims 23 to 27, It is characterized in that The chip further includes a fourth buffer, a second scheduling module and a third processing element; and the method further includes: The chip caches a fourth vector, the fourth vector is cached in the fourth cache, and the fourth vector belongs to a row of the first matrix except the row where the first vector is located; The second scheduling module of the chip generates a second selection signal according to the bitmap of the fourth vector, the second selection signal is used to enable the third processing element to obtain a fourth group of non-zero elements in the fourth vector from the fourth buffer; and to enable the third processing element to obtain a fifth group of elements in the second vector from the second buffer, the bitmap of the fourth vector indicating the non-zero elements in the fourth vector; The third processing element of the chip implements an operation on the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

29. The method according to claim 28, It is characterized in that The chip further includes a fourth multiplexer and a fifth multiplexer, and the method further includes: The fourth multiplexer of the chip obtains the fourth group of non-zero elements in the fourth vector from the fourth buffer according to the second selection signal, and inputs the fourth group of non-zero elements into the third processing element; The fifth multiplexer of the chip obtains the fifth group of elements in the second vector from the second buffer according to the second selection signal, and inputs the fifth group of elements into the third processing element.

30. The method according to claim 29, It is characterized in that The fourth multiplexer includes K multiplexers, and the fifth multiplexer includes K multiplexers; the fourth buffer includes W rows and K columns of data units, each data unit is used to cache one element; Each multiplexer in the fourth multiplexer is connected to a plurality of data units in the fourth buffer, and each data unit is connected to at least one multiplexer; the connection relationship between the i-th multiplexer in the fourth multiplexer and the data unit in the fourth buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer; The connection relationship between the i-th multiplexer in the fifth multiplexer and the data unit in the fifth buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data unit in the first buffer.

31. The method according to claim 30, It is characterized in that The second scheduling module of the chip generates a second selection signal according to the bitmap of the fourth vector, including: The second scheduling module of the chip determines, according to the bitmap of the fourth vector, that the element stored in the mth data unit in the data unit connected to the jth multiplexer of the fourth multiplexer is a non-zero element, generates a selection signal of the jth multiplexer of the fourth multiplexer, and sends the selection signal of the jth multiplexer of the fourth multiplexer to the jth multiplexer of the fourth multiplexer and the jth multiplexer of the fifth multiplexer, and the second selection signal includes the selection signal of the jth multiplexer of the fourth multiplexer.

32. The method according to claim 31, It is characterized in that The fourth multiplexer of the chip obtains the fourth group of non-zero elements in the fourth vector from the fourth buffer according to the second selection signal, and inputs the fourth group of non-zero elements into the third processing element; the fifth multiplexer of the chip obtains the fifth group of elements in the second vector from the second buffer according to the second selection signal, and inputs the fifth group of elements into the third processing element, including: The fourth multiplexer of the chip obtains the fourth element in the mth data unit in the data units connected to the jth multiplexer of the fourth multiplexer through the jth multiplexer of the fourth multiplexer according to the selection signal of the jth multiplexer of the fourth multiplexer, and inputs the fourth element to the third processing element, wherein the fourth element is one of the fourth group of non-zero elements; The fifth multiplexer of the chip obtains the fifth element in the mth data unit in the data unit connected to the jth multiplexer of the fifth multiplexer through the jth multiplexer of the fifth multiplexer according to the selection signal of the jth multiplexer of the fourth multiplexer, and inputs the fifth element into the third processing element, and the fifth element is one of the fifth group of elements.

33. A matrix computing device, It is characterized in that The matrix calculation device includes a first scheduling unit and a first processing unit, wherein: The first scheduling unit is configured to generate a first gating signal according to a bitmap of a first vector, the bitmap of the first vector indicating non-zero elements in the first vector; The first processing unit is used to obtain a first group of non-zero elements in the first vector from the first buffer based on the first selection signal, obtain a second group of elements in the second vector from the second buffer, and implement operations between the first vector and the second vector according to the first group of non-zero elements and the second group of elements.

34. The device according to claim 33, It is characterized in that The first scheduling unit is further configured to generate an erase signal after the first processing unit completes the operation between the first vector and the second vector, wherein the erase signal instructs the first buffer and the second buffer to erase currently cached data.

35. The device according to claim 33 or 34, It is characterized in that The first vector belongs to any row in the first matrix, and the second vector belongs to any column in the second matrix.

36. The device according to claim 35, It is characterized in that The matrix calculation device further includes a second processing unit, the first selection signal is further used to enable the second processing unit to obtain a third group of elements in the third vector, the third vector belongs to a column of the second matrix other than the column where the second vector is located, The second processing unit is used to implement an operation between the first vector and the third vector according to the first group of non-zero elements and the third group of elements.

37. The device according to claim 35 or 36, It is characterized in that The matrix calculation device further includes a second scheduling unit and a third processing unit. The second scheduling unit is used to generate a second selection signal according to the bitmap of the fourth vector, and the second selection signal is used to enable the third processing unit to obtain a fourth group of non-zero elements in the fourth vector; causing the third processing unit to obtain a fifth group of elements in the second vector, the bitmap of the fourth vector indicating non-zero elements in the fourth vector, the fourth vector belonging to a row of the first matrix other than a row where the first vector is located; The third processing unit is used to implement an operation between the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

38. A computing device, It is characterized in that The computing device comprises a chip and a memory, wherein the memory is used to store codes, and the chip is used to execute the codes to implement the method according to any one of claims 17 to 32.

Images