A matrix calculation method, chip and related device

Abstract

The application provides a matrix calculation method, a chip and related equipment; the chip comprises a first buffer, a second buffer, a scheduling module and a processing element; the first buffer is used for buffering a first vector, and the second buffer is used for buffering a second vector; the scheduling module generates a gating signal according to a bitmap of the first vector, the gating signal can enable the processing element to obtain a group of non-zero elements in the first vector from the first buffer and enable the processing element to obtain a group of elements in the second vector from the second buffer; and then, the operation of the first vector and the second vector is performed according to the group of non-zero elements in the first vector and the group of elements in the second vector, wherein the bitmap of the first vector is used for indicating the non-zero elements in the first vector. When the operation of two vectors is performed by the chip provided in the application, the elements with a value of 0 in one vector can not participate in the calculation, so that the calculation amount can be reduced and the calculation efficiency can be improved.

Description

[0001] This application is a divisional application, the original application's application number is 202210460849.1, the original application's original date is April 28, 2022, and the original application's entire content is incorporated by reference in this application. TECHNICAL FIELD

[0002] The present application relates to the technical field of computers, and in particular to a matrix calculation method, a chip and related equipment. BACKGROUND

[0003] In the fields of artificial intelligence (AI) and high performance computing (HPC), a large number of matrix multiplication operations are involved, such as AI model training and AI inference. In order to speed up the calculation and improve the calculation efficiency, the matrix is usually sparsified by pruning technology, and then the sparsified matrix is used for calculation, thereby reducing the calculation amount in the calculation process. For example, after AI training is completed, the trained weight matrix is structured pruned to complete the sparsification of the weight matrix, and the structured pruned weight matrix is used for inference in AI inference. However, the above method only supports pruning a certain matrix and using the pruned matrix for calculation, and does not support scenarios where the sparsity of the matrix dynamically changes during calculation. SUMMARY

[0004] The present application provides a matrix calculation method, a chip and related equipment, which can support the calculation of a matrix with dynamically changing sparsity, reduce the calculation amount in the calculation, improve the calculation efficiency, and not reduce the calculation accuracy.

[0005] In a first aspect, the present application provides a chip, comprising: a first buffer, a second buffer, a first scheduling module and a first processing element, wherein the first buffer is configured to buffer a first vector; the second buffer is configured to buffer a second vector; the first scheduling module is configured to generate a first gating signal according to a bitmap of the first vector, the first gating signal being capable of causing the first processing element to obtain a first group of non-zero elements in the first vector from the first buffer and to obtain a second group of elements in the second vector from the second buffer; and the first processing element is configured to implement an operation of the first vector and the second vector according to the first group of non-zero elements in the first vector and the second group of elements in the second vector, wherein the bitmap of the first vector indicates non-zero elements in the first vector.

[0006] Each bit in the bitmap of the first vector corresponds to an element in the first vector, and the value of each bit indicates whether the corresponding element in the first vector is 0. For example, if the value of a bit is 0, it indicates that the value of the corresponding element in the first vector is 0, and if the value of a bit is 1, it indicates that the value of the corresponding element in the first vector is not 0. The first scheduling module can determine which elements in the first vector are non-zero elements according to the bitmap of the first vector, so that the first processing element can only obtain the non-zero elements in the first vector, and then obtain the elements at the corresponding positions in the second vector to perform the operation between the first vector and the second vector, thereby reducing the amount of calculation and improving the calculation efficiency. Moreover, in the process of performing the dot product of vectors, the product of an element with a value of 0 in one vector and a non-zero element in the other vector is 0, which does not affect the result of the dot product of the two vectors. Therefore, only the non-zero elements in the first vector are obtained to perform the operation of the first vector and the second vector, which does not reduce the calculation accuracy.

[0007] In a possible implementation, the chip further includes a first multiplexer and a second multiplexer; the first multiplexer is configured to obtain a first group of non-zero elements in the first vector from the first buffer according to a first gate signal and input the first processing element; and the second multiplexer is configured to obtain a second group of elements in the second vector from the second buffer according to the first gate signal and input the first processing element.

[0008] In a possible implementation, the first multiplexer and the second multiplexer each include K multiplexers; the first buffer and the second buffer each include W rows and K columns of data units, and each data unit is configured to buffer an element in a vector or a matrix; each multiplexer in the first multiplexer is connected to a plurality of data units in the first buffer, and each data unit is connected to at least one multiplexer; and the connection relationship between the i th multiplexer in the second multiplexer and the data units in the second buffer is the same as the connection relationship between the i th multiplexer in the first multiplexer and the data units in the first buffer.

[0009] In the operation of the first vector and the second vector, the rth element of the first vector and the rth element of the second vector need to be multiplied, and the rth element of the first vector is stored in the same relative position of the first register as the rth element of the second vector is stored in the second register when the elements of the first vector are stored in the first register and the elements of the second vector are stored in the second register. For example, the rth element of the first vector is stored in the rth data unit of the first row of the first register, and the rth element of the second vector is stored in the rth data unit of the first row of the second register. Meanwhile, the ith multiplexer in the first multiplexer is connected to a plurality of data units in the first register. The relative positions of the plurality of data units in the first register are the same as the relative positions of the plurality of data units in the second register connected to the ith multiplexer in the second multiplexer. For example, the ith multiplexer in the first multiplexer is connected to the second data unit of the first row and the fifth data unit of the second row in the first register, and the ith multiplexer in the second multiplexer is connected to the second data unit of the first row and the fifth data unit of the second row in the second register. In this way, the ith multiplexer of the first multiplexer and the ith multiplexer of the second multiplexer can read data in the data units with the same relative positions in the first register and the second register according to the same strobe signal, so that the first multiplexer and the second multiplexer read exactly the elements that need to be multiplied in the first vector and the second vector according to the same strobe signal.

[0010] In a possible implementation, the first scheduling module is specifically configured to: determine, according to the bitmap of the first vector, that the kth data unit connected to the jth multiplexer of the first multiplexer stores a non-zero element, and then generate a strobe signal of the jth multiplexer of the first multiplexer, and send the strobe signal of the jth multiplexer of the first multiplexer to the jth multiplexer of the first multiplexer and the jth multiplexer of the second multiplexer. The first strobe signal includes the strobe signal of the jth multiplexer of the first multiplexer.

[0011] The first multiplexer includes K multiplexers, and the first scheduling module generates the respective strobe signals of the K multiplexers in one period, that is, the first strobe signal includes the respective strobe signals of the K multiplexers. In this way, each multiplexer in the first multiplexer and the second multiplexer reads a data unit from the connected data unit according to the received strobe signal;

[0012] The plurality of data units connected by the one multiplexer have different priorities. Before generating the strobe signal of the one multiplexer, the first scheduling module determines whether the element of the first vector stored in the data unit with the highest priority among the data units connected by the one multiplexer is 0 according to the bitmap of the first vector. If the element stored in the data unit with the highest priority is not 0, the strobe signal corresponding to the data unit with the highest priority is generated. If the element stored in the data unit with the highest priority is 0, whether the element stored in the data unit with the second highest priority is 0 is determined according to the bitmap of the first vector. If the element stored in the data unit with the second highest priority is not 0, the strobe signal corresponding to the data unit with the second highest priority is generated. If the element stored in the data unit with the second highest priority is 0, whether the element stored in the data unit with the third highest priority is 0 is determined according to the bitmap of the first vector. The process is repeated until the strobe signal of the one multiplexer is generated. By setting the priorities, the elements in the plurality of data units connected by each multiplexer can be read in order.

[0013] It should be noted that after the first scheduling module determines that the element stored in one data unit is not 0 and generates the strobe signal corresponding to the data unit, the bit position corresponding to the element stored in the data unit in the bitmap is set to 0, so as to prevent the element in the data unit from being read again and causing calculation errors.

[0014] In a possible implementation, the first multiplexer is specifically configured to: acquire, according to the strobe signal of the jth multiplexer of the first multiplexer, a first element in the kth data unit connected by the jth multiplexer of the first multiplexer, and input the first element to the first processing element, the first element being one of the first set of non-zero elements; and the second multiplexer is specifically configured to: acquire, according to the strobe signal of the jth multiplexer of the first multiplexer, a second element in the kth data unit connected by the jth multiplexer of the second multiplexer, and input the second element to the first processing element, the second element being one of the second set of elements.

[0015] In one cycle, the first scheduling module generates, in sequence, a corresponding gating signal for each of the K multiplexers in the first multiplexer, so that the first multiplexer and the second multiplexer each take out K elements from the connected buffers, and then perform the dot product of the first vector and the second vector based on the K pairs of data taken out. It should be understood that the first multiplexer may not be able to take out all the non-zero elements in the first buffer in one cycle, but the first multiplexer can take out K non-zero elements at a time, and the first buffer stores W rows of K elements, so that all the non-zero elements in the first buffer can be taken out in at most W cycles, and the operation between the first vector and the second vector is completed.

[0016] In one possible implementation, after the first processing element completes the operation of the first vector and the second vector, the first scheduling module generates an erase signal for instructing the first buffer and the second buffer to erase the currently buffered data for buffering data required for the next calculation.

[0017] In one possible implementation, the first vector can be a part of a vector, and the second vector can be a part of another vector; or the first vector can be a row vector, and the second vector can be a column in a matrix; or the first vector can belong to any row in a first matrix, and the second vector can belong to any column in a second matrix. Through multiple calculation processes described above, the chip can implement the operation of vectors and vectors, the operation of vectors and matrices, or the operation of matrices and matrices.

[0018] In one possible implementation, the chip further includes a third buffer and a second processing element, the third buffer is used to buffer a third vector, the third vector belongs to a column in the second matrix other than the column in which the second vector is located; the first gating signal is further used to make the second processing element obtain a third group of elements in the third vector from the third buffer; and the second processing element is used to perform the operation of the first vector and the third vector based on the first group of non-zero elements and the third group of elements.

[0019] By adding the third buffer and the second processing element, the operation of the first vector and the third vector can be completed by the second processing element, and the operation of the first vector and the second vector can be completed by the first processing element, and the second vector and the third vector both belong to the second matrix, so that the chip can implement the operation of vectors and matrices.

[0020] In one possible implementation, the chip further includes a third multiplexer, the third multiplexer is used to obtain a third group of elements in the third vector from the third buffer based on the first gating signal, and input the second processing element.

[0021] In a possible implementation, the third multiplexer includes K multiplexers, and the third buffer includes W rows and K columns of data units, each of which is configured to buffer one element; and a connection relationship between the i th multiplexer in the third multiplexer and the data units in the third buffer is the same as that between the i th multiplexer in the first multiplexer and the data units in the first buffer.

[0022] The third multiplexer is configured to obtain elements of the third vector from the third buffer and input the elements to the second processing element, so that the second processing element implements the operation of the first vector and the third vector. In the operation of the first vector and the third vector, the r th element of the first vector and the r th element of the third vector need to be multiplied, and when the elements of the first vector are stored in the first buffer and the elements of the third vector are stored in the third buffer, the relative positions of the r th element of the first vector stored in the first buffer and the r th element of the third vector stored in the third buffer are the same. For example, the r th element of the first vector is stored in the r th data unit of the first row of the first buffer, and the r th element of the third vector is stored in the r th data unit of the first row of the third buffer. Meanwhile, the i th multiplexer in the first multiplexer is connected to a plurality of data units in the first buffer. The relative positions of the plurality of data units in the first buffer are the same as those of a plurality of data units connected to the i th multiplexer in the third multiplexer in the third buffer. For example, the i th multiplexer in the first multiplexer is connected to the second data unit of the first row and the fifth data unit of the second row in the first buffer, and the i th multiplexer in the third multiplexer is connected to the second data unit of the first row and the fifth data unit of the second row in the third buffer. In this way, the i th multiplexer in the first multiplexer and the i th multiplexer in the third multiplexer can read data in data units with the same relative positions in the first buffer and the third buffer according to the same strobe signal, so that the first multiplexer and the third multiplexer read exactly the elements that need to be multiplied in the first vector and the third vector according to the same strobe signal.

[0023] In a possible implementation, the third multiplexer is specifically configured to: according to the strobe signal of the j th multiplexer in the first multiplexer, obtain, by the j th multiplexer in the third multiplexer, a third element in the k th data unit of the data units connected to the j th multiplexer in the third multiplexer, and input the third element to the second processing element, where the third element is one of the third group of elements.

[0024] In a possible implementation, the chip further includes a fourth buffer, a second scheduling module, and a third processing element; the fourth buffer is configured to buffer a fourth vector, the fourth vector belonging to a row of the first matrix other than the row in which the first vector is located;

[0025] The second scheduling module is configured to generate a second strobe signal according to a bitmap of the fourth vector, the second strobe signal being used to enable the third processing element to obtain a fourth set of non-zero elements in the fourth vector from the fourth buffer; and enable the third processing element to obtain a fifth set of elements in the second vector from the second buffer; the bitmap of the fourth vector indicates the non-zero elements in the fourth vector.

[0026] The third processing element is configured to implement an operation between the fourth vector and the second vector according to the fourth set of non-zero elements and the fifth set of elements.

[0027] By adding the fourth buffer, the second scheduling module, and the third processing element after adding the third buffer and the second processing element, the operation between the fourth vector in the first matrix and the second vector in the second matrix can be completed by the third processing element, the operation between the first vector in the first matrix and the third vector in the second matrix can be completed by the second processing element, and the operation between the first vector in the first matrix and the second vector in the second matrix can be completed by the second element, so that the chip can implement the operation between matrices.

[0028] In a possible implementation, the chip further includes a fourth multiplexer and a fifth multiplexer; the fourth multiplexer is configured to obtain the fourth set of non-zero elements in the fourth vector from the fourth buffer according to the second strobe signal and input the third processing element; and the fifth multiplexer is configured to obtain the fifth set of elements in the second vector from the second buffer according to the second strobe signal and input the third processing element.

[0029] In a possible implementation, the fourth multiplexer includes K multiplexers, and the fifth multiplexer includes K multiplexers; the fourth buffer includes W rows of K columns of data units, and each data unit is configured to buffer an element;

[0030] Each multiplexer in the fourth multiplexer is connected with a plurality of data units in the fourth buffer, and each data unit is connected with at least one multiplexer; a connection relationship between the i th multiplexer in the fourth multiplexer and the data units in the fourth buffer is the same as a connection relationship between the i th multiplexer in the first multiplexer and the data units in the first buffer; and a connection relationship between the i th multiplexer in the fifth multiplexer and the data units in the second buffer is the same as the connection relationship between the i th multiplexer in the first multiplexer and the data units in the first buffer.

[0031] It should be noted that the fifth multiplexer can also be connected with a fifth buffer, but since the fifth multiplexer is used to obtain the elements of the second vector, and the second buffer stores the second vector, the fifth multiplexer can share the second multiplexer with the second multiplexer, so as to reduce the complexity of the chip, reduce the size of the chip, and reduce the cost.

[0032] The fourth multiplexer is used to obtain the elements of the third vector from the fourth buffer and input the third processing element, so that the third processing element implements the operation of the fourth vector and the second vector. In the operation of the fourth vector and the second vector, the rth element of the fourth vector and the rth element of the second vector need to be multiplied. When the elements of the fourth vector are stored in the fourth buffer and the elements of the second vector are stored in the second buffer, the relative position of the rth element of the fourth vector stored in the fourth buffer is the same as the relative position of the rth element of the second vector stored in the second buffer. For example, the rth element of the second vector is stored in the rth data unit of the first row of the second buffer, and the rth element of the fourth vector is stored in the rth data unit of the first row of the fourth buffer. At the same time, the i th multiplexer in the fourth multiplexer is connected with a plurality of data units in the fourth buffer. The relative positions of the plurality of data units in the fourth buffer are the same as the relative positions of the plurality of data units in the second buffer connected with the i th multiplexer in the fifth multiplexer. In this way, the i th multiplexer of the fourth multiplexer and the i th multiplexer of the fifth multiplexer can read the data in the data units with the same relative positions in the fourth buffer and the second buffer according to the same strobe signal, so that the fourth multiplexer and the fifth multiplexer read exactly the elements that need to be multiplied in the fourth vector and the second vector according to the same strobe signal.

[0033] In a possible implementation, the second scheduling module is specifically configured to: determine, according to the bitmap of the fourth vector, that the m th data unit of the data units connected with the j th multiplexer of the fourth multiplexer stores a non-zero element, and the first scheduling module generates the strobe signal of the j th multiplexer, and sends the strobe signal of the j th multiplexer to the j th multiplexer of the fourth multiplexer and the j th multiplexer of the fifth multiplexer. The second strobe signal includes the strobe signal of the j th multiplexer of the fourth multiplexer.

[0034] In a possible implementation, the fourth multiplexer is specifically configured to: according to the strobe signal of the jth multiplexer of the fourth multiplexer, acquire, through the jth multiplexer of the fourth multiplexer, a third element in the mth data unit of the data units connected to the jth multiplexer of the fourth multiplexer, and input the third element to the third processing element, the third element being one of the fourth set of non-zero elements; and the fifth multiplexer is specifically configured to: according to the strobe signal of the jth multiplexer of the fourth multiplexer, acquire, through the jth multiplexer of the fifth multiplexer, a fourth element in the mth data unit of the data units connected to the jth multiplexer of the fifth multiplexer, and input the fourth element to the third processing element, the fourth element being one of the fifth set of elements.

[0035] In one cycle, the second scheduling module generates the strobe signals corresponding to the K multiplexers in the fourth multiplexer in sequence, so that the fourth multiplexer and the fifth multiplexer respectively take out K elements from the connected buffers, and then perform the dot product of the fourth vector and the second vector based on the K pairs of data. It should be understood that the second multiplexer may not be able to take out all the non-zero elements in the fourth buffer in one cycle, but the fourth multiplexer can take out K non-zero elements at a time, and the fourth buffer stores W rows of K columns of elements, so that all the non-zero elements in the fourth buffer can be taken out in at most W cycles, and the operation between the fourth vector and the second vector is completed.

[0036] In a second aspect, the present application provides a matrix calculation method applied to a chip, the chip including a first buffer, a second buffer, a first scheduling module, and a first processing element. The method includes: the chip buffering a first vector through the first buffer and buffering a second vector through the second buffer; after buffering the first vector and the second vector, the first scheduling module of the chip generates a first strobe signal according to a bitmap of the first vector, the first strobe signal enabling the first processing element to acquire a first set of non-zero elements in the first vector from the first buffer and enabling the first processing element to acquire a second set of elements in the second vector from the second buffer, wherein the bitmap of the first vector indicates the non-zero elements in the first vector; and then the chip performing an operation between the first vector and the second vector through the first processing element according to the first set of non-zero elements and the second set of elements.

[0037] In a possible implementation, the chip further includes a first multiplexer and a second multiplexer; the first multiplexer of the chip acquires the first set of non-zero elements in the first vector from the first buffer according to the first strobe signal and inputs the first processing element; and the second multiplexer of the chip acquires the second set of elements in the second vector from the second buffer according to the first strobe signal and inputs the first processing element.

[0038] In a possible implementation, the first multiplexer and the second multiplexer each include K multiplexers; the first buffer and the second buffer each include W rows and K columns of data units, each data unit being configured to buffer one element; each multiplexer in the first multiplexer is connected to a plurality of data units in the first buffer, and each data unit is connected to at least one multiplexer; the i th multiplexer in the second multiplexer has the same connection relationship with the data units in the second buffer as the i th multiplexer in the first multiplexer has with the data units in the first buffer.

[0039] In a possible implementation, the first scheduling module of the chip generates the first strobe signal according to the bitmap of the first vector, including: the chip determines, according to the bitmap of the first vector, that the k th data unit in the data units connected to the j th multiplexer of the first multiplexer stores a non-zero element, and the first scheduling module generates the strobe signal of the j th multiplexer, and sends the strobe signal of the j th multiplexer to the j th multiplexer of the first multiplexer and the j th multiplexer of the second multiplexer, where the first strobe signal includes the strobe signal of the j th multiplexer of the first multiplexer.

[0040] In a possible implementation, the first multiplexer of the chip acquires the first group of non-zero elements in the first vector from the first buffer according to the first strobe signal and inputs the first processing element, and the second multiplexer of the chip acquires the second group of elements in the second vector from the second buffer according to the first strobe signal and inputs the first processing element, including: the first multiplexer of the chip acquires the first element in the k th data unit in the data units connected to the j th multiplexer of the first multiplexer through the j th multiplexer of the first multiplexer according to the strobe signal of the j th multiplexer of the first multiplexer, and inputs the first element to the first processing element; and the second multiplexer of the chip acquires the second element in the k th data unit in the data units connected to the j th multiplexer of the second multiplexer through the j th multiplexer of the second multiplexer according to the strobe signal of the j th multiplexer of the first multiplexer, and inputs the second element to the first processing element, where the first element is one of the first group of non-zero elements, and the second element is one of the second group of elements.

[0041] In a possible implementation, the first vector belongs to any row in the first matrix, and the second vector belongs to any column in the second matrix. That is, the first vector can be part or all of the elements in any row of a matrix, and the second vector can be part or all of the elements in any column of another matrix.

[0042] In a possible implementation, the chip further includes a third buffer and a second processing element; and the method further includes: the chip buffering, by the third buffer, a third vector belonging to a column of the second matrix other than the column in which the second vector is located, the first selection signal being capable of causing the second processing element to obtain a third set of elements in the third vector from the third buffer; and the second processing element of the chip performing an operation on the first vector and the third vector according to the first set of non-zero elements and the third set of elements.

[0043] In a possible implementation, the chip further includes a third multiplexer, and the method further includes: the third multiplexer of the chip obtaining the third set of elements in the third vector from the third buffer according to the first selection signal, and inputting the second processing element.

[0044] In a possible implementation, the third multiplexer includes K multiplexers, and the third buffer includes W rows and K columns of data units, each data unit being configured to buffer an element; a connection relationship between the i th multiplexer in the third multiplexer and the data units in the third buffer is the same as a connection relationship between the i th multiplexer in the first multiplexer and the data units in the first buffer.

[0045] In a possible implementation, the third multiplexer of the chip obtains the third set of elements in the third vector from the third buffer according to the first selection signal, and inputs the second processing element, including:

[0046] The third multiplexer of the chip obtains, according to the selection signal of the j th multiplexer of the first multiplexer, a third element in the k th data unit of the data units connected to the j th multiplexer of the third multiplexer through the j th multiplexer of the third multiplexer, and inputs the third element to the second processing element, where the third element is one of the third set of elements.

[0047] In a possible implementation, the chip further includes a fourth buffer, a second scheduling module, and a third processing element; and the method further includes: the chip buffering, by the fourth buffer, a fourth vector belonging to a row of the first matrix other than the row in which the first vector is located; the second scheduling module of the chip generating a second selection signal according to a bitmap of the fourth vector, the second selection signal being used to cause the third processing element to obtain a fourth set of non-zero elements in the fourth vector from the fourth buffer; and the third processing element of the chip being caused to obtain a fifth set of elements in the second vector from the second buffer, where the bitmap of the fourth vector indicates non-zero elements in the fourth vector; and the third processing element of the chip performing an operation on the fourth vector and the second vector according to the fourth set of non-zero elements and the fifth set of elements.

[0048] In a possible implementation, the chip further includes a fourth multiplexer and a fifth multiplexer, and the method further includes: the fourth multiplexer of the chip acquires a fourth group of non-zero elements in a fourth vector from a fourth buffer according to the second selection signal and inputs the third processing element; and the fifth multiplexer of the chip acquires a fifth group of elements in the second vector from the second buffer according to the second selection signal and inputs the third processing element.

[0049] In a possible implementation, the fourth multiplexer and the fifth multiplexer each include K multiplexers; and the fourth buffer includes W rows and K columns of data units, each of which is configured to buffer an element.

[0050] Each multiplexer in the fourth multiplexer is connected with a plurality of data units in the fourth buffer, and each data unit is connected with at least one multiplexer; a connection relationship between an i th multiplexer in the fourth multiplexer and data units in the fourth buffer is the same as a connection relationship between an i th multiplexer in the first multiplexer and data units in the first buffer; and a connection relationship between an i th multiplexer in the fifth multiplexer and data units in the fifth buffer is the same as the connection relationship between the i th multiplexer in the first multiplexer and the data units in the first buffer.

[0051] In a possible implementation, the second scheduling module of the chip generates the second selection signal according to the bitmap of the fourth vector, including: determining, according to the bitmap of the fourth vector, that an element stored in an m th data unit of data units connected with a j th multiplexer of the fourth multiplexer is a non-zero element, and then generating, by the second scheduling module, a selection signal of the j th multiplexer, and sending the selection signal of the j th multiplexer to the j th multiplexer of the fourth multiplexer and the j th multiplexer of the fifth multiplexer; and the second selection signal includes the selection signal of the j th multiplexer of the fourth multiplexer.

[0052] In a possible implementation, the fourth multiplexer of the chip obtains the fourth set of non-zero elements in the fourth vector from the fourth register according to the second selection signal and inputs the third processing element; the fifth multiplexer of the chip obtains the fifth set of elements in the second vector from the second register according to the second selection signal and inputs the third processing element, including: the fourth multiplexer of the chip obtains the fourth element in the mth data unit in the data unit connected to the jth multiplexer of the fourth multiplexer through the jth multiplexer of the fourth multiplexer according to the selection signal of the jth multiplexer of the fourth multiplexer, and inputs the fourth element to the third processing element; the fifth multiplexer of the chip obtains the fifth element in the mth data unit in the data unit connected to the jth multiplexer of the fifth multiplexer through the jth multiplexer of the fifth multiplexer according to the selection signal of the jth multiplexer of the fourth multiplexer, and inputs the fifth element to the third processing element, wherein the fourth element is one of the fourth set of non-zero elements, and the fifth element is one of the fifth set of elements.

[0053] In a third aspect, the present application provides a matrix calculation device, including a first scheduling unit and a first processing unit, wherein the first scheduling unit is configured to generate a first selection signal according to a bitmap of a first vector, the first selection signal being capable of enabling the first processing unit to obtain a first set of non-zero elements in the first vector from a first register and to obtain a second set of elements in a second vector from a second register, and the first processing unit is configured to implement an operation between the first vector and the second vector according to the first set of non-zero elements in the first vector and the second set of elements in the second vector, wherein the bitmap of the first vector indicates non-zero elements in the first vector.

[0054] In a possible implementation, after the first processing unit implements the operation between the first vector and the second vector according to the first set of non-zero elements and the second set of elements, the first scheduling unit generates an erase signal, the erase signal indicating the first register and the second register to erase the data currently stored in the respective registers.

[0055] In a possible implementation, the first vector belongs to part or all elements of any row in a first matrix, and the second vector belongs to part or all elements of any column in a second matrix.

[0056] In a possible implementation, the matrix calculation device further includes a second processing unit, and the first selection signal is further capable of enabling the second processing unit to obtain a third set of elements in a third vector; the second processing unit is configured to implement an operation between the first vector and the third vector according to the first set of non-zero elements and the third set of elements; and the third vector belongs to a column in the second matrix other than the column in which the second vector is located.

[0057] In a possible implementation, the matrix calculation apparatus further includes a second scheduling unit and a third processing unit. The second scheduling unit is configured to generate a second selection signal according to a bitmap of the fourth vector, the second selection signal being used to enable the third processing unit to obtain a fourth set of non-zero elements in the fourth vector; and enable the third processing unit to obtain a fifth set of elements in the second vector, wherein the bitmap of the fourth vector indicates non-zero elements in the fourth vector, and the fourth vector belongs to a row in the first matrix other than the row in which the first vector is located. The third processing unit is configured to implement the operation of the fourth vector and the second vector according to the fourth set of non-zero elements and the fifth set of elements.

[0058] In a fourth aspect, the present application provides a computing device, including a chip and a memory, the memory being configured to store a code, and the chip being configured to execute the code to implement the method in the second aspect and any possible implementation of the second aspect.

[0059] In a fifth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing instructions, when the instructions are executed on a computing device, causing the computing device to execute the method in the second aspect and any possible implementation of the second aspect.

[0060] In a sixth aspect, a computer program product is provided, including a computer program or instructions, when the computer program or instructions are executed on a computing device, causing the computing device to execute the method in the second aspect and any possible implementation of the second aspect.

[0061] On the basis of the implementation manners of the aspects provided in the present application, further combinations can be made to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0063] Figure 1 A structural schematic diagram of a chip provided by an embodiment of the present application is shown in the following figure.

[0064] Figure 2 A schematic diagram of a processing element provided by an embodiment of the present application is shown in the following figure.

[0065] Figure 3 A schematic diagram of a buffer and a multiplexer provided by an embodiment of the present application is shown in the following figure.

[0066] Figure 4A schematic diagram of a multiplexer and a buffer connection provided by an embodiment of the present application;

[0067] Figure 5 A schematic diagram of data unit priority provided by an embodiment of the present application;

[0068] Figure 6 A schematic diagram of data in a buffer and corresponding bitmap provided by an embodiment of the present application;

[0069] Figure 7 A schematic diagram of implementing vector and vector operation provided by an embodiment of the present application;

[0070] Figure 8 A schematic diagram of another chip structure provided by an embodiment of the present application;

[0071] Figure 9 A schematic diagram of implementing vector and matrix operation provided by an embodiment of the present application;

[0072] Figure 10 A schematic diagram of another chip structure provided by an embodiment of the present application;

[0073] Figure 11 A schematic diagram of implementing matrix and matrix operation provided by an embodiment of the present application;

[0074] Figure 12 A schematic diagram of a matrix calculation method provided by an embodiment of the present application;

[0075] Figure 13 A schematic diagram of a matrix calculation device provided by an embodiment of the present application;

[0076] Figure 14 A schematic diagram of a computing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0077] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0078] In the current scenario involving matrix multiplication operation, in order to improve the calculation efficiency, pruning technology is usually used to sparsify the matrix to reduce the calculation amount and improve the calculation efficiency. For example, after AI training is completed, the trained weight matrix is structured pruned to complete the sparsification of the weight matrix, and the structured pruned weight matrix is used for inference in AI inference. However, the above method is only suitable for pruning the matrix after determining the matrix participating in the calculation, such as the AI inference scenario, but is not suitable for other scenarios where the sparsity of the matrix dynamically changes. For example, during AI training, the weight matrix, the gradient matrix or the activation matrix dynamically changes, and the sparsity of the matrix also dynamically changes. If the sparsification of the matrix is to be realized during the AI training, a large time cost is required, which will offset the acceleration effect brought by the sparsification of the matrix through pruning. Therefore, how to realize the acceleration of matrix calculation under the condition of dynamic change of matrix sparsity is a technical problem to be solved.

[0079] The present application provides a chip capable of realizing the operation of a matrix with dynamically changing sparsity. As shown in Figure 1 Figure 1 is a structural schematic diagram of a chip provided by an embodiment of the present application. The chip 100 includes processing elements (PE) 110, a first buffer 120, a first multiplexer (MUX) 130, a second buffer 140, a second multiplexer 150, and a scheduling module 160. The input end of the first multiplexer 130 is connected with the first buffer 120, and the output end of the first multiplexer 130 is connected with the processing element 110. The input end of the second multiplexer 150 is connected with the second buffer 140, and the output end of the second multiplexer 150 is connected with the processing element 110. The control ends of the first multiplexer 130 and the second multiplexer 150 are connected with the scheduling module 160, respectively.

[0080] When the operation of the first vector and the second vector is needed, the first buffer 120 is used to buffer all or part of the elements in the first vector, and the second buffer 140 is used to buffer all or part of the elements in the second vector. The scheduling module 160 is used to generate a selection signal according to the data in the first buffer 120, and send the selection signal to the first multiplexer 130 and the second multiplexer 150. The selection signal is used to make the first multiplexer 130 obtain a group of non-zero elements from the connected first buffer 120, and make the second multiplexer obtain a group of elements from the connected second buffer 140. The first multiplexer 130 and the second multiplexer 150 input the group of data obtained from the corresponding buffer to the processing element 110, so that the processing element 110 performs the operation between the first vector and the second vector. ​

[0081] The chip 100 includes various parts, which will be described in detail below with reference to the accompanying drawings. As shown in Figure 2 , Figure 2 is a schematic diagram of a processing element provided by an embodiment of the present application. The processing element 110 is a point product processing module with a length of K, including K multiply-accumulate (MAC) units and K-1 adders (or addition trees); a processing element 110 can obtain K elements from the first buffer 120 in one cycle, and can obtain K elements from the second buffer 140 in one cycle, that is, a processing element 110 can obtain K pairs of values in one cycle, perform K multiplication and K-1 addition operations, and implement the operation of a1b1+a2b2+…+a K b K . Wherein a1, a2, a3, …, a K are elements in the first vector, and b1, b2, b3, …, b K are elements in the second vector.

[0082] As shown in Figure 3 , Figure 3 is a schematic diagram of a buffer and a multiplexer provided by an embodiment of the present application. Each buffer includes W rows and K columns of data units, and each data unit is used to buffer a value of an element, Figure 3 Taking W equal to 4 and K equal to 8 as an example in the above. Each buffer corresponds to K read ports, each read port is implemented as a multiplexer, and the multiplexer corresponding to a buffer includes K multiplexers, that is, the first multiplexer 130 and the second multiplexer 150 in the above Figure 2 each include K multiplexers. Each multiplexer is connected to a plurality of data units in the buffer, and each data unit is connected to at least one multiplexer. Each multiplexer can be used to read data in a data unit connected to the multiplexer from a plurality of data units connected to the multiplexer.

[0083] As shown in Figure 4 , Figure 4 is a schematic diagram of a multiplexer connected to a buffer provided by an embodiment of the present application. A multiplexer includes a plurality of multiplexers, each multiplexer is connected to a plurality of data units in the buffer, and the plurality of data units connected to a multiplexer have different priorities. In the present application, the data units in each buffer are represented as (r, c), as Figure 4As shown in the table, the data unit in the third column of the first row is represented as (1, 3), the data unit in the fifth column of the fourth row is represented as (4, 5), and so on. Here, r is an integer in the range of [1, W], and c is an integer in the range of [1, K]. In the table, Figure 4 For example, the connection relationship between the multiplexers and the buffers is shown in the table, i.e., each multiplexer is connected with eight data units of the buffers. The priority of the eight data units connected by the ith multiplexer is in descending order as follows: (1, i), (2, i), (3, i), (4, i), (2, i+1), (2, mod(i-1, 8)), (3, mod(i-2, 8)), (4, mod(i-3, 8)). Here, mod is a modulo operation.

[0084] For example, Figure 5 As shown in the table, the priority order of the eight data units connected by the first multiplexer (i.e., i=1) and the fourth multiplexer (i.e., i=4) is shown. Here, Figure 5 In the table, the numbers 1-8 in the data units represent the priority, and the smaller the number, the higher the priority. The priority of the eight data units connected by the first multiplexer is in descending order as follows: (1, 1), (2, 1), (3, 1), (4, 1), (2, 2), (2, 8), (3, 7), (4, 6). The priority of the eight data units connected by the fourth multiplexer is in descending order as follows: (1, 4), (2, 4), (3, 4), (4, 4), (2, 5), (2, 3), (3, 2), (4, 1).

[0085] It should be noted that the above Figure 4 The connection of eight data units by each multiplexer shown in the table is only an example and should not be construed as a specific limitation. Each multiplexer can also be connected with more or fewer data units, such as seven data units, or ten, fifteen, or sixteen data units, etc. However, each data unit is connected with at least one multiplexer. The above Figure 4The connection relationship between each multiplexer and the plurality of data units is only an example and should not be construed as a specific limitation. The application does not specifically limit the connection relationship between each multiplexer and the plurality of data units. However, in the same chip, each buffer should include W rows and K columns of data units. A buffer and a multiplexer connected to the buffer are referred to as a cache module. The connection relationship between the multiplexer and the data units in any two cache modules in the same chip is the same, that is, if a chip includes a plurality of cache modules, the connection relationship between the multiplexer and the data units in each cache module is the same. For example, the relative positions of the plurality of data units connected to the first multiplexer in the first multiplexer 130 in the first buffer 120 are the same as the relative positions of the plurality of data units connected to the first multiplexer in the second multiplexer 150 in the second buffer 140. For example, the i-th multiplexer in the first multiplexer 130 is connected to the first buffer 120 (1, i), (2, i), (3, i), (4, i), (2, i+1), (2, mod(i-1, 8)), (3, mod(i-2, 8)), and (4, mod(i-3, 8)) eight data units. The i-th multiplexer in the second multiplexer is also connected to the second buffer (1, i), (2, i), (3, i), (4, i), (2, i+1), (2, mod(i-1, 8)), (3, mod(i-2, 8)), and (4, mod(i-3, 8)) eight data units. In this way, the i-th multiplexer of the first multiplexer and the i-th multiplexer of the second multiplexer can read data from the data units with the same relative positions in the first buffer 120 and the second buffer 140 according to the same selection signal, so that the first multiplexer and the second multiplexer can read the elements for multiplication in the first vector and the second vector according to the same selection signal.

[0086] In the embodiments of the present application, each multiplexer reads a non-zero element from the connected plurality of data units each time. When determining which data unit the multiplexer reads data from, the scheduling module 160 determines whether the element in the data unit with the highest priority is zero according to the priority level, starting from the data unit with the highest priority. If the element in the data unit with the highest priority is not zero, the scheduling module 160 generates a gating signal of the data unit with the highest priority, so that the multiplexer reads the element in the data unit with the highest priority. If the element in the data unit with the highest priority is zero, the scheduling module 160 determines whether the element in the data unit with the second highest priority is zero. If the element in the data unit with the second highest priority is not zero, the scheduling module 160 generates a gating signal of the data unit with the second highest priority, so that the multiplexer reads the element in the data unit with the second highest priority. If the element in the data unit with the second highest priority is zero, the scheduling module 160 determines whether the element in the data unit with the third highest priority is zero. By analogy, the scheduling module 160 determines the element in the data unit with the highest priority in sequence until a data unit with a stored element that is not zero is found, and then generates a gating signal corresponding to the data unit, so that the multiplexer reads the element in the data unit and sends it to the processing element 110, so that the processing element 110 performs dot product operation.

[0087] The above one buffer is connected with K multiplexers, and the scheduling module 160 needs to generate K scheduling signals DS j for the K multiplexers corresponding to one buffer in one period. Each multiplexer corresponds to a gating signal. Wherein, j = 1, 2, …, 8, j is a positive integer. That is, in each period, each multiplexer needs to read an element from the buffer according to the gating signal of the scheduling module 160 and send it to the processing element 110. In one period, the multiplexer can obtain K elements from the connected buffer through the K multiplexers. It should be noted that if the elements in the plurality of data units connected to the multiplexer are all zero, the multiplexer sends the element 0 to the processing element 110.

[0088] In the embodiments of the present application, the scheduling module 160 determines whether the element in each data unit is zero according to the bitmap corresponding to the data stored in the buffer. Specifically, as shown in Figure 6 Figure 6 ​is a schematic diagram of the data in the buffer and the corresponding bitmap. After storing the data, each buffer generates a bitmap according to the data stored in the data unit, and each bit in the bitmap corresponds to a data unit in the buffer. The value of each bit in the bitmap is 0 or 1, indicating whether the element in the corresponding data unit is 0. For example, when a bit in the bitmap is 0, it indicates that the value of the element in the data unit corresponding to the bit is 0; when a bit in the bitmap is 1, it indicates that the value in the data unit corresponding to the bit is not 0; or when a bit in the bitmap is 0, it indicates that the value in the data unit corresponding to the bit is not 0; when a bit in the bitmap is 1, it indicates that the value in the data unit corresponding to the bit is 0. In the embodiments of the present application, the value of a bit in the bitmap is 0 to indicate that the value of the element in the data unit corresponding to the bit is 0, and the value of a bit in the bitmap is 1 to indicate that the value of the element in the data unit corresponding to the bit is not 0.

[0089] When the scheduling module 160 needs to generate a multiplexer selection signal, the scheduling module 160 first determines whether the value of the bit corresponding to the data unit with priority 1 connected to the multiplexer in the bitmap is 0. If the value of the bit corresponding to the data unit with priority 1 in the bitmap is 1, it indicates that the element in the data unit with priority 1 is not 0, and the scheduling module 160 generates a selection signal corresponding to the data unit with priority 1 and sends it to the multiplexer. If the value of the bit corresponding to the data unit with priority 1 in the bitmap is 0, it indicates that the element in the data unit with priority 1 is 0, and the scheduling module 160 determines whether the value of the bit corresponding to the data unit with priority 2 connected to the multiplexer in the bitmap is 0. If the value of the bit corresponding to the data unit with priority 2 in the bitmap is 1, it indicates that the element in the data unit with priority 2 is not 0, and the scheduling module generates a selection signal corresponding to the data unit with priority 2 and sends it to the multiplexer. If the value of the bit corresponding to the data unit with priority 2 in the bitmap is 0, it indicates that the element in the data unit with priority 2 is 0, and the scheduling module 160 determines whether the value of the bit corresponding to the data unit with priority 3 connected to the multiplexer in the bitmap is 0. And so on, which will not be repeated here.

[0090] It should be noted that after the scheduling module 160 controls the multiplexer to obtain a non-zero element from a data unit, the scheduling module 160 needs to set the corresponding position of the element stored in the data unit in the bitmap to 0 to prevent the element in the data unit from being read repeatedly and to avoid the non-zero element in the data unit with a lower priority from not being read and participating in the calculation, resulting in calculation errors.

[0091] The above Figure 2 to Figure 6 The above Figure 1The working principle of each part of the chip 100 shown is described below with reference to the accompanying drawings. The methods for performing vector-to-vector operations, vector-to-matrix operations, and matrix-to-matrix operations using the chip 100 are as follows.

[0092] like Figure 7 As shown, Figure 7 This is a schematic diagram illustrating an implementation of vector operations provided in an embodiment of this application. Figure 1 Taking the implementation of the dot product operation between vector C and vector D by chip 100 as an example, the first buffer 120 and the second buffer 140 in chip 100 each include data units of W rows and K columns, and the first multiplexer 130 and the second multiplexer 150 each include K multiplexers. Vector C is a 1*WK vector, and vector D is a WK*1 vector, that is, vector C includes 1 row and W*K columns of elements, and vector D includes W*K rows and 1 column of elements.

[0093] When performing dot product operations through chip 100, chip 100 loads the elements included in vector C into the data unit of the first buffer 120 and loads the elements included in vector D into the data unit of the second buffer 140. When loading data of vector C into the first buffer 120 and data of vector D into the second buffer 140, the data from column 1 to column K of vector C are sequentially stored in the first row of the first buffer 120, and the data from row 1 to row K of vector D are sequentially stored in the first row of the second buffer 140; the data from column K+1 to column 2K of vector C are sequentially stored in the second row of the first buffer 120, and the data from row K+1 to column 2K of vector D are sequentially stored in the second row of the second buffer 140; and so on, until the data from column (W-1)K to column WK of vector C are sequentially stored in the first row of the first buffer 120, and the data from row (W-1)K to column WK of vector D are sequentially stored in the second row of the second buffer 140.

[0094] After the W*K data elements of vector C are stored in the first buffer 120, the first buffer 120 generates a corresponding bitmap based on the value of the element stored in each data unit. If the value of an element in a data unit is not 0, the corresponding bit in the bitmap for that data unit is set to 1; if the value of an element in a data unit is 0, the corresponding bit in the bitmap for that data unit is set to 0. For example... Figure 6 As shown, Figure 6 The bitmap shown is the bitmap corresponding to the data cached in the first buffer 120. Figure 6 Taking W = 4 and K = 8 as an example, the connection relationship between each multiplexer in the first multiplexer 130 and the data unit in the first buffer 120 is as follows:Figure 4 It should be understood that the corresponding bitmap can also be generated by the second buffer 140 according to the value in each data unit, and embodiments of the present application are not limited in this regard.

[0095] After the first buffer 120 generates the bitmap, the first buffer 120 sends the bitmap to the scheduling module 160. After receiving the bitmap, the scheduling module 160 needs to first generate the strobe signal DS1 of the first multiplexer in the first multiplexer 130. Specifically, the scheduling module 160 determines whether the value of the bit corresponding to the data unit with priority 1 in the bitmap is 0 according to the priority of the plurality of data units connected to the first multiplexer in the first multiplexer 130. If the value of the bit corresponding to the data unit with priority 1 in the bitmap is not 0, the scheduling module 160 generates the strobe signal 000, and sends the strobe signal 000 to the first multiplexer of the first multiplexer 130 and the first multiplexer of the second multiplexer 150. The strobe signal 000 is used to make the first multiplexer of the first multiplexer 130 read the element in the data unit with priority 1 and send it to the processing element 110, and make the first multiplexer of the second multiplexer 150 read the element in the data unit with priority 1 and send it to the processing element 110.

[0096] If the scheduling module 160 determines that the value of the bit corresponding to the data unit with priority 1 in the bitmap is 0, the scheduling module 160 further determines whether the value of the bit corresponding to the data unit with priority 2 in the bitmap is 0. If the value of the bit corresponding to the data unit with priority 2 in the bitmap is not 0, the scheduling module 160 generates the strobe signal 001, and sends the strobe signal 001 to the first multiplexer of the first multiplexer 130 and the first multiplexer of the second multiplexer 150. The strobe signal 001 is used to make the first multiplexer of the first multiplexer 130 read the element in the data unit with priority 2 and send it to the processing element 110, and make the first multiplexer of the second multiplexer 150 read the element in the data unit with priority 2 and send it to the processing element 110.

[0097] If the scheduling module 160 determines that the value of the bit corresponding to the data unit with priority 2 in the bitmap is 0, the scheduling module 160 determines whether the value of the bit corresponding to the data unit with priority 3 in the bitmap is 0. By analogy, the first multiplexer in the first multiplexer 130 reads a data c1 from the first buffer 120 and sends it to the processing element 110, and the first multiplexer in the second multiplexer 150 reads a data d1 from the second buffer 140 and sends it to the processing element 110, so that the processing element 110 performs the operation of c1*d1. It should be noted that if there is data other than 0 in the elements cached by the plurality of data units connected to the first multiplexer in the first multiplexer 130, the value of c1 is not 0, and d1 can be 0 or not 0; if the data cached by the plurality of data units connected to the first multiplexer in the first multiplexer 130 is all 0, the value of c1 is 0, and d1 can be 0 or not 0.

[0098] For the 2nd to Kth multiplexers of the first multiplexer 130, the scheduling module 160 generates corresponding strobe signals DS2-DSK by the same method as described above. K , so that each multiplexer in the first multiplexer 130 and the second multiplexer 150 outputs one data to the processing element 110. In the first cycle, each multiplexer in the first multiplexer 130 and the second multiplexer 150 outputs one data to the processing element 110, so that the processing element 110 completes K times of multiplication operations and K-1 times of addition operations. The K times of multiplication operations and K-1 times of addition operations are: e1=c1d1+c2d2+…+c t d t +…+c K d K , where c t represents the data output by the tth multiplexer in the first multiplexer 130, d t represents the data output by the tth multiplexer in the second multiplexer 150, and t is a positive integer greater than 0 and less than or equal to K.

[0099] It should be noted that after the scheduling module 160 generates a strobe signal and sends it to the first multiplexer 130, the first multiplexer 130 reads a data from a data unit, and the scheduling module 160 sets the value of the bit corresponding to the data unit in the bitmap to 0.

[0100] In the second cycle and in each subsequent cycle, the scheduling module 160 continues to execute the operations performed in the first cycle, causing each multiplexer in the first multiplexer 130 and the second multiplexer 150 to output one data point to the processing element 110. The processing element 110 performs K product operations and K-1 addition operations until all bits in the bitmap are 0. Finally, the values ​​obtained by the processing element 110 after performing K product operations and K-1 addition operations in each cycle are added together, which is the dot product of vector C and vector D.

[0101] Since a buffer contains W*K data units, and a cycle multiplexer can read data from K data units for computation, the dot product operation of vectors C and D can be completed in at most W cycles. If vector C has a certain sparsity, meaning that some elements in vector C have a value of 0, the chip provided above ensures that even if the sparsity of vector C is transformed each time it is input into the buffer, the elements with a value of 0 in vector C will not participate in the calculation. This reduces the amount of computation and improves computational efficiency without reducing the accuracy of the calculation.

[0102] It should be noted that after the scheduling module 160 determines that all values ​​in the bitmap are 0, the scheduling module 160 will generate an erase signal and send it to the first buffer 120 and the second buffer 140 so that the first buffer 120 and the second buffer 140 will erase the currently cached data so that it can be used to cache the next batch of data.

[0103] It should be understood that the aforementioned vector C can be a part of vector X, and vector D can be a part of vector Y. For example, vector X is a 1*Z vector, and vector Y is a Z*1 vector, where Z is greater than W*K. Since a register in chip 100 can only store W*K data at a time, vectors X and Y are split, and at most W*K elements are stored in the register of chip 100 for calculation at each time. The aforementioned vector C can be a row vector of 1 row and W*K columns, and vector D can be any column of a W*K row and T column matrix. The result of the operation between vector C and this matrix is ​​a vector of 1 row and T columns. During the operation between vector C and this matrix, each column of this matrix is ​​cached in the second register 140 to obtain one element of the operation result.

[0104] like Figure 8 As shown, Figure 8 This is a schematic diagram of another chip provided in an embodiment of this application. The chip 200 includes N processing elements PE1 to PE2. N N+1 buffers B0 to B N N+1 multiplexers M0 to M NAnd a scheduling module 210. Here, N is an integer greater than or equal to 2. Buffer B0 is connected to scheduling module 210, which in turn is connected to each multiplexer. There is a one-to-one correspondence between the N+1 buffers and the N+1 multiplexers, with each multiplexer connected to one buffer. Multiplexer M0 is connected to all N processing elements, and data read by multiplexer M0 is synchronously sent to all N processing elements each time. The connection relationship between each multiplexer and its corresponding buffer can be referred to the above. Figure 3 and Figure 4 The corresponding descriptions will not be repeated here.

[0105] In this embodiment of the application, the chip 200 is capable of performing vector and matrix multiplication operations. Figure 8 Taking the multiplication of vector C and matrix B by chip 200 as an example, each buffer in chip 200 includes a data unit with W rows and K columns, and each multiplexer includes K multiplexers. Vector C is a 1*WK vector, and matrix B is a WK*N matrix, meaning vector C includes 1 row and W*K columns, and matrix B includes W*K rows and N columns. It should be understood that matrix B is equivalent to N WK*1 vectors, that is, matrix B is equivalent to N of the aforementioned vector D; the multiplication of vector C and matrix B is equivalent to performing a dot product operation between vector C and each column of matrix B. Therefore, matrix B can be considered as including vectors D1 to D2. N There are N vectors, each corresponding to a column in matrix B.

[0106] like Figure 9 As shown, Figure 9 This is a schematic diagram illustrating a vector and matrix operation implementation provided in an embodiment of this application. Wherein, Figure 9 In vector C and matrix B, the black squares represent non-zero elements. When performing a multiplication operation between vector C and matrix B using chip 200, chip 200 loads the elements of vector C into the data unit of buffer B0, loads the elements of vector D1 corresponding to the first column of matrix B into the data unit of buffer B1, loads the elements of vector D2 corresponding to the second column of matrix B into the data unit of buffer B2, loads the elements of vector D3 corresponding to the third column of matrix B into the data unit of buffer B3, and so on, until the elements of vector D1 corresponding to the Nth column of matrix B are multiplied. N The element is loaded into cache B N The data unit. The method for loading vector C into buffer B0 can be referred to the above. Figure 7 The corresponding embodiment loads vector C into the first buffer 120, and loads vectors D1 to D2 into the first buffer 120. N The method for loading into the corresponding caches can be referred to above. Figure 7The method of loading the vector D into the second buffer 140 in the corresponding embodiment will not be described here again.

[0107] After the WK data of the vector C are stored in the buffer B0, the buffer B0 generates a corresponding bitmap. The method of generating the bitmap by the buffer B0 can refer to the method of generating the bitmap by the first buffer 120, which will not be described here again. After the bitmap is generated, the buffer B0 sends the bitmap to the scheduling module 210. After receiving the bitmap, the scheduling module 210 first generates a gate signal DS1 of the first multiplexer in the multiplexer M0. The method of generating the gate signal DS1 by the scheduling module 210 can refer to the method of generating the gate signal DS1 by the scheduling module 160, which will not be described here again.

[0108] In the embodiment of the application, after the scheduling module 210 generates the gate signal DS1, the scheduling module 210 sends the gate signal DS1 to the first multiplexer in the multiplexer M0-M N K. The first multiplexer in the multiplexer M0 reads one data according to the gate signal DS1 and sends the data to the N processing elements PE0-PE N . The first multiplexers in the multiplexers M1-M N K read one data according to the gate signal DS1 respectively and send the data to the processing elements connected respectively.

[0109] For the second to Kth multiplexers in the multiplexers M0-M N , the scheduling module 160 generates corresponding gate signals DS2-DS K by the same method described above in sequence, so that each multiplexer in the multiplexers M0-M N outputs one data to the processing element 110. After the scheduling module 210 generates the gate signals DS1-DS K in sequence in one period, each processing element acquires K pairs of data and completes K times of multiplication operations and K-1 times of addition operations.

[0110] It should be noted that after the scheduling module 210 generates one gate signal and sends the gate signal to the multiplexer M0, the scheduling module 210 sets the position 0 of the data unit corresponding to the gate signal in the bitmap after the multiplexer M0 reads one data from the data unit.

[0111] In the second period and each subsequent period, the scheduling module 210 continues to perform the operations performed in the first period, so that each multiplexer in the multiplexers M0-M N outputs one data to the processing element 110, and the processing element 110 completes K times of multiplication operations and K-1 times of addition operations, until the values of all bits in the bitmap are 0. For any processing element PE hThe value obtained after K multiplication operations and K-1 addition operations in each cycle is added, and the result is the dot product of vector C and vector D. h , where h is a positive integer greater than or equal to 1 and less than or equal to N.

[0112] The multiplication operation result of vector C and matrix B is a 1*N vector H, where the processing element PE h outputs the value obtained after the processing element PE h performs the dot product operation of vector C and vector D.

[0113] It should be understood that the above vector C can be part of vector X, or part or all elements of a row of matrix E; matrix B can be part of matrix F, for example, vector X is a 1*Z vector, and matrix F is a Z*N vector, where Z is greater than W*K. Since a buffer in chip 200 can only store W*K data at a time, vector X and matrix F are divided, and W*K elements of vector X are stored in buffer B0 of chip 200 at a time, and W*K elements of the hth column of matrix F are stored in buffer B h of chip 200, that is, W*K elements of the 1st column to the Nth column of matrix F are distributed and stored in buffers B0 to B N of chip 200.

[0114] As shown in Figure 10 , another schematic diagram of a chip provided by an embodiment of the present application is shown in Figure 10 . The chip 300 includes M*N processing elements PE, M+N buffers, M*(N+1) multiplexers, and M scheduling modules S1 to S M . The connection relationship of each part included in the above chip 300 is shown in Figure 10 , the M*N processing elements are distributed in matrix form, and the M*N processing elements are distributed in M rows, and each row includes N processing elements. Figure 10 Each row in the structure of chip 300 corresponds to the structure of chip 200 shown in Figure 8 , and chip 300 corresponds to including M chips 200 shown in Figure 8 .

[0115] In the embodiment of the present application, the chip 300 can implement matrix multiplication. Taking Figure 10The chip 300 shown above can perform the multiplication operation of the matrix A and the matrix B. Each buffer in the chip 300 includes W rows and K columns of data units, and each multiplexer includes K multiplexers. The matrix A is an M*WK matrix, and the matrix B is a WK*N matrix, i.e., the matrix A includes M rows and W*K columns of elements, and the matrix B includes W*K rows and N columns of elements. It should be understood that the matrix A is equivalent to M 1*WK vectors, i.e., the matrix A is equivalent to M vectors C described above; therefore, the multiplication operation of the matrix A and the matrix B is equivalent to performing the vector-matrix multiplication operation of each row of the matrix A and the matrix B respectively. Therefore, the matrix A can be regarded as including M vectors C1-C M .

[0116] The chip 200 shown above can perform the multiplication operation of the vector C and the matrix B, and the matrix A is equivalent to M vectors C, and the chip 300 is equivalent to the structure of the chip 200 shown in Figure 8 or Figure 9 . Therefore, as shown in Figure 11 , Figure 11 is a schematic diagram for implementing the matrix-matrix operation according to an embodiment of the present application. The chip 300 can calculate the multiplication operation of M vectors and the matrix B according to the method of calculating the multiplication operation of the vector C and the matrix B by the chip 200, i.e., the chip 300 can calculate the multiplication operation of M vectors (vectors C1-C M ) included in the matrix A and the matrix B, and further obtain the multiplication operation result of the matrix A and the matrix B. The matrix A is an M*WK matrix, the matrix B is a WK*N matrix, the multiplication operation result of the matrix A and the matrix B is an M*N matrix Q, and the 1*N vector output by the multiplication operation of the vector C g and the matrix B by the N processing elements in the gth row of the chip 300 is the gth row of elements in the matrix Q, where g is a positive integer greater than or equal to 1 and less than or equal to M. The process of implementing the vector-matrix multiplication by the buffers, multiplexers, scheduling modules and processing elements in each row of the chip 300 described above can refer to the process of implementing the multiplication operation of the vector C and the matrix B by the chip 200 shown above, which will not be described herein again. Figure 9 The chip 200 shown above can perform the multiplication operation of the vector C and the matrix B, which will not be described herein again.

[0117] It should be noted that each buffer in the buffers B 10 , B 20 , …B g0 , …B M0 in the chip 300 will generate the bitmap corresponding to the data stored in each buffer respectively after obtaining the elements in the matrix A respectively, and send the bitmap to the scheduling module connected to each buffer respectively. For example, the bitmap generated by the buffer B 10 is sent to the scheduling module S1, the bitmap generated by the buffer B Figure 1 is sent to the scheduling module S2, and the bitmap generated by the buffer B Figure 2 is sent to the scheduling module S3.20 generate bits Figure 2 to the scheduling module S2, B g0 generate bits Figure 1 to the scheduling module S g and so on.

[0118] It should be understood that the above-mentioned matrix A can be part of the matrix G; the matrix B can be part of the matrix F, for example, the matrix A is an M*Z vector, and the matrix F is a Z*N vector, wherein Z is greater than W*K. Since a buffer in the chip 300 can only store W*K data at a time, the matrix G and the matrix F are divided, and W*K elements in the gth row of the matrix G are stored in the buffer B g0 in the chip 200, and W*K elements in the hth column of the matrix F are stored in the buffer B 1h in the chip 200.

[0119] Through the above-mentioned chip 300, the element with a value of 0 in the matrix can not participate in the calculation in the process of performing the matrix multiplication operation, so that the calculation amount can be reduced, the calculation efficiency can be improved, and the calculation accuracy can not be reduced.

[0120] The following will be described in combination with the above-mentioned Figure 1 to Figure 11 , a matrix calculation method provided by an embodiment of the present application is introduced, which is applied to a chip capable of realizing matrix calculation, for example, the chip 100 shown in the above-mentioned Figure 1 , wherein the related structure of the chip 100 can refer to the related description of the above-mentioned Figure 1 to Figure 5 , which will not be described here again. As shown in the above-mentioned Figure 12 , Figure 12 is a flowchart of a matrix calculation method provided by an embodiment of the present application, and the matrix calculation method comprises the following S121 to S123.

[0121] S121. The chip buffers a first vector and a second vector.

[0122] Wherein, the chip can be the chip 100 in the above-mentioned Figure 1 . The chip 100 buffers the first vector in the first buffer and buffers the second vector in the second buffer. The method that the chip 100 buffers the first vector in the first buffer can refer to the method that the chip 100 buffers the vector C in the first buffer 120, and the method that the chip 100 buffers the second vector in the second buffer can refer to the method that the chip 100 buffers the vector D in the second buffer 140, which will not be described here again.

[0123] It should be noted that the first vector can be the vector C in the above-mentioned embodiment shown in the above-mentioned Figure 7 , and the second vector can be the vector D in the above-mentioned Figure 7The first vector can be a part of the vector X, and the second vector can be a part of the vector Y. For example, the vector X is a 1*Z vector, and the vector Y is a Z*1 vector, where Z is greater than W*K. Since a buffer in the chip 100 can only store W*K data at a time, the vector X and the vector Y are split, and at most W*K elements of the vector X and the vector Y are stored in the buffer of the chip 100 at a time for calculation. The first vector can also be a row of the first matrix or a part of elements of a row of the first matrix, and the second vector can also be a column of the second matrix or a part of elements of a column of the second matrix. For example, the vector X is a row of the first matrix, and the vector Y is a column of the second matrix.

[0124] S122. The first scheduling module of the chip generates a first strobe signal according to the bit map of the first vector.

[0125] The bit map of the first vector indicates non-zero elements in the first vector. The bit map of the vector can refer to the description of the bit map of the vector in the above Figure 6 corresponding description, which is not described here again. The first strobe signal can enable the first processing element to obtain a first group of non-zero elements in the first vector from the first buffer and enable the first processing element to obtain a second group of elements in the second vector from the second buffer. The second group of elements obtained from the second vector can include non-zero elements or all-zero elements.

[0126] In the embodiment of the application, the chip 100 further includes a first multiplexer and a second multiplexer. The first processing element obtains the first group of non-zero elements of the first vector from the first buffer through the first multiplexer and obtains the second group of non-zero elements of the second vector from the second buffer through the second multiplexer. The first scheduling module can be the scheduling module 160 in the Figure 7 , the first multiplexer can be the first multiplexer 130 in the Figure 7 , and the second multiplexer can be the second multiplexer 150 in the Figure 7 . The first strobe signal is the strobe signal DS1-DS Figure 7 in the corresponding embodiment. K The first scheduling module generates the first strobe signal according to the bit map of the first vector. The process can refer to the process in which the scheduling module 160 generates the strobe signal DS1-DS Figure 7 in the corresponding embodiment, and the process is not described here again. K

[0127] S123. The first processing element of the chip implements an operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements.

[0128] The first processing element can be the processing element 110 in the Figure 8 ​The first group of non-zero elements is K data obtained by K multiplexers of the first multiplexer from the first buffer according to the received strobe signal, and the second group of elements is K data obtained by K multiplexers of the second multiplexer from the second buffer according to the received strobe signal. The method that the first multiplexer and the second multiplexer obtain data in the buffer according to the received strobe signal can refer to the method that the first multiplexer 130 and the second multiplexer 150 obtain data in the buffer according to the received strobe signal, and the method that the first processing element implements the operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements can refer to the method that the processing element 110 implements the operation between the vector C and the vector D, which will not be described herein again.

[0129] In the embodiment of the application, the chip further includes a third buffer and a second processing element, the third buffer is configured to buffer a third vector, the first strobe signal is further configured to enable the second processing element to obtain a third group of elements in the third vector from the third buffer and enable the second processing element to obtain a first group of non-zero elements in the first vector from the first buffer. The second processing element implements an operation between the first vector and a third vector according to the first group of non-zero elements in the first vector and the third group of elements in the third vector. The third vector belongs to a column in the second matrix except the column in which the second vector is located.

[0130] In a possible implementation, the first buffer can be the buffer B0 in Figure 8 , the second buffer can be the buffer B1 in Figure 8 , the first multiplexer can be the multiplexer M0 in Figure 8 , the second multiplexer can be the multiplexer M1 in Figure 8 , the first processing element can be the processing element PE1 in Figure 8 , and the third buffer can be the buffer B2 to B Figure 8 in N , or one or more of the above, the second processing element is a processing element connected with the third buffer, and the third multiplexer is a multiplexer connected with the third buffer. That is, when the chip includes a third ring buffer and a second processing element, the chip can be the chip 200 shown in Figure 8 , and the method that the chip 200 implements the operation between the first vector and the second vector and the operation between the first vector and the third vector can refer to the related description in the above Figure 9 and Figure 10 corresponding embodiments, which will not be described herein again.

[0131] In this embodiment, the chip, in addition to including a first buffer, a second buffer, a first scheduling module, a first processing element, a third buffer, and a second processing element, may further include a fourth buffer, a second scheduling module, and a third processing element. The fourth buffer is used to buffer a fourth vector, wherein the fourth vector belongs to a row in the first matrix other than the row containing the first vector. The second scheduling module generates a second strobe signal based on the bitmap of the fourth vector. The second strobe signal enables the third processing element to retrieve the fourth set of non-zero elements from the fourth vector in the fourth buffer; and enables the third processing element to retrieve the fifth set of elements from the second vector in the second buffer. The second processing element can perform operations on the fourth vector and the second vector based on the fourth set of non-zero elements and the fifth set of elements in the second vector. The bitmap of the fourth vector indicates the non-zero elements in the fourth vector.

[0132] In one possible implementation, the aforementioned first buffer could be Figure 10 Middle Buffer B 10 The second buffer can be Figure 10 Buffer B in 11 The first multiplexer can be Figure 10 Multiplexer M in 10 The second multiplexer can be Figure 10 Multiplexer M in 11 The first processing element can be Figure 10 Processing element PE in 11 The third buffer can be Figure 10 Buffer B in 12 ~B 1N If any one or more of the above are present, then the second processing element is a processing element connected to the third buffer, and the third multiplexer is a multiplexer connected to the third buffer; the second scheduling module can be... Figure 10 The scheduling modules S2 to S3 are in the middle. M If any one or more of these are specified, then the fourth buffer is a buffer connected to the second scheduling module, and the third processing element is... Figure 10 One or more processing elements on the same row as the second scheduling module. That is, when the chip, in addition to including the first buffer, second buffer, first scheduling module, first processing element, third buffer, and second processing element, also includes a fourth buffer, second scheduling module, and third processing element, the chip can be as described above. Figure 10 The chip 300 shown above, and the methods by which chip 300 performs operations on the first vector and the second vector, the first vector and the third vector, and the fourth vector and the second vector, can refer to the above. Figure 11 and Figure 1 to Figure 12 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0133] For the method embodiments described above, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, and those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present application.

[0134] Other reasonable step combinations that those skilled in the art can think of based on the above description are also within the protection scope of the present application. In addition, those skilled in the art should also be familiar with the fact that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present application.

[0135] The above describes the chip and the method for performing matrix calculation according to the chip provided by the embodiments of the present application in detail. Figure 13 The embodiments of the present application provide a matrix calculation device and equipment. As shown in Figure 14 and Figure 13 , the embodiments of the present application provide a matrix calculation device and equipment. As shown in Figure 13 , the embodiments of the present application provide a matrix calculation device and equipment. As shown in Figure 7 is a schematic diagram of a matrix calculation device provided by the embodiments of the present application. The matrix calculation device can be the chip 100, the chip 200 or the chip 300 described above. The matrix calculation device 131 includes a first scheduling unit 132 and a first processing unit 133, wherein the first scheduling unit 132 is configured to generate a first gating signal according to a bitmap of a first vector, the first gating signal being capable of causing the first processing unit 133 to obtain a first group of non-zero elements in the first vector from a first buffer and to obtain a second group of elements in a second vector from a second buffer, and the first processing unit 133 is configured to implement an operation between the first vector and the second vector according to the first group of non-zero elements in the first vector and the second group of elements in the second vector, wherein the bitmap of the first vector indicates the non-zero elements in the first vector. Specifically, the matrix calculation device 131 can be the chip 100 shown in Figure 8 , the first scheduling unit 132 can be configured to implement the functions implemented by the scheduling module 160 described above, and the first processing unit 133 can be configured to implement the functions implemented by the processing element 110 described above, which will not be described here.

[0136] In a possible implementation, after the first processing unit 133 implements the operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements, the matrix calculation device further includes that the first scheduling unit 132 generates an erasing signal, the erasing signal indicating that the first buffer and the second buffer erase the currently cached data.

[0137] In one possible implementation, the first vector belongs to some or all of the elements of any row in the first matrix, and the second vector belongs to some or all of the elements of any column in the second matrix.

[0138] In one possible implementation, the matrix calculation device further includes a second processing unit 134. The first strobe signal can also enable the second processing unit 134 to acquire the third set of elements in the third vector and the first set of non-zero elements in the first vector. The second processing unit 134 performs operations on the first vector and the third vector based on the first set of non-zero elements and the third set of elements. The third vector belongs to a column in the second matrix other than the column containing the second vector. Specifically, when the matrix calculation device includes the second processing unit 134, the matrix calculation device 131 can be as described above. Figure 9 or Figure 9 The chip 200 shown above, the first scheduling unit 132 can be used to implement the above. Figure 9 The functions implemented by the scheduling module 210 can be achieved by the first processing unit 133. Figure 9 The functions implemented by the processing element PE1 are carried out by the second processing unit 134. Figure 10 The functions implemented by the processing elements other than PE1 are not described in detail here.

[0139] In one possible implementation, the matrix calculation device further includes a second scheduling unit 135 and a third processing unit 136. The second scheduling unit 135 generates a second strobe signal based on the bitmap of the fourth vector. The second strobe signal is used to enable the third processing unit 136 to acquire the fourth group of non-zero elements in the fourth vector and to acquire the fifth group of elements in the second vector. The bitmap of the fourth vector indicates the non-zero elements in the fourth vector. The third processing unit 136 is used to perform operations on the fourth vector and the second vector based on the fourth group of non-zero elements and the fifth group of elements.

[0140] Specifically, when the matrix computing device further includes a second processing unit 134, a second scheduling unit 135, and a third processing unit 136, the matrix computing device 131 can be as described above. Figure 11 or Figure 11 The chip 300 shown above, the first scheduling unit 132 can be used to implement the above. Figure 11 The functions implemented by the scheduling module S1 can be achieved by the first processing unit 133. Figure 11 PE processing components 11 The second processing unit 134 is used to implement the functions described above. Figure 11 The first row of the middle section excludes the processing element PE. 11The first scheduling unit 132 can be used to implement the functions of other processing elements. Figure 14 The functions implemented by the scheduling modules other than the scheduling module S1 will not be described in detail here.

[0141] Figure 7 to Figure 12 This is a schematic diagram of a computing device provided in an embodiment of this application. The computing device 141 includes the aforementioned chip 100, chip 200, or chip 300. The computing device 141 also includes a processor 142, a chip 143, a memory 144, and a communication interface 145, wherein the processor 142, the memory 144, and the communication interface 145 communicate via a bus 146.

[0142] Chip 143 can be any one of the above-mentioned chips 100, 200 or 300, and can assist computing device 141 in realizing the various functions implemented by the above-mentioned chips 100, 200 or 300.

[0143] Chip 143 can achieve the above under the scheduling of processor 142. Figure 1 to Figure 12 The operation described in the corresponding embodiment. The processor 142 can have various specific implementations. For example, the processor 142 can be a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processing unit (NPU), or a tensor processing unit (TPU). The processor 142 can also be a single-core processor or a multi-core processor. The processor 142 can be a combination of a CPU and a hardware chip. The aforementioned hardware chip can be an ASIC, a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The processor 142 can also be implemented solely using a logic device with built-in processing logic, such as an FPGA or a digital signal processor (DSP).

[0144] The memory 144 can be a non-volatile memory, for example, a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The memory 144 can also be a volatile memory, which can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, for example, a static RAM (SRAM), a dynamic RAM (DRAM), a synchronous DRAM (SDRAM), a double data rate SDRAM (DDR SDRAM), an enhanced SDRAM (ESDRAM), a synchlink DRAM (SLDRAM), and a direct rambus RAM (DR RAM).

[0145] The memory 144 can be used to store program codes and data, for example, to cache the above-mentioned vectors or matrices, so that the chip 143 can call the program codes stored in the memory 144 to perform the operation steps in the corresponding embodiments. Figure 14 The memory 144 can be used to store program codes and data, for example, to cache the above-mentioned vectors or matrices, so that the chip 143 can call the program codes stored in the memory 144 to perform the operation steps in the corresponding embodiments.

[0146] The communication interface 145 is a wired interface (for example, an Ethernet interface), an internal interface (for example, a Peripheral Component Interconnect express (PCIE) bus interface), a wired interface (for example, an Ethernet interface), or a wireless interface (for example, a cellular network interface or a wireless local area network interface), which is used to communicate with other computing devices or modules.

[0147] The bus 146 is a Peripheral Component Interconnect Express (PCIE) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), or the like. The bus 146 includes an out-of-band bus and a high-speed bus, and the like, and various buses are all marked as the bus 146 for the purpose of clear illustration.

[0148] It should be noted that, Figure 1 to Figure 12 Only one possible implementation of the embodiments of the present application is described herein, and more or less components can be included in the computing device 141 in actual applications, which are not limited herein.

[0149] The embodiments of the present application provide a computer readable storage medium, including: computer instructions stored in the computer readable storage medium; when the computer instructions run on the computing device, the computing device executes the operations in the above Figure 1 to Figure 12 Embodiments corresponding to the operations.

[0150] The embodiments of the present application provide a computer program product containing instructions, including a computer program or instructions, when the computer program or instructions run on the computer, the computing device executes the operations in the above ​ Embodiments corresponding to the operations.

[0151] The above-described embodiments, all or part of which, are implemented by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments, all or part of which, are implemented in the form of a computer program product. The computer program product includes at least one computer instruction. When the computer program instruction is loaded or executed in a computer, the flow or function according to the embodiments of the present application is generated, all or part of which. The computer is a general-purpose computer, a special-purpose computer, a computer network, or other programming devices. The computer instruction is stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instruction is transmitted from one website site, computer, server or data center to another website site, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) way. The computer readable storage medium is any medium that the computer can access or is a data storage node such as a server, data center, etc. containing at least one medium collection. The medium is a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a high-density digital video disc (digital video disc, DVD), or a semiconductor medium.

[0152] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A chip, characterized by The chip comprises a first buffer, a second buffer, a first scheduling module and a first processing element, The first buffer is used for buffering a first vector; The second buffer is used for buffering a second vector; The first scheduling module is used for generating a first strobe signal according to a bitmap of the first vector, the bitmap of the first vector indicating non-zero elements in the first vector; The first processing element is used for obtaining a first group of non-zero elements in the first vector from the first buffer, obtaining a second group of elements in the second vector from the second buffer based on the first strobe signal, and implementing an operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements. The chip further comprises a first multiplexer and a second multiplexer, the first multiplexer is used for obtaining the first group of non-zero elements in the first vector from the first buffer according to the first strobe signal, and inputting the first processing element; The second multiplexer is used for obtaining the second group of elements in the second vector from the second buffer according to the first strobe signal, and inputting the first processing element; The first multiplexer comprises K multiplexers, the first buffer comprises a plurality of data units, each data unit is used for buffering an element; the plurality of data units have different priorities, and the first scheduling module is specifically used for sequentially determining non-zero elements in the first vector according to priorities of the plurality of data units and the bitmap of the first vector.

2. The chip according to claim 1, characterized in that, The first multiplexer comprises K multiplexers, and the second multiplexer comprises K multiplexers; the first buffer and the second buffer each comprise W rows and K columns of data units, each data unit is used for buffering an element; Each multiplexer in the first multiplexer is connected with a plurality of data units in the first buffer, and each data unit is connected with at least one multiplexer; The connection relationship between the i th multiplexer in the second multiplexer and the data units in the second buffer is the same as the connection relationship between the i th multiplexer in the first multiplexer and the data units in the first buffer.

3. The chip of claim 2, wherein The first scheduling module is specifically used for: determining, according to the bitmap of the first vector, that an element stored in the k th data unit of a data unit connected with the j th multiplexer of the first multiplexer is a non-zero element, generating a strobe signal of the j th multiplexer of the first multiplexer, and sending the strobe signal of the j th multiplexer of the first multiplexer to the j th multiplexer of the first multiplexer and the j th multiplexer of the second multiplexer, wherein the first strobe signal comprises the strobe signal of the j th multiplexer of the first multiplexer.

4. The chip of claim 3, wherein The first multiplexer is specifically configured to: according to a gate signal of a jth multiplexer of the first multiplexer, acquire a first element in a kth data unit of data units connected to the jth multiplexer of the first multiplexer through the jth multiplexer of the first multiplexer, and input the first element to the first processing element, the first element being one of the first set of non-zero elements. The second multiplexer is specifically configured to: according to a gate signal of a jth multiplexer of the first multiplexer, acquire a second element in a kth data unit of data units connected to the jth multiplexer of the second multiplexer through the jth multiplexer of the second multiplexer, and input the second element to the first processing element, the second element being one of the second set of elements.

5. The chip according to any one of claims 1 to 4, characterized in that The first scheduling module is further configured to generate an erasing signal after implementing the operation between the first vector and the second vector, the erasing signal being used to instruct the first buffer and the second buffer to erase currently cached data.

6. The chip of claim 3, wherein The first vector belongs to any row in a first matrix, and the second vector belongs to any column in a second matrix.

7. The chip of claim 6, wherein The chip further comprises a third buffer and a second processing element; wherein, The third buffer is configured to cache a third vector, the third vector belonging to a column in the second matrix other than a column where the second vector is located; and the first gate signal is further used to instruct the second processing element to acquire a third set of elements in the third vector from the third buffer. The second processing element is configured to implement an operation between the first vector and the third vector according to the first set of non-zero elements and the third set of elements.

8. The chip of claim 7, wherein, The chip further comprises a third multiplexer configured to acquire the third set of elements in the third vector from the third buffer according to the first gate signal and input the second processing element.

9. The chip of claim 8, wherein The third multiplexer comprises K multiplexers, and the third buffer comprises W rows of K-column data units, each data unit being configured to cache an element. A connection relationship between an ith multiplexer in the third multiplexer and data units in the third buffer is the same as a connection relationship between the ith multiplexer in the first multiplexer and data units in the first buffer.

10. The chip of claim 9, wherein, The third multiplexer is specifically configured to: according to a gate signal of a jth multiplexer of the first multiplexer, acquire a third element in a kth data unit of data units connected to the jth multiplexer of the third multiplexer through the jth multiplexer of the third multiplexer, and input the third element to the second processing element, the third element being one of the third set of elements.

11. The chip according to any of claims 6 to 10, characterized in that The chip further comprises a fourth buffer, a second scheduling module and a third processing element; wherein, The fourth buffer is configured to cache a fourth vector, the fourth vector belonging to a row in the first matrix other than a row where the first vector is located. The second scheduling module is configured to generate a second selection signal according to a bitmap of the fourth vector, the second selection signal being used to enable the third processing element to obtain a fourth group of non-zero elements in the fourth vector from the fourth cache, and to enable the third processing element to obtain a fifth group of elements in the second vector from the second cache, the bitmap of the fourth vector indicating non-zero elements in the fourth vector. The third processing element is configured to implement an operation between the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

12. The chip of claim 11, wherein, The chip further comprises a fourth multiplexer and a fifth multiplexer, The fourth multiplexer is configured to obtain the fourth group of non-zero elements in the fourth vector from the fourth cache according to the second selection signal and input the third processing element. The fifth multiplexer is configured to obtain the fifth group of elements in the second vector from the second cache according to the second selection signal and input the third processing element.

13. The chip of claim 12, wherein, The fourth multiplexer comprises K multiplexers, and the fifth multiplexer comprises K multiplexers; the fourth cache comprises W rows and K columns of data units, and each data unit is configured to cache an element. Each multiplexer in the fourth multiplexer is connected to a plurality of data units in the fourth cache, and each data unit is connected to at least one multiplexer; the connection relationship between the i th multiplexer in the fourth multiplexer and the data units in the fourth cache is the same as the connection relationship between the i th multiplexer in the first multiplexer and the data units in the first cache. The connection relationship between the i th multiplexer in the fifth multiplexer and the data units in the second cache is the same as the connection relationship between the i th multiplexer in the first multiplexer and the data units in the first cache.

14. The chip of claim 13, wherein, The second scheduling module is specifically configured to: determine, according to the bitmap of the fourth vector, that an element stored in the m th data unit connected to the j th multiplexer of the fourth multiplexer is a non-zero element, and generate a selection signal of the j th multiplexer of the fourth multiplexer, and send the selection signal of the j th multiplexer of the fourth multiplexer to the j th multiplexer of the fourth multiplexer and the j th multiplexer of the fifth multiplexer, the second selection signal comprising the selection signal of the j th multiplexer of the fourth multiplexer.

15. The chip of claim 14, wherein The fourth multiplexer is specifically configured to obtain, according to the selection signal of the j th multiplexer of the fourth multiplexer, a fourth element in the m th data unit connected to the j th multiplexer of the fourth multiplexer through the j th multiplexer of the fourth multiplexer, and input the fourth element to the third processing element, the fourth element being one of the fourth group of non-zero elements. ​ The fifth multiplexer is specifically configured to: according to a strobe signal of the jth multiplexer of the fourth multiplexer, acquire a fifth element in an mth data unit in data units connected to the jth multiplexer of the fifth multiplexer through the jth multiplexer of the fifth multiplexer, and input the fifth element to the third processing element, the fifth element being one of the fifth group of elements.

16. A method of matrix computation, characterized by, The application is applied to a chip, and the chip comprises: The chip caches a first vector and a second vector, wherein the first vector is cached in a first cache of the chip, and the second vector is cached in a second cache of the chip. A first scheduling module of the chip generates a first strobe signal according to a bitmap of the first vector, and the bitmap of the first vector indicates non-zero elements in the first vector. A first processing element of the chip acquires a first group of non-zero elements in the first vector from the first cache and a second group of elements in the second vector from the second cache based on the first strobe signal, and implements an operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements. A first multiplexer of the chip acquires the first group of non-zero elements in the first vector from the first cache according to the first strobe signal and inputs the first processing element. A second multiplexer of the chip acquires the second group of elements in the second vector from the second cache according to the first strobe signal and inputs the first processing element. The first multiplexer comprises K multiplexers, and the first cache comprises a plurality of data units, each data unit being used for caching an element; the plurality of data units have different priorities, and the first scheduling module generates the first strobe signal according to the bitmap of the first vector specifically comprises sequentially determining the non-zero elements in the first vector according to the priorities of the plurality of data units and the bitmap of the first vector.

17. The method of claim 16, wherein, The first multiplexer comprises K multiplexers, and the second multiplexer comprises K multiplexers; the first cache and the second cache each comprise W rows and K columns of data units, each data unit being used for caching an element; Each multiplexer in the first multiplexer is connected to a plurality of data units in the first cache, and each data unit is connected to at least one multiplexer; The connection relationship between the ith multiplexer in the second multiplexer and the data units in the second cache is the same as the connection relationship between the ith multiplexer in the first multiplexer and the data units in the first cache.

18. The method of claim 17, wherein, The first scheduling module of the chip generates a first strobe signal according to a bitmap of the first vector, comprising: The first scheduling module of the chip determines, according to the bitmap of the first vector, that the kth data unit of the data unit connected to the jth multiplexer of the first multiplexer stores a non-zero element, generates a gate signal of the jth multiplexer, and sends the gate signal of the jth multiplexer to the jth multiplexer of the first multiplexer and the jth multiplexer of the second multiplexer, wherein the first gate signal includes the gate signal of the jth multiplexer of the first multiplexer.

19. The method of claim 18, wherein, The first multiplexer of the chip acquires the first group of non-zero elements in the first vector from the first buffer according to the first gate signal and inputs the first group of non-zero elements into the first processing element, and the second multiplexer of the chip acquires the second group of elements in the second vector from the second buffer according to the first gate signal and inputs the second group of elements into the first processing element, including: The first multiplexer of the chip acquires, according to the gate signal of the jth multiplexer of the first multiplexer, a first element in the kth data unit of the data unit connected to the jth multiplexer of the first multiplexer through the jth multiplexer of the first multiplexer, and inputs the first element into the first processing element, wherein the first element is one of the first group of non-zero elements. The second multiplexer of the chip acquires, according to the gate signal of the jth multiplexer of the first multiplexer, a second element in the kth data unit of the data unit connected to the jth multiplexer of the second multiplexer through the jth multiplexer of the second multiplexer, and inputs the second element into the first processing element, wherein the second element is one of the second group of elements.

20. The method according to any one of claims 16-19, characterized by, The first processing element of the chip further implements an operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements. The first scheduling module of the chip generates an erase signal, and the erase signal instructs the first buffer and the second buffer to erase currently cached data.

21. The method of claim 18, wherein, The first vector belongs to any row in a first matrix, and the second vector belongs to any column in a second matrix.

22. The method of claim 21, wherein, The chip further includes a third buffer and a second processing element, and the method further includes: The chip caches a third vector, the third vector is cached in the third buffer, the third vector belongs to a column in the second matrix except a column in which the second vector is located, and the first gate signal is further used to cause the second processing element to acquire a third group of elements in the third vector from the third buffer. The second processing element of the chip implements an operation between the first vector and the third vector according to the first group of non-zero elements and the third group of elements.

23. The method of claim 22, wherein, The chip further includes a third multiplexer, and the method further includes: The third multiplexer of the chip acquires the third group of elements in the third vector from the third buffer according to the first gate signal and inputs the third group of elements into the second processing element.

24. The method of claim 23, wherein, The third multiplexer comprises K multiplexers, and the third buffer comprises W rows and K columns of data units, each of which is used to buffer an element. The connection relationship between the i th multiplexer in the third multiplexer and the data units in the third buffer is the same as that between the i th multiplexer in the first multiplexer and the data units in the first buffer.

25. The method of claim 24, wherein, The third multiplexer of the chip acquires the third group of elements in the third vector from the third buffer according to the first strobe signal and inputs the second processing element, and the method comprises the following steps: The third multiplexer of the chip acquires the third element in the k th data unit connected to the j th multiplexer of the third multiplexer from the data unit according to the strobe signal of the j th multiplexer of the first multiplexer through the j th multiplexer of the third multiplexer, and inputs the third element into the second processing element, wherein the third element is one of the third group of elements.

26. The method of any one of claims 21-25, wherein, The chip further comprises a fourth buffer, a second scheduling module and a third processing element, and the method further comprises the following steps: The chip buffers a fourth vector, the fourth vector is buffered in the fourth buffer, and the fourth vector belongs to a row in the first matrix except the row where the first vector is located; The second scheduling module of the chip generates a second strobe signal according to the bitmap of the fourth vector, the second strobe signal is used to make the third processing element acquire a fourth group of non-zero elements in the fourth vector from the fourth buffer and acquire a fifth group of elements in the second vector from the second buffer, and the bitmap of the fourth vector indicates the non-zero elements in the fourth vector; The third processing element of the chip implements the operation of the fourth vector and the second vector according to the fourth group of non-zero elements and the fifth group of elements.

27. The method of claim 26, wherein, The chip further comprises a fourth multiplexer and a fifth multiplexer, and the method further comprises the following steps: The fourth multiplexer of the chip acquires the fourth group of non-zero elements in the fourth vector from the fourth buffer according to the second strobe signal and inputs the third processing element; The fifth multiplexer of the chip acquires the fifth group of elements in the second vector from the second buffer according to the second strobe signal and inputs the third processing element.

28. The method of claim 27, wherein, The fourth multiplexer comprises K multiplexers, the fifth multiplexer comprises K multiplexers, the fourth buffer comprises W rows and K columns of data units, and each of the data units is used to buffer an element; Each multiplexer in the fourth multiplexer is connected to a plurality of data units in the fourth buffer, and each data unit is connected to at least one multiplexer; the connection relationship between the i th multiplexer in the fourth multiplexer and the data units in the fourth buffer is the same as that between the i th multiplexer in the first multiplexer and the data units in the first buffer. The connection relationship between the i-th multiplexer in the fifth multiplexer and the data units in the second buffer is the same as the connection relationship between the i-th multiplexer in the first multiplexer and the data units in the first buffer.

29. The method of claim 28, wherein, The second scheduling module of the chip generates a second strobe signal according to the bitmap of the fourth vector, including: The second scheduling module of the chip determines, according to the bitmap of the fourth vector, that the m-th data unit of the data unit connected to the j-th multiplexer of the fourth multiplexer stores a non-zero element, generates a strobe signal of the j-th multiplexer of the fourth multiplexer, and sends the strobe signal of the j-th multiplexer of the fourth multiplexer to the j-th multiplexer of the fourth multiplexer and the j-th multiplexer of the fifth multiplexer. The second strobe signal includes the strobe signal of the j-th multiplexer of the fourth multiplexer.

30. The method of claim 29, wherein, The fourth multiplexer of the chip acquires the fourth group of non-zero elements in the fourth vector from the fourth buffer according to the second strobe signal and inputs the third processing element; and the fifth multiplexer of the chip acquires the fifth group of elements in the second vector from the second buffer according to the second strobe signal and inputs the third processing element, including: The fourth multiplexer of the chip acquires, according to the strobe signal of the j-th multiplexer of the fourth multiplexer, a fourth element in the m-th data unit connected to the j-th multiplexer of the fourth multiplexer through the j-th multiplexer of the fourth multiplexer, and inputs the fourth element to the third processing element. The fourth element is one of the fourth group of non-zero elements. The fifth multiplexer of the chip acquires, according to the strobe signal of the j-th multiplexer of the fourth multiplexer, a fifth element in the m-th data unit connected to the j-th multiplexer of the fifth multiplexer through the j-th multiplexer of the fifth multiplexer, and inputs the fifth element to the third processing element. The fifth element is one of the fifth group of elements.

31. A matrix computation device, characterized by The matrix calculation device includes a first scheduling unit and a first processing unit, wherein The first scheduling unit is configured to generate a first strobe signal according to a bitmap of a first vector, the bitmap of the first vector indicating non-zero elements in the first vector. The first processing unit is configured to acquire a first group of non-zero elements in the first vector from a first buffer and a second group of elements in a second vector from a second buffer based on the first strobe signal, and implement an operation between the first vector and the second vector according to the first group of non-zero elements and the second group of elements. The device further comprises a first multiplexer and a second multiplexer, the first multiplexer is configured to obtain the first set of non-zero elements in the first vector from the first buffer according to the first selection signal and input the first processing unit; the second multiplexer is configured to obtain the second set of elements in the second vector from the second buffer according to the first selection signal and input the first processing unit; The first multiplexer comprises K multiplexers, and the first buffer comprises a plurality of data units, each of which is configured to buffer an element; the plurality of data units have different priorities, and the first scheduling unit is specifically configured to determine the non-zero elements in the first vector in an order according to the priorities of the plurality of data units and the bitmap of the first vector.

32. The apparatus of claim 31, wherein, The first scheduling unit is further configured to generate an erase signal after the first processing unit completes the operation between the first vector and the second vector, the erase signal instructing the first buffer and the second buffer to erase the currently buffered data.

33. The apparatus of claim 31 or 32, wherein, The first vector belongs to any row in a first matrix, and the second vector belongs to any column in a second matrix.

34. The apparatus of claim 33, wherein, The matrix calculation device further comprises a second processing unit, and the first selection signal is further configured to enable the second processing unit to obtain a third set of elements in a third vector, the third vector belonging to a column in the second matrix other than the column where the second vector is located, The second processing unit is configured to implement an operation between the first vector and the third vector according to the first set of non-zero elements and the third set of elements.

35. The apparatus of claim 33, wherein, The matrix calculation device further comprises a second scheduling unit and a third processing unit, The second scheduling unit is configured to generate a second selection signal according to a bitmap of a fourth vector, the second selection signal being configured to enable the third processing unit to obtain a fourth set of non-zero elements in the fourth vector; The third processing unit is enabled to obtain a fifth set of elements in the second vector, the bitmap of the fourth vector indicating the non-zero elements in the fourth vector, and the fourth vector belonging to a row in the first matrix other than the row where the first vector is located; The third processing unit is configured to implement an operation between the fourth vector and the second vector according to the fourth set of non-zero elements and the fifth set of elements.

36. A computing device, comprising: The computing device comprises a chip and a memory, the memory is configured to store code, and the chip is configured to execute the code to implement the method according to any one of claims 16 to 30.

Images