Data processing method, chip and computing device

Abstract

The invention discloses a data processing method, a chip and computing equipment. The chip can automatically detect whether the matrix operation of the two matrixes belongs to the small matrix operation or the large matrix operation without active setting of a user, so that the calculation efficiency of the chip is improved. According to the invention, the chip can dynamically select one computing unit suitable for matrix operation of the two matrixes, so that the chip is prevented from operating various computing units at the same time, and the computing units of different types cannot be in the optimal working frequency at the same moment. Besides, the chip selects the calculation unit with the bit width suitable for the matrix operation of the two matrixes, so that the filling rate of the calculation unit is relatively high, and the calculation efficiency of the processing unit can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method, chip and computing device. Background Art

[0002] The general matrix multiply (GEMM) calculation is a calculation for dense matrices and is widely used in deep learning, for example, modern science and technology such as weather forecasting, oil exploration, and nuclear physics. Deep learning is a method in machine learning based on characterization learning of data. In the fields of high performance computing (HPC) and artificial intelligence (AI), there are a large number of small-scale dense GEMM scenarios. There are many types of computing units in chips that run GEMM. When a user calls a chip to run GEMM, one or more types of computing units need to be selected in advance to run the GEMM operator. The user needs to set up the chip every time it runs GEMM, resulting in relatively low computing efficiency of the chip. Summary of the invention

[0003] In order to solve the above problems, a method for data processing is provided in an embodiment of the present application, and the chip can automatically detect whether the matrix operation of two matrices belongs to a small matrix operation or a large matrix operation, without the need for the user to actively set it, thereby improving the computing efficiency of the chip. In addition, the chip can dynamically select a computing unit suitable for the matrix operation of the two matrices, avoiding the chip from running multiple types of computing units at the same time, so that different types of computing units cannot be at the optimal operating frequency at the same time. In addition, the present application also provides a chip and a computing device corresponding to the data processing method.

[0004] To this end, the following technical solutions are adopted in the embodiments of the present application:

[0005] In a first aspect, an embodiment of the present application provides a method for data processing, which is executed by a chip, and the chip includes a first computing unit and a second computing unit; the method includes: obtaining a first matrix and a second matrix for matrix multiplication to be performed; determining whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation according to the number of rows M of the first matrix, the number of columns N of the second matrix, and the number of columns K of the first matrix and the number of rows K of the second matrix, the amount of calculation of the large matrix operation is greater than the amount of calculation of the small matrix operation, and M, N and K are all positive integers greater than or equal to 1; when the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation, the first matrix and the second matrix are transferred to the first computing unit for processing; when the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation, the first matrix and the second matrix are transferred to the second computing unit for processing.

[0006] In this implementation, the chip can automatically detect whether the matrix operation of the two matrices belongs to a small matrix operation or a large matrix operation based on the number of rows and columns of the two matrices input, without the need for the user to actively set it, thereby improving the computing efficiency of the chip. In addition, the chip can select a computing unit suitable for the matrix operation of the two matrices, avoiding the chip from running multiple types of computing units at the same time, so that different types of computing units cannot be at the optimal operating frequency at the same time.

[0007] In one implementation, the first computing unit is a scalable vector extension (SVE) computing unit, and the second computing unit is a scalable matrix extension (SME) computing unit.

[0008] In this embodiment, the SVE calculation unit is based on a vector calculation engine, which multiplies each element of a row of the first set sequence number of the first matrix with each element of a column of the second set sequence number of the second matrix, and superimposes each product to obtain an element of the first set sequence number row and the second set sequence number column. The chip can use the SVE calculation unit to run small matrix operations. The SME calculation unit is based on a matrix outer product calculation engine, which calculates the outer product of each element of each column of the first matrix with each element of the row of the second matrix with the same sequence number as the column of the first matrix, and superimposes each product of the same row and column to obtain a result matrix. The chip can use the SME calculation unit to run large matrix operations.

[0009] In one embodiment, determining whether the matrix multiplication operation of the first matrix and the second matrix belongs to large matrix operation or small matrix operation according to the number of rows M of the first matrix, the number of columns N of the second matrix, the number of columns K of the first matrix, and the number of rows K of the second matrix includes: obtaining a threshold k from a preset mapping relationship according to M and N, where different values of M and N in the preset mapping relationship correspond to the threshold k; when K >= k, determining that the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation; when K < k, determining that the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation.

[0010] In this embodiment, a lookup table can be pre-stored inside the chip. The lookup table records the mapping relationship between different M and N and the threshold k. After the chip determines the corresponding threshold k according to M and N, it can compare K with the threshold k to determine whether the matrix operation of the two matrices belongs to small matrix operation or large matrix operation, so as to implement a calculation unit in the chip that selects a suitable matrix operation for the two matrices.

[0011] In one embodiment, determining whether the matrix multiplication operation of the first matrix and the second matrix belongs to large matrix operation or small matrix operation according to the number of rows M of the first matrix, the number of columns N of the second matrix, the number of columns K of the first matrix, and the number of rows K of the second matrix includes: calculating the product of M, N, and K; when the product of M, N, and K is greater than or equal to a threshold, determining that the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation; when the product of M, N, and K is less than the threshold, determining that the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation.

[0012] In this embodiment, the chip can determine the matrix volume after the matrix operation of the two matrices according to the product of M, N, and K, and based on the comparison between the matrix volume and the threshold, determine whether the matrix operation of the two matrices belongs to small matrix operation or large matrix operation, so as to implement a calculation unit in the chip that selects a suitable matrix operation for the two matrices.

[0013] Second aspect, an embodiment of the present application provides a chip, including: a first computing unit, a second computing unit, and a processing unit. The processing unit is configured to obtain a first matrix and a second matrix for which matrix multiplication is to be performed; determine whether the matrix multiplication of the first matrix and the second matrix belongs to large matrix operation or small matrix operation according to the number of rows M of the first matrix, the number of columns N of the second matrix, the number of columns K of the first matrix, and the number of rows K of the second matrix. The amount of computation of the large matrix operation is greater than that of the small matrix operation, and M, N, and K are all positive integers greater than or equal to 1; when the matrix multiplication of the first matrix and the second matrix belongs to the small matrix operation, transmit the first matrix and the second matrix to the first computing unit for processing; and when the matrix multiplication of the first matrix and the second matrix belongs to the large matrix operation, transmit the first matrix and the second matrix to the second computing unit for processing.

[0014] In one embodiment, the first computing unit is a Scalable Vector Extension (SVE) computing unit, and the second computing unit is a Scalable Matrix Extension (SME) computing unit.

[0015] In one embodiment, the processing unit is specifically configured to obtain a threshold k from a preset mapping relationship according to M and N. Different values of M and N are set with corresponding thresholds k in the preset mapping relationship; when K >= k, it is determined that the matrix multiplication of the first matrix and the second matrix belongs to the large matrix operation; when K < k, it is determined that the matrix multiplication of the first matrix and the second matrix belongs to the small matrix operation.

[0016] In one embodiment, the processing unit is specifically configured to calculate the product of M, N, and K; when the product of M, N, and K is greater than or equal to a threshold, it is determined that the matrix multiplication of the first matrix and the second matrix belongs to the large matrix operation; when the product of M, N, and K is less than the threshold, it is determined that the matrix multiplication of the first matrix and the second matrix belongs to the small matrix operation.

[0017] Third aspect, an embodiment of the present application provides a computing device, including: at least one memory, at least one chip. The at least one chip is configured to execute instructions stored in the at least one memory, so that the chip executes each possible implementation embodiment of the first aspect.

[0018] Fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program instructions are executed by a chip, the chip executes each possible implementation embodiment of the first aspect.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, characterized in that the computer program product stores instructions, and when the instructions are executed by a computer, the computer implements each possible implementation embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The following is a brief introduction to the drawings required for describing the embodiments or prior art.

[0021] Figure 1 A schematic diagram of a result matrix multiplication;

[0022] Figure 2 A schematic diagram of the structure of a chip provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of an implementation flow of a data processing method provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0026] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0027] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0028] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0029] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0030] like Figure 1 As shown, the result matrix can be obtained by multiplying two matrices. Among them, matrix A is an M×K matrix, and M and K are both positive integers greater than or equal to 1. M represents the number of rows (row) of matrix A, and K represents the number of columns (column) of matrix A. Matrix B is a K×N matrix, and N is a positive integer greater than or equal to 1. K represents the number of rows of matrix B, and N represents the number of columns of matrix B. The number of columns of matrix A is the same as the number of rows of matrix B. After matrix A is multiplied by matrix B, matrix C is obtained. Matrix C is an M×N matrix.

[0031] The GEMM operator is generally run by a chip or a computing device equipped with a chip. The chip can be divided into multiple computing units according to the minimum unit for calculating a GEMM. Each computing unit can calculate a part of a GEMM, or calculate a part of a GEMM in batches. In the new generation of chips, new functions can be added to the computing units. For example, a computing unit with a scalable vector extension (SVE) function can be called an SVE computing unit. For another example, a computing unit with a scalable matrix extension (SME) function can be called an SME computing unit.

[0032] The SVE calculation unit is based on a vector calculation engine, which multiplies each element of a row of a first set sequence number of the first matrix with each element of a column of a second set sequence number of the second matrix, and superimposes each product to obtain an element of the first set sequence number row and the second set sequence number column. In one embodiment, Figure 1 As shown, each element of the i-th row of matrix A is multiplied with each element of the j-th column of matrix B, and each product is superimposed to obtain an element of the i-th row and j-th column. Both i and j are positive integers greater than or equal to 1.

[0033] The SME calculation unit is a matrix-based outer product calculation engine, which calculates the outer product of each element of each column of the first matrix with each element of the row of the second matrix with the same sequence number as the column of the first matrix, and superimposes the products of the same row and column to obtain a result matrix. In one embodiment, GEMM is specifically:

[0034]

[0035] When the SME calculation unit calculates the result matrix, it multiplies the first column of the first matrix by the first row of the second matrix to obtain a×a of the first row and first column, a×b of the first row and second column, c×a of the second row and first column, and c×b of the second row and second column. The SME calculation unit multiplies the second column of the first matrix by the second row of the second matrix to obtain b×c of the first row and first column, b×d of the first row and second column, d×c of the second row and first column, and d×d of the second row and second column. The SME calculation unit superimposes the products of the same row and column to obtain a×a+b×c of the first row and first column, a×b+b×d of the first row and second column, c×a+d×c of the second row and first column, and c×b+d×d of the second row and second column.

[0036] The optimal operating frequency of the SVE computing unit is generally around 2.0 GHz. The optimal operating frequency of the SME computing unit is generally around 1.6 GHz. Since the optimal operating frequencies of the SVE computing unit and the SME computing unit are different, the chip cannot allow the SVE computing unit and the SME computing unit to operate at the optimal frequency at the same time. Moreover, when the SME computing unit runs the GEMM operator, the scale of the GEMM is smaller than the bit width of the SME computing unit, which will reduce the computing efficiency of the SME computing unit.

[0037] In order to solve the defects of existing chips, the embodiments of the present application design a chip, a method for data processing and a computing device. A processing unit with an automatic discrimination function can be added inside the chip. The processing unit can automatically detect whether the matrix operation of two matrices belongs to a small matrix operation or a large matrix operation, without the need for the user to actively set it, thereby improving the computing efficiency of the chip. In addition, the chip can dynamically select a computing unit suitable for the matrix operation of the two matrices, avoiding the chip from running multiple types of computing units at the same time, so that different types of computing units cannot be at the optimal operating frequency at the same time. In addition, the processing unit selects a computing unit with a bit width suitable for the matrix operation of the two matrices, so that the fill rate of the computing unit is relatively high, which can improve the computing efficiency of the processing unit.

[0038] Figure 2 Schematic diagram of the structure of a chip provided in the embodiment of the present application. Figure 2 As shown, the chip 200 includes a plurality of first computing units 210, a plurality of second computing units 220 and a processing unit 230. The processing unit 230 is respectively connected to the plurality of first computing units 210 and the plurality of second computing units 220 to achieve data transmission.

[0039] In the embodiment of the present application, the chip 200 may be a central processing unit (CPU), a graphics processing unit (GPU) or other chips with processing functions. The chip 200 may be divided into a plurality of first computing units 210, a plurality of second computing units 220 and a processing unit 230 according to the execution function.

[0040] The first calculation unit 210 may be an SVE calculation unit, a calculation unit having the function of calculating the inner product of a matrix. The first calculation unit 210 is used to multiply each element of a row of the first matrix with each element of a column of the second matrix, and to superimpose each product. In the embodiment of the present application, after receiving the first matrix and the second matrix, the first calculation unit 210 may multiply each element of a row of the first set sequence number of the first matrix with each element of a column of the second set sequence number of the second matrix, and then superimpose each product to obtain an element of the first set sequence number row and the second set sequence number column. By analogy, the first calculation unit 210 can calculate each element to obtain a result matrix.

[0041] The bit widths of some of the multiple first computing units 210 of the chip 200 are the same. The bit widths of some of the multiple first computing units 210 of the chip 200 are different. For the first computing unit 210, the bit width refers to the number of elements in a column of the second matrix calculated by the first computing unit 210. First computing units 210 with different bit widths can call different GEMM operators. The fill rate of the GEMM operator of the first computing unit 210 is in a fully loaded state, which can improve the computing efficiency of the first computing unit 210.

[0042] The second calculation unit 220 can be an SME calculation unit, a calculation unit having the function of calculating the outer product of a matrix. The second calculation unit 220 is used to calculate the outer product of each element of each column of the first matrix with each element of the row of the second matrix with the same sequence number as the column of the first matrix, and superimpose each product of the same row and column to obtain a result matrix. In the embodiment of the present application, after receiving the first matrix and the second matrix, the second calculation unit 220 can calculate the outer product of each element of a column of the first matrix with each element of the row of the second matrix with the same sequence number as the column of the first matrix, and obtain the product of different rows and columns. By analogy, the second calculation unit 220 can calculate the outer product of each element of the other columns of the first matrix with each element of the row of the second matrix with the same sequence number as the column of the first matrix, and obtain the product of different rows and columns. After the second calculation unit 220 calculates each product of different rows and columns, it superimposes each product of the same row and column to obtain a result matrix.

[0043] The bit widths of some of the multiple second computing units 220 of the chip 200 are the same. The bit widths of some of the multiple second computing units 220 of the chip 200 are the same. For the second computing unit 220, the bit width refers to the larger value between the number of elements in a row of the first matrix and the number of elements in a column of the second matrix calculated by the second computing unit 220. Second computing units 220 with different bit widths can call different GEMM operators. The fill rate of the GEMM operator of the second computing unit 220 is in a fully loaded state, which can improve the computing efficiency of the second computing unit 220.

[0044] The processing unit 230 is used to receive two matrices. The two matrices are respectively a first matrix and a second matrix. After receiving the first matrix and the second matrix, the processing unit 230 can identify the number of rows and columns of the two matrices. Usually, the number of columns of the first matrix is ​​the same as the number of rows of the second matrix. The number of rows and columns of the two matrices identified by the processing unit 230 are three values, namely, the number of rows of the first matrix, the number of columns of the second matrix, and the number of columns of the first matrix (or "the number of rows of the second matrix"). In a possible implementation example, as Figure 1 As shown, the first matrix is ​​matrix A and the second matrix is ​​matrix B. The result matrix C is obtained by multiplying matrix A and matrix B. The processing unit 230 obtains that the number of rows of matrix A is M and the number of columns is K. The processing unit 230 obtains that the number of rows of matrix B is K and the number of columns is N. The number of rows of the first matrix obtained by the processing unit 230 is M, the number of columns of the second matrix is ​​N, and the number of columns of the first matrix is ​​K.

[0045] The processing unit 230 determines whether the matrix multiplication operation of the matrix A and the matrix B is a large matrix operation or a small matrix operation according to the number of rows of the first matrix being M, the number of columns of the second matrix being N, and the number of columns of the first matrix being K. In one case, the matrix multiplication operation of the matrix A and the matrix B is a small matrix operation, and the matrix A and the matrix B are transmitted to the SVE calculation unit for matrix multiplication operation. In another case, the matrix multiplication operation of the matrix A and the matrix B is a large matrix operation, and the matrix A and the matrix B are transmitted to the SME calculation unit for matrix multiplication operation. The amount of operation of the large matrix operation is greater than the amount of operation of the small matrix operation.

[0046] A method for determining whether the matrix multiplication operation of the matrix A and the matrix B is a small matrix operation provided in an embodiment of the present application is: after obtaining the number of rows M of the first matrix and the number of columns N of the second matrix, the processing unit 230 can obtain a lookup table stored in the memory, and compare the number of rows M of the first matrix and the number of columns N of the second matrix in the lookup table to determine the computing unit for running the GEMM operator. In an embodiment of the present application, the designer of the chip 200 scans the number of rows M of the first matrix, the number of columns N of the second matrix, and the number of columns K of the first matrix of two different matrices, and obtains the running time of the GEMM operator using different computing units under this scale through actual simulation operation, and finds the dividing line of the performance threshold k corresponding to the number of rows of the first matrix and the number of columns of the second matrix. The designer of the chip 200 establishes a two-dimensional threshold boundary lookup table based on the number of rows M of the first matrix, the number of columns N of the second matrix, and the threshold k.

[0047] Table 1 Two-dimensional threshold boundary lookup table

[0048]

[0049]

[0050] As shown in Table 1, each of the number of rows M of the first matrix and the number of columns N of the second matrix is ​​mapped to a threshold k. After receiving the first matrix and the second matrix, the processing unit 230 compares the number of rows M of the first matrix and the number of columns N of the second matrix with the lookup table based on the two matrices to determine the corresponding threshold k.

[0051] After the processing unit 230 determines the threshold value k, the number of columns K of the first matrix can be compared with the threshold value k to detect whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation, and select one of the first calculation unit 210 and the second calculation unit 220 to perform GEMM calculation. In one case, when the processing unit 230 determines that the number of columns K of the first matrix is ​​less than the threshold value k, it can be considered that the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation. The processing unit 230 can transfer the first matrix and the second matrix to the first calculation unit 210, so that the first calculation unit 210 performs GEMM calculation. In another case, when the processing unit 230 determines that the number of columns K of the first matrix is ​​greater than or equal to the threshold value k, it can be considered that the matrix multiplication operation of the first matrix and the second matrix belongs to a small matrix operation. The processing unit 230 can transfer the first matrix and the second matrix to the second calculation unit 220, so that the second calculation unit 220 performs GEMM calculation.

[0052] Another method provided in an embodiment of the present application for determining whether the matrix multiplication operation between the matrix A and the matrix B is a large matrix operation or a small matrix operation is:

[0053] , the processing unit 230 can compare the product of the number of rows M of the first matrix, the number of columns N of the second matrix and the number of columns K of the first matrix with the threshold k to detect whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation, and select one of the first calculation unit 210 and the second calculation unit 220 to perform GEMM calculation. In one case, when the processing unit 230 determines that the product of the number of rows M of the first matrix, the number of columns N of the second matrix and the number of columns K of the first matrix is ​​less than the threshold k, it can be considered that the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation. The processing unit 230 can transfer the first matrix and the second matrix to the first calculation unit 210, and let the first calculation unit 210 perform GEMM calculation. In another case, when the processing unit 230 determines that the product of the number of rows M of the first matrix, the number of columns N of the second matrix and the number of columns K of the first matrix is ​​greater than or equal to the threshold k, it can be considered that the matrix multiplication operation of the first matrix and the second matrix belongs to a small matrix operation. The processing unit 230 may transmit the first matrix and the second matrix to the second computing unit 220 , so that the second computing unit 220 performs GEMM calculation.

[0054] The above two implementation methods are only examples of methods for determining whether the matrix multiplication operation of the matrix A and the matrix B is a large matrix operation or a small matrix operation, but are not limited to the above two methods. Other methods that can determine whether the matrix multiplication operation of the matrix A and the matrix B is a large matrix operation or a small matrix operation are also included in the scope of the disclosure of this application.

[0055] In the embodiment of the present application, the processing unit 230 can detect whether the matrix multiplication operation of the first matrix and the second matrix is ​​a large matrix operation or a small matrix operation based on the product of the number of columns K of the first matrix of the two matrices or the number of rows M of the first matrix, the number of columns N of the second matrix and the number of columns K of the first matrix, and select one of the first computing unit 210 and the second computing unit 220 to perform GEMM calculation to avoid the chip 200 running the first computing unit 210 and the second computing unit 220 at the same time, so that the first computing unit 210 and the second computing unit 220 cannot be at the optimal operating frequency at the same time.

[0056] Figure 3 Schematic diagram of the implementation flow of the data processing method provided in the first embodiment of the present application. Figure 3 As shown, the data processing method is generally performed by the processing unit 230 of the chip 200, and the specific implementation process is as follows:

[0057] Step S301: The processing unit 230 obtains a first matrix and a second matrix, wherein the number of columns of the second matrix is ​​the same as the number of rows of the second matrix.

[0058] In step S302 , the processing unit 230 obtains the number of rows M of the first matrix, the number of columns N of the second matrix, and the number of columns K of the first matrix.

[0059] The result matrix calculated by the GEMM operator is obtained by multiplying the first matrix and the second matrix. The first matrix and the second matrix both include the number of columns and the number of rows, and the number of columns of the first matrix is ​​the same as the number of rows of the second matrix. After obtaining the first matrix and the second matrix, the processing unit 230 identifies the number of columns and rows of the first matrix and the second matrix to obtain the number of rows M of the first matrix, the number of columns N of the second matrix, and the number of columns K of the first matrix. The number of rows M of the first matrix is ​​the number of rows of the first matrix. The number of columns N of the second matrix is ​​the number of columns of the second matrix. The number of columns K of the first matrix is ​​the number of columns of the first matrix and the number of rows of the second matrix.

[0060] In step S303 , the processing unit 230 determines a threshold k for matrix operations of the first matrix and the second matrix.

[0061] After the designer of chip 200 scans the number of rows M of the first matrix, the number of columns N of the second matrix, and the number of columns K of the first matrix of the two different matrices, the running time of the GEMM operator using different computing units under this scale is obtained through actual simulation operation, and the dividing line of the threshold k of the performance corresponding to the number of rows of the first matrix and the number of columns of the second matrix is ​​found. The designer of chip 200 establishes a two-dimensional threshold boundary lookup table according to the number of rows M of the first matrix, the number of columns N of the second matrix, and the threshold k. A mapping relationship is established between the number of rows M of each first matrix and the number of columns N of each second matrix and a threshold k. In the embodiment of the present application, after receiving the first matrix and the second matrix, the processing unit 230 can obtain the lookup table stored in the memory, and compare the number of rows M of the first matrix and the number of columns N of the second matrix of the two matrices with the lookup table to determine the corresponding threshold k.

[0062] In step S304, the processing unit 230 detects whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation. In one case, when the processing unit 230 determines that the matrix multiplication operation of the first matrix and the second matrix belongs to a small matrix operation, step S305 is executed. In another case, when the processing unit 230 determines that the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation, step S306 is executed.

[0063] In step S305 , the processing unit 230 transmits the first matrix and the second matrix to the first calculation unit 210 .

[0064] In step S306 , the processing unit 230 transmits the first matrix and the second matrix to the second calculation unit 220 .

[0065] In one implementation, after the processing unit 230 determines the threshold value k, the number of columns K of the first matrix can be compared with the threshold value k to detect whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation, and select one of the first calculation unit 210 and the second calculation unit 220 to perform GEMM calculation. In one case, when the processing unit 230 determines that the number of columns K of the first matrix is ​​less than the threshold value k, it can be considered that the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation. The processing unit 230 can transfer the first matrix and the second matrix to the first calculation unit 210, and let the first calculation unit 210 perform GEMM calculation. The first calculation unit 210 calls the GEMM operator and lets the GEMM operator calculate the first matrix and the second matrix to obtain a result matrix. In another case, when the processing unit 230 determines that the number of columns K of the first matrix is ​​greater than or equal to the threshold value k, it can be considered that the matrix multiplication operation of the first matrix and the second matrix belongs to a small matrix operation. The processing unit 230 can transfer the first matrix and the second matrix to the second calculation unit 220, and let the second calculation unit 220 perform GEMM calculation. The second calculation unit 220 calls the GEMM operator to let the GEMM operator calculate the first matrix and the second matrix to obtain a result matrix.

[0066] Each of the first computing unit 210 and the second computing unit 220 corresponds to an operator library. The operator library stores GEMM operators, allowing the first computing unit 210 and the second computing unit 220 to perform GEMM calculations. The operator library is generally a basic linear algebra subprograms library.

[0067] The BLAS library is a function library that provides some basic linear algebra operations. The BLAS library usually contains two levels. The first level contains basic vector and matrix operations, such as vector addition, matrix multiplication, etc. The second level contains advanced matrix operations, such as LU decomposition, QR decomposition, matrix inversion, etc. The function interface of the BLAS library usually adopts the Fortran format naming convention. The GEMM function in the BLAS library is used to perform matrix multiplication operations. It accepts four parameters, namely matrix A, matrix B, result matrix C, and the sizes of matrix A, matrix B, and matrix C. The GEMM function in the BLAS library is a highly optimized matrix multiplication implementation that can quickly perform matrix multiplication operations on multiple CPU architectures and hardware accelerators.

[0068] In the second embodiment of the present application, the processing unit 230 can compare the product of the number of rows M of the first matrix, the number of columns N of the second matrix and the number of columns K of the first matrix with the threshold value k according to the product of the number of rows M of the first matrix, the number of columns N of the second matrix and the number of columns K of the first matrix, so as to detect whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation, and select one of the first calculation unit 210 and the second calculation unit 220 to perform GEMM calculation. In one case, when the processing unit 230 determines that the product of the number of rows M of the first matrix, the number of columns N of the second matrix and the number of columns K of the first matrix is ​​less than the threshold value k, it can be considered that the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation. The processing unit 230 can transmit the first matrix and the second matrix to the first calculation unit 210, and let the first calculation unit 210 perform GEMM calculation. The first calculation unit 210 calls the GEMM operator, and lets the GEMM operator calculate the first matrix and the second matrix to obtain a result matrix. In another case, when the processing unit 230 determines that the product of the number of rows M of the first matrix, the number of columns N of the second matrix, and the number of columns K of the first matrix is ​​greater than or equal to the threshold k, it can be considered that the matrix multiplication operation of the first matrix and the second matrix belongs to a small matrix operation. The processing unit 230 can transmit the first matrix and the second matrix to the second calculation unit 220, and let the second calculation unit 220 perform GEMM calculation. The second calculation unit 220 calls the GEMM operator and lets the GEMM operator calculate the first matrix and the second matrix to obtain a result matrix.

[0069] Typically, the chip 200 has a plurality of first computing units 210 and second computing units 220 with different bit widths, and can run different GEMM operators. The bit widths of the first computing unit 210 and the second computing unit 220 are related to the number of columns K of the first matrix of the two matrices. In the embodiment of the present application, the processing unit 230 can scan the bit widths of the plurality of first computing units 210 and the bit widths of the plurality of second computing units 220, and pre-store the bit widths of the plurality of first computing units 210 and the bit widths of the plurality of second computing units 220 in the memory.

[0070] When the processing unit 230 determines to transmit the first matrix and the second matrix to the first computing unit 210, the number of columns K of the first matrix can be compared with the bit widths of the plurality of first computing units 210 to determine a first computing unit 210 with a bit width that is less than the number of columns K of the first matrix and close to the number of columns K of the first matrix, so as to obtain a first computing unit 210 with a suitable bit width. The processing unit 230 inputs the first matrix and the second matrix into the first computing unit 210 with a suitable bit width, so that the GEMM operator of the first computing unit 210 is in a fully loaded state as much as possible, so as to improve the computing efficiency of the first computing unit 210.

[0071] Similarly, when the processing unit 230 determines to transmit the first matrix and the second matrix to the second computing unit 220, the number of columns K of the first matrix can be compared with the bit widths of the plurality of second computing units 220 to determine a second computing unit 220 with a bit width that is less than the number of columns K of the first matrix and close to the number of columns K of the first matrix, so as to obtain a second computing unit 220 with a suitable bit width. The processing unit 230 inputs the first matrix and the second matrix into the second computing unit 220 with a suitable bit width, so that the GEMM operator of the second computing unit 220 is in a fully loaded state as much as possible, so as to improve the computing efficiency of the second computing unit 220.

[0072] It should be noted that the processing unit 230 can select a computing unit with a suitable bit width and select the first computing unit or the second computing unit in no particular order. In one case, the processing unit 230 can preferentially select the first computing unit 210 and the second computing unit 220 with a suitable bit width based on the number of columns K of the first matrix. Then, the processing unit 230 detects whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation, and selects one of the first computing unit 210 and the second computing unit 220 to perform GEMM calculation. In another case, the processing unit 230 detects whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation, and selects one of the first computing unit 210 and the second computing unit 220 to perform GEMM calculation. Then, the processing unit 230 can select the first computing unit 210 and the second computing unit 220 with a suitable bit width based on the number of columns K of the first matrix.

[0073] In the embodiment of the present application, the processing unit 230 can automatically detect whether the matrix operation of the two matrices belongs to a small matrix operation or a large matrix operation, and dynamically select a computing unit suitable for the matrix operation of the two matrices, so as to avoid the chip 200 running multiple types of computing units at the same time, so that different types of computing units cannot be at the optimal operating frequency at the same time. In addition, the processing unit 230 selects a computing unit with a bit width suitable for the matrix operation of the two matrices, so that the filling rate of the computing unit is relatively high, which can improve the computing efficiency of the processing unit.

[0074] Figure 4 Schematic diagram of a computing device provided in an embodiment of the present application. Figure 4 As shown, the computing device 400 includes a bus 410, a chip 420, a memory 430, and a communication interface 440. The chip 420, the memory 430, and the communication interface 440 communicate with each other through the bus 410. The computing device 400 can be a server, a computer, a portable notebook, a cabinet, etc. It should be understood that the present application does not limit the number of chips and memories in the computing device 400.

[0075] The bus 410 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus 410 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 410 may include a path for transmitting information between various components of the computing device 400 (for example, the chip 420, the memory 430, and the communication interface 440).

[0076] The chip 420 may be the chip 200 mentioned above.

[0077] The memory 430 may include a volatile memory, such as a random access memory (RAM). The memory 430 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0078] The memory 430 stores executable program codes, such as the program of the processing unit, the program of the first computing unit, and the program of the second computing unit. The program of the first computing unit and the program of the second computing unit are, for example, GEMM operators. The program of the processing unit is, for example, a program with a discrimination function. The chip 420 executes the executable program code to respectively implement the functions of the aforementioned first computing unit 210, the second computing unit 220, and the processing unit 230, thereby implementing the method of data processing. That is, the memory 430 stores instructions for executing the method of data processing.

[0079] The communication interface 440 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 400 and other devices or a communication network.

[0080] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device is caused to perform a data processing method.

[0081] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a high-density digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to perform a method for data processing.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for data processing, characterized in that: The method is executed by a chip, and the chip includes a first computing unit and a second computing unit; the method includes: Obtaining a first matrix and a second matrix for which matrix multiplication is to be performed; Determining whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation according to the number of rows M of the first matrix, the number of columns N of the second matrix, the number of columns K of the first matrix, and the number of rows K of the second matrix, where the amount of computation of the large matrix operation is greater than that of the small matrix operation, and M, N, and K are all positive integers greater than or equal to 1; When the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation, transmitting the first matrix and the second matrix to the first computing unit for processing; When the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation, transmitting the first matrix and the second matrix to the second computing unit for processing.

2. The method according to claim 1, characterized in that The first computing unit is a Scalable Vector Extension (SVE) computing unit, and the second computing unit is a Scalable Matrix Extension (SME) computing unit.

3. The method according to claim 1 or 2, characterized in that: The determining whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation according to the number of rows M of the first matrix, the number of columns N of the second matrix, the number of columns K of the first matrix, and the number of rows K of the second matrix includes: Obtaining a threshold value k from a preset mapping relationship according to M and N, where different threshold values k corresponding to different values of M and N are set in the preset mapping relationship; When K >= k, determining that the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation; When K < k, determining that the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation.

4. The method according to claim 1 or 2, characterized in that: The determining whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation according to the number of rows M of the first matrix, the number of columns N of the second matrix, the number of columns K of the first matrix, and the number of rows K of the second matrix includes: Calculating the product of M, N, and K; When the product of M, N, and K is greater than or equal to a threshold value, determining that the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation; When the product of M, N, and K is less than the threshold value, determining that the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation.

5. A chip, characterized in that: Includes: A first computing unit (210), a second computing unit (220), and a processing unit (230), The processing unit is configured to obtain a first matrix and a second matrix for which matrix multiplication is to be performed; Determine whether the matrix multiplication operation of the first matrix and the second matrix belongs to a large matrix operation or a small matrix operation according to the number of rows M of the first matrix, the number of columns N of the second matrix, the number of columns K of the first matrix, and the number of rows K of the second matrix, where the amount of computation of the large matrix operation is greater than that of the small matrix operation, and M, N, and K are all positive integers greater than or equal to 1; When the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation, the first matrix and the second matrix are transmitted to the first computing unit for processing; and When the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation, the first matrix and the second matrix are transmitted to the second computing unit for processing.

6. The chip according to claim 5, characterized in that: The first computing unit is a Scalable Vector Extension (SVE) computing unit, and the second computing unit is a Scalable Matrix Extension (SME) computing unit.

7. The chip according to claim 5 or 6, characterized in that: The processing unit is specifically configured to obtain a threshold value k from a preset mapping relationship according to M and N, where different threshold values k corresponding to different values of M and N are set in the preset mapping relationship; When K >= k, it is determined that the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation; When K < k, it is determined that the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation.

8. The chip according to claim 5 or 6, characterized in that: The processing unit is specifically configured to calculate the product of M, N, and K; When the product of M, N, and K is greater than or equal to the threshold value, it is determined that the matrix multiplication operation of the first matrix and the second matrix belongs to the large matrix operation; When the product of M, N, and K is less than the threshold value, it is determined that the matrix multiplication operation of the first matrix and the second matrix belongs to the small matrix operation.

9. A computing device, characterized in that Comprising: at least one memory, at least one chip, where the at least one chip is configured to execute instructions stored in the at least one memory, so that the chip executes the method according to any one of claims 1-4.

10. A computer-readable storage medium, characterized in that: Comprising computer program instructions, when the computer program instructions are executed by a chip, the chip executes the method according to any one of claims 1-4.

Images