A multi-threaded parallel computing system for matrix calculations

Abstract

The present application relates to the field of GPU parallel processing technology, and in particular to a multi-threaded parallel computing system for matrix calculation. The system includes: a GPU, a processor and a memory storing a computer program. When the computer program is executed by the processor, the following steps are implemented: each second vector in the second matrix B is directly sent to the corresponding computing submodule, and the row vector in the first matrix A is distributed to the computing submodule, so as to form a first vector in the computing submodule. After the first vector and the second vector are calculated, they are superimposed to directly obtain the matrix multiplication result. Compared with the existing method, there is no need to store the matrix to be calculated in the local storage of the parallel computing module, and there is no need to read the matrix elements from the local storage, which greatly reduces the data preparation time of the computing unit, thereby improving the efficiency of matrix calculation.

Description

Technical Field

[0001] The present invention relates to the field of GPU parallel processing technology, and in particular to a multi-threaded parallel computing system for matrix computing. Background Art

[0002] Since GPU has more Arithmetic And Logic Unit (ALU) for data processing, it is suitable for parallel processing of dense data and is good at large-scale concurrent computing, such as multiplication calculations between large-size matrices. Usually, GPU includes multiple parallel computing modules, and accordingly, the multiplication calculations between large-size matrices are also split into multiplication calculations between multiple groups of smaller-size matrices, so that each parallel computing module performs multiplication calculations between a group of smaller-size matrices, thereby improving the efficiency of matrix calculations.

[0003] In the matrix multiplication calculation scenario, to calculate an element in the matrix multiplication result, it is necessary to multiply and add the elements at the corresponding positions in a row of a matrix to be calculated and a column of another matrix. Obviously, the calculation of an element in the multiplication result requires reading the element multiple times. In the existing method, after reading an element in a row of a matrix, the element is multiplied with the corresponding row in the other matrix, and then superimposed with other multiplication results to obtain a row of elements in the matrix multiplication result, thereby reducing the number of times the elements are read, saving the time of data reading, and improving the efficiency of matrix calculation.

[0004] However, in the existing method, each time each element in a row is read, each row of another matrix needs to be read accordingly, and the reading efficiency is still low, resulting in poor efficiency of matrix calculation. Therefore, how to improve the efficiency of matrix calculation has become an urgent problem to be solved. Summary of the invention

[0005] In view of the above technical problems, the technical solution adopted by the present invention is:

[0006] A multithreaded parallel computing system for matrix computing, the system comprising: a GPU, a processor and a memory storing a computer program, wherein the GPU comprises a parallel computing module, the parallel computing module comprises J computing submodules, one computing submodule corresponds to a computing unit and a storage unit, the memory comprises a first matrix A and a second matrix B to be executed for multiplication computing, A and B are stored in row order, the size corresponding to A is I×J, and the size corresponding to B is J×K, when the computer program is executed by the processor, the following steps are implemented:

[0007] S101, sending the j-th row vector of the second matrix B as the j-th second vector to the computing unit of the j-th computing submodule, where j is an integer in the range of [1, J].

[0008] S102, initialize p=1.

[0009] S103: At the pth preset time point, J elements in the pth row of the first matrix A are respectively sent to the storage units of J computing submodules.

[0010] S104, update p=p+1, return to execute step S103, until p=I+1, execute step S105.

[0011] S105, forming a j-th first vector using I elements in the storage unit corresponding to the j-th computing submodule.

[0012] S106, multiply the j-th first vector by the j-th second vector to obtain the j-th sub-matrix.

[0013] S107: Add all sub-matrices to obtain a multiplication result of the first matrix A and the second matrix B.

[0014] Compared with the prior art, the present invention has obvious beneficial effects. By means of the above technical solution, a storage access optimization system for parallel computing provided by the present invention can achieve considerable technical advancement and practicality, and has wide industrial utilization value, and has at least the following beneficial effects:

[0015] The present invention provides a multi-threaded parallel computing system for matrix computing, the system comprising: a GPU, a processor and a memory storing a computer program, wherein the GPU comprises a parallel computing module, the parallel computing module comprises J computing submodules, one computing submodule corresponds to a computing unit and a storage unit, the memory comprises a first matrix A and a second matrix B to be executed for multiplication computing, A and B are stored in row order, the size corresponding to A is I×J, and the size corresponding to B is J×K, when the computer program is executed by the processor, the following steps are implemented: S101, sending the j-th row vector of the second matrix B as the j-th second vector to the j-th computing submodule The computing unit of the block, j is an integer in the range of [1, J], S102, initialize p=1, S103, at the p-th preset time point, send the J elements in the p-th row of the first matrix A to the storage units of the J computing submodules respectively, S104, update p=p+1, return to execute step S103, until p=I+1, execute step S105, S105, form the j-th first vector by the I elements in the storage unit corresponding to the j-th computing submodule, S106, multiply the j-th first vector by the j-th second vector to obtain the j-th submatrix, S107, add all the submatrices to obtain the multiplication result of the first matrix A and the second matrix B.

[0016] It can be seen that each second vector in the second matrix B does not need to be used multiple times, but is directly sent to the corresponding calculation submodule, and the row vectors in the first matrix A are distributed to the calculation submodule, so as to form the first vector in the calculation submodule. After the first vector and the second vector are calculated, they are superimposed to directly obtain the matrix multiplication result. Compared with the existing method, there is no need to store the matrix to be calculated in the local storage of the parallel computing module, nor is there any need to read the matrix elements from the local storage, which greatly reduces the data preparation time of the computing unit, thereby improving the efficiency of matrix calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flowchart of a computer program executed by a processor in a multi-threaded parallel computing system for matrix computing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] This embodiment provides a multi-threaded parallel computing system for matrix computing, the system comprising: a GPU, a processor and a memory storing a computer program, wherein the GPU comprises a parallel computing module, the parallel computing module comprises J computing submodules, one computing submodule corresponds to a computing unit and a storage unit, the memory comprises a first matrix A and a second matrix B to be multiplied, A and B are stored in row order, the size corresponding to A is I×J, the size corresponding to B is J×K, see Figure 1 , is a flow chart of a computer program executed by a processor in a multi-threaded parallel computing system for matrix computing provided by an embodiment of the present invention. When the computer program is executed by the processor, the following steps are implemented:

[0021] S101, sending the j-th row vector of the second matrix B as the j-th second vector to the computing unit of the j-th computing submodule, where j is an integer in the range of [1, J];

[0022] S102, initialization p=1;

[0023] S103, at the pth preset time point, sending J elements in the pth row of the first matrix A to the storage units of J computing submodules respectively;

[0024] S104, update p=p+1, return to step S103, until p=I+1, execute step S105;

[0025] S105, forming a j-th first vector using I elements in the storage unit corresponding to the j-th computing submodule;

[0026] S106, multiplying the j-th first vector by the j-th second vector to obtain the j-th sub-matrix;

[0027] S107: Add all sub-matrices to obtain a multiplication result of the first matrix A and the second matrix B.

[0028] Among them, the first matrix A and the second matrix B can refer to a group of smaller matrices respectively cut out from the two large-size matrices to be calculated. According to the calculation formula of matrix multiplication, it can be known that the matrix multiplication results corresponding to multiple groups of smaller matrices cut out from the two large-size matrices to be calculated can be spliced ​​into the multiplication results of the two large-size matrices. Therefore, the sizes corresponding to the first matrix A and the second matrix B should enable the first matrix A and the second matrix B to perform matrix multiplication calculation. In this embodiment, the size corresponding to A is expressed as I×J, and the size corresponding to B is expressed as J×K.

[0029] It should be noted that the number of computing sub-modules included in the parallel computing module can be flexibly adjusted. In this embodiment, in order to adapt to the sizes of A and B, the number of computing sub-modules is adjusted to J.

[0030] Specifically, in this embodiment, the jth second vector is only used in the calculation unit of the jth calculation submodule, so the jth second vector is directly sent to the calculation unit of the jth calculation submodule. Compared with the prior art, there is no need to repeatedly read the elements or rows in B, so there is no need to store B in the local storage (LocalMemory) corresponding to the parallel calculation module. Similarly, in this embodiment, the elements in the first matrix A are also only used once, so there is no need to store A in the local storage corresponding to the parallel calculation module. After reading the row vector in A, the elements in the row vector are directly distributed to the storage units of different calculation submodules. It can be seen that compared with the prior art, the calculation is performed in units of row vectors, and the present embodiment is calculated in units of matrices. By converting the form of the parallel calculation module when performing matrix calculation, the time-consuming transfer of the matrix from the external storage to the local storage is saved, and the time-consuming reading of data from the local storage is also saved, thereby improving the efficiency of matrix calculation.

[0031] In a specific implementation, the sending of J elements in the p-th row of the first matrix A to storage units of J computing submodules respectively includes:

[0032] The elements corresponding to the p-th row and j-th column of the first matrix A are sent to the storage unit of the j-th computing submodule.

[0033] Among them, since the elements corresponding to the p-th row and j-th column in the first matrix A will only be operated with the vectors in the j-th row in the second matrix, and the vectors in the j-th row in the second matrix are located in the computing unit of the j-th computing sub-module, when distributing the elements of the p-th row of the first matrix A, the elements corresponding to the p-th row and j-th column of the first matrix A are sent to the storage unit of the j-th computing sub-module.

[0034] In a specific implementation, the j-th first vector is formed by I elements in the storage unit corresponding to the j-th computing submodule, including:

[0035] The j-th first vector is obtained by concatenating the I elements in the storage unit corresponding to the j-th calculation submodule in the sending order.

[0036] Among them, the earlier the sending order is, the smaller the corresponding number of rows is.

[0037] In a specific implementation, the size corresponding to the first vector is I×1.

[0038] Among them, the first vector is a column vector.

[0039] In a specific implementation, the size of the second vector is 1×K.

[0040] Among them, the second vector is a row vector.

[0041] In a specific implementation, the size of the submatrix is ​​I×K.

[0042] Among them, each element in the first vector is multiplied by the second vector to obtain I row vectors, and the number of columns corresponding to each row vector is K, so the corresponding size of the submatrix is ​​I×K.

[0043] In a specific implementation, the time interval between two adjacent preset time points is a preset period.

[0044] The preset period may refer to a machine period, a clock period, etc., and the preset period may be set by the implementer.

[0045] In a specific implementation, a single computing unit includes I×K arithmetic logic units.

[0046] The arithmetic logic unit can be represented as ALU, which is a component responsible for calculation. When a single calculation unit includes I×K arithmetic logic units, only one round is needed to complete the calculation of the sub-matrix.

[0047] Specifically, considering the limitation on the number of operation logic units, I can also be set to a smaller value, for example, to 4. Accordingly, the large-size matrix will include multiple first matrices of size I×K. Furthermore, after the calculation of the kth column in the previous first matrix is ​​completed, the calculation of the kth column of the next first matrix can be continued, thereby improving the efficiency of matrix calculation.

[0048] It should be noted that the implementer can divide the number of operation logic units contained in the computing unit according to the actual hardware situation, which has no effect on the efficiency of matrix calculation. When the number of operation logic units contained in the parallel computing module is fixed, since the number of element multiplications and additions of the matrix calculation is also fixed, the time required to calculate the final result is also fixed.

[0049] In this embodiment, each second vector in the second matrix B does not need to be used multiple times, but is directly sent to the corresponding calculation sub-module, and the row vectors in the first matrix A are distributed to the calculation sub-module, so as to form the first vector in the calculation sub-module. After the first vector and the second vector are calculated, they are superimposed to directly obtain the matrix multiplication result. Compared with the existing method, there is no need to store the matrix to be calculated in the local storage of the parallel computing module, nor is there any need to read the matrix elements from the local storage, which greatly reduces the data preparation time of the computing unit, thereby improving the efficiency of matrix calculation.

[0050] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A multi-threaded parallel computing system for matrix computing, characterized in that: The system includes: a GPU, a processor, and a memory storing a computer program, wherein the GPU includes a parallel computing module, the parallel computing module includes J computing submodules, one computing submodule corresponds to a computing unit and a storage unit, the memory includes a first matrix A and a second matrix B to be executed for multiplication, A and B are stored in row order, the size corresponding to A is I×J, and the size corresponding to B is J×K, and when the computer program is executed by the processor, the following steps are implemented: S101, sending the j-th row vector of the second matrix B as the j-th second vector to the computing unit of the j-th computing submodule, where j is an integer in the range of [1, J]; S102, initialization p=1; S103, at the p-th preset time point, sending J elements in the p-th row of the first matrix A to storage units of J computing submodules respectively, wherein sending the J elements in the p-th row of the first matrix A to storage units of J computing submodules respectively includes: Sending the element corresponding to the p-th row and the j-th column of the first matrix A to the storage unit of the j-th computing submodule; S104, update p=p+1, return to step S103, until p=I+1, execute step S105; S105, forming a j-th first vector from I elements in the storage unit corresponding to the j-th computing submodule, wherein forming the j-th first vector from I elements in the storage unit corresponding to the j-th computing submodule includes: The j-th first vector is obtained by concatenating the I elements in the storage unit corresponding to the j-th calculation submodule in the order of transmission; S106, multiplying the j-th first vector by the j-th second vector to obtain the j-th sub-matrix; S107: Add all sub-matrices to obtain a multiplication result of the first matrix A and the second matrix B.

2. The multi-threaded parallel computing system for matrix computing according to claim 1, characterized in that: The size of the first vector is I×1.

3. The multi-threaded parallel computing system for matrix computing according to claim 1, characterized in that: The size of the second vector is 1×K.

4. The multi-threaded parallel computing system for matrix computing according to claim 1, characterized in that: The size of the submatrix is ​​I×K.

5. The multi-threaded parallel computing system for matrix computing according to claim 1, characterized in that: The time interval between two adjacent preset time points is a preset period.

6. The multi-threaded parallel computing system for matrix computing according to claim 1, characterized in that: A single computing unit includes I×K arithmetic logic units.

Images