A matrix parallel computing system based on local storage
By distributing the row vectors of the first matrix to the computation submodules and broadcasting the row vectors of the second matrix in the matrix parallel computing system, the problem of low efficiency in matrix parallel computing is solved, and more efficient computing is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- METAX INTEGRATED CIRCUITS (SHANGHAI) CO LTD
- Filing Date
- 2024-11-19
- Publication Date
- 2026-05-29
AI Technical Summary
In existing technologies, parallel matrix computation is inefficient, especially when reading matrix elements, resulting in low computational efficiency.
A matrix parallel computing system based on local storage is adopted. By distributing the row vectors of the first matrix to the computing sub-modules and using broadcast to send the row vectors of the second matrix to each sub-module, the number of times elements are read is reduced and the reading efficiency is improved.
It improves the efficiency of parallel matrix computing, saves reading time, and enhances the speed and efficiency of computation.
Smart Images
Figure CN122111306A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of GPU parallel processing technology, and in particular to a matrix parallel computing system based on local storage. Background Technology
[0002] GPUs, with their greater number of Arithmetic And Logic Units (ALUs) for data processing, are well-suited for parallel processing of dense data and excel at large-scale concurrent computations, such as multiplication of large matrices.
[0003] In matrix multiplication, calculating an element in the result requires multiplying the corresponding elements of a row in one matrix and a column in another matrix, then summing the results. Clearly, calculating a single element in the multiplication result requires multiple element reads. Current methods, after reading an element from a row in one matrix, multiply that element by the corresponding row in the other matrix, then sum the results to obtain the element in the multiplication result. This reduces the number of element reads, saves data reading time, and improves the efficiency of matrix calculations.
[0004] However, existing methods require reading each row of a matrix for every single element read, resulting in low reading efficiency and consequently poor efficiency in matrix parallel computation. Therefore, improving the efficiency of matrix parallel computation has become an urgent problem to be solved. Summary of the Invention
[0005] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:
[0006] A matrix parallel computing system based on local storage, the system comprising: a GPU, a processor, and a memory storing a computer program, wherein the GPU includes a parallel computing module, the parallel computing module comprising M computing sub-modules, the parallel computing module corresponding to local storage, and the memory further including a first matrix A and a second matrix B to be multiplied, A and B being stored in row order, A having a size of I×J, B having a size of J×K, I / M=N, where N is a positive integer, and when the computer program is executed by the processor, the following steps are implemented:
[0007] S101, store the second matrix B to the local storage.
[0008] S102, initialize k=0, p=1.
[0009] S103, the (k+1)th to (k+M)th first row vectors in the first matrix A are sent to the M calculation submodules respectively.
[0010] S104, at the p-th preset time point, the q-th second row vector of the second matrix B is broadcast from the local storage to the M calculation sub-modules, where q = g(p) and g() is a mapping function.
[0011] S105, in the m-th calculation submodule, the q-th element of the (k+m)-th first row vector is multiplied by the q-th second row vector to obtain the intermediate vector e. k(m) q m is an integer in the range [1, M].
[0012] S106, update p = p + 1, return to execute steps S104 to S105 until p = (k + 1)J + 1, then execute step S107.
[0013] S107, e k(m) 1. e k(m) 2、…、e k(m) J Adding them together, we get the (k+m)th subvector f. k+m .
[0014] S108, update k = k + M, return to execute steps S103 to S107 until k = (N + 1) × M, then execute step S109.
[0015] S109, the result of multiplying the first matrix A and the second matrix B is formed by all the sub-vectors.
[0016] Compared with the prior art, the present invention has significant advantages. Through the above technical solution, the matrix parallel computing system based on local storage provided by the present invention achieves considerable technological progress and practicality, and has broad industrial application value. It has at least the following advantages:
[0017] This invention provides a matrix parallel computing system based on local storage. The system includes a GPU, a processor, and a memory storing a computer program. The GPU includes a parallel computing module, which comprises M computing sub-modules. Each parallel computing module corresponds to local storage. The memory also includes a first matrix A and a second matrix B to be multiplied. A and B are stored in row-order. The size of A is I×J, and the size of B is J×K, where I / M = N, and N is a positive integer. When the computer program is executed by the processor, the following steps are implemented: S101, the matrix... The second matrix B is stored in the local storage. S102, initialize k = 0, p = 1. S103, send the (k+1)th to (k+M)th first row vectors of the first matrix A to the M computational submodules respectively. S104, at the p-th preset time point, broadcast the q-th second row vector of the second matrix B from the local storage to the M computational submodules, where q = g(p), g() is a mapping function. S105, in the m-th computational submodule, multiply the q-th element of the (k+m)-th first row vector with the q-th second row vector to obtain the intermediate vector e. k(m) q m is an integer in the range [1, M]. S106, update p = p + 1, return to execute steps S104 to S105 until p = (k + 1)J + 1, execute step S107, S107, set e k(m) 1. e k (m) 2、…、e k(m) J Adding them together, we get the (k+m)th subvector f. k+m S108, update k = k + M, return to execute steps S103 to S107 until k = (N + 1) × M, execute step S109, S109, form the multiplication result of the first matrix A and the second matrix B from all sub-vectors.
[0018] It can be seen that by directly distributing the first row vector of the first matrix A to each computation submodule and broadcasting the second row vector of the second matrix B to each computation submodule, it is not necessary to read the elements of the first row vector multiple times, thus reducing the number of reads. Furthermore, since both the first matrix A and the second matrix B are read in units of row vectors, which is consistent with their initial storage order, it is more user-friendly for the reading process, thereby improving reading efficiency, saving reading time, and ultimately improving the efficiency of matrix parallel computation. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a flowchart illustrating the execution of a computer program by a processor in a matrix parallel computing system based on local storage, as provided in an embodiment of the present invention. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] This embodiment provides a matrix parallel computing system based on local storage. The system includes a GPU, a processor, and a memory storing a computer program. The GPU includes a parallel computing module, which comprises M computing sub-modules. Each parallel computing module corresponds to local storage. The memory also includes a first matrix A and a second matrix B to be used for multiplication calculations. A and B are stored in row-major order. The size of A is I×J, and the size of B is J×K, where I / M = N, and N is a positive integer. See [link to documentation]. Figure 1 This is a flowchart illustrating the execution of a computer program by a processor in a matrix parallel computing system based on local storage, according to an embodiment of the present invention. When the computer program is executed by the processor, the following steps are implemented:
[0023] S101, store the second matrix B to the local storage;
[0024] S102, initialize k=0, p=1;
[0025] S103, the (k+1)th to (k+M)th first row vectors in the first matrix A are respectively sent to the M calculation submodules;
[0026] S104, at the p-th preset time point, the q-th second row vector of the second matrix B is broadcast from the local storage to the M calculation sub-modules, where q = g(p) and g() is a mapping function;
[0027] S105, in the m-th calculation submodule, the q-th element of the (k+m)-th first row vector is multiplied by the q-th second row vector to obtain the intermediate vector e. k(m) q m is an integer in the range [1, M].
[0028] S106, update p = p + 1, return to execute steps S104 to S105 until p = (k + 1)J + 1, then execute step S107;
[0029] S107, e k(m) 1. e k(m) 2、…、e k(m) J Adding them together, we get the (k+m)th subvector f. k+m ;
[0030] S108, update k = k + M, return to execute steps S103 to S107 until k = (N + 1) × M, then execute step S109;
[0031] S109, the result of multiplying the first matrix A and the second matrix B is formed by all the sub-vectors.
[0032] GPUs typically include multiple parallel computing modules. This embodiment describes a matrix multiplication task performed by a single parallel computing module as an example. Implementers should understand that the matrix multiplication task performed by a single parallel computing module can be a part of the matrix multiplication task performed by the GPU. The matrix corresponding to the matrix multiplication task performed by the GPU can be a first initial matrix R and a second initial matrix S. Accordingly, the first matrix A can refer to a part of the first initial matrix R, and the second matrix B can refer to a part of the second initial matrix S. The first matrix A and the second matrix B have a corresponding relationship to perform matrix multiplication calculations.
[0033] Local memory has a smaller storage capacity than the memory where A and B are initially stored, but it has a faster read speed. In this embodiment, since the second row vector in B needs to be reused, while the first row vector in A is only used once, only B is stored in local memory, saving the time spent reading A from local memory when it is needed.
[0034] k can be used to identify the k / M+1th calculation round. A calculation round corresponds to a submatrix in A. A submatrix in A includes M row vectors. Then the k / M+1th calculation round can be used to calculate the result of multiplying the submatrix formed by the (k+1)th to the (k+M)th first row vectors in A with B.
[0035] Specifically, broadcasting the q-th second row vector of the second matrix B from local storage to M computational submodules can mean sending the q-th second row vector of the second matrix B to each computational submodule.
[0036] According to the principles of matrix computation, the q-th element in the (k+m)-th first row vector will only be multiplied by the q-th second row vector. Therefore, in the m-th computation submodule, the q-th element in the (k+m)-th first row vector is multiplied by the q-th second row vector to obtain the intermediate vector e. k(m) q When q changes from 1 to J, the calculation of the (k+m)th first row vector and B is completed, resulting in J intermediate vectors. The J intermediate vectors are added together to obtain a sub-vector, which is one of the row vectors in the result of multiplying A and B.
[0037] In one specific implementation, sending the (k+1)th to (k+M)th first row vectors in the first matrix A to the M computational submodules respectively includes:
[0038] Send the (k+m)th first row vector in the first matrix A to the mth calculation submodule.
[0039] In this embodiment, the (k+m)th first row vector in A is directly sent to the corresponding computational logic unit (ALU) in the mth computational submodule. In this embodiment, a single computational submodule may include J groups of computational logic units. Each group of computational logic units corresponds to an element in the first row vector. Each group of computational logic units includes K computational logic units. When a second row vector is received, the elements of the second row vector are distributed to the K computational logic units in the corresponding group of computational logic units, and then the computation can be performed.
[0040] In one specific implementation, q = g(p), including:
[0041] When p mod J = 0, q = J;
[0042] When p mod J ≠ 0, q = p mod J.
[0043] For example, let J = 4. When p = 1, q = 1; when p = 2, q = 2; when p = 3, q = 3; when p = 4, q = 4; when p = 5, q = 1; when p = 6, q = 2; when p = 7, q = 3; when p = 8, q = 4. It can be seen that the second row vector in B broadcasts in a cyclical manner.
[0044] In one specific implementation, when the computer program is executed by the processor, in addition to implementing step S103, the following steps are also implemented:
[0045] Load the (k+M+1)th to (k+2M)th first row vectors from the first matrix A into the memory.
[0046] Since B is stored in local storage, the M first row vectors in A can wait for the second row vector in B to be calculated after being loaded. During the calculation process, memory access is idle. Therefore, in order to improve the efficiency of parallel computing, when the (k+1)th to (k+M)th first row vectors in A are calculated, the (k+M+1)th to (k+2M)th first row vectors in A can be loaded from memory at the same time. This allows the calculation process and the loading process to be parallelized, thereby improving the efficiency of matrix parallel computing.
[0047] In one specific implementation, the memory is a DDR memory.
[0048] In one specific implementation, storing the second matrix B to the local storage includes:
[0049] The second matrix B is stored in the cache, and then stored from the cache to the local storage.
[0050] In this case, when reading a continuous segment of data, if the addresses corresponding to that continuous segment of data are also continuous, it is friendly to the cache line and can improve the efficiency of cache reading. In this embodiment, B is stored in row order and is also stored in the cache according to the row vector, which can improve the efficiency of cache reading.
[0051] In one specific implementation, the time interval between two adjacent preset time points is a preset period.
[0052] The preset period can refer to the instruction period, which is the time required to complete one calculation.
[0053] In one specific implementation, the step of forming the product of the first matrix A and the second matrix B from all sub-vectors includes:
[0054] Concatenate all subvectors row by row, and use the concatenated result as the product of the first matrix A and the second matrix B.
[0055] In the k / M+1th calculation round, the sub-vectors from row k+1 to row k+M in the multiplication result can be calculated. Therefore, the multiplication result of A and B can be obtained by concatenating the sub-vectors from all calculation rounds.
[0056] In this embodiment, the first row vector in the first matrix A is directly distributed to each computation submodule, and the second row vector of the second matrix B is sent to each computation submodule via broadcast. This eliminates the need to read the elements in the first row vector multiple times, reducing the number of reads. Furthermore, both the first matrix A and the second matrix B are read in units of row vectors, consistent with their initial storage order, which is more user-friendly for the reading process, thereby improving reading efficiency, saving reading time, and ultimately improving the efficiency of matrix parallel computation.
[0057] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.
Claims
1. A matrix parallel computing system based on local storage, characterized in that, The system includes a GPU, a processor, and a memory storing a computer program. The GPU includes a parallel computing module, which comprises M computing sub-modules. Each parallel computing module corresponds to local storage. The memory also includes a first matrix A and a second matrix B to be used for multiplication calculations. A and B are stored in row-major order. A has a size of I×J, and B has a size of J×K, where I / M = N, and N is a positive integer. When the computer program is executed by the processor, the following steps are implemented: S101, store the second matrix B to the local storage; S102, initialize k=0, p=1; S103, the (k+1)th to (k+M)th first row vectors in the first matrix A are respectively sent to the M calculation submodules; S104, at the p-th preset time point, the q-th second row vector of the second matrix B is broadcast from the local storage to the M calculation sub-modules, where q = g(p) and g() is a mapping function; S105, in the m-th calculation submodule, the q-th element of the (k+m)-th first row vector is multiplied by the q-th second row vector to obtain the intermediate vector e. k(m) q m is an integer in the range [1, M]. S106, update p = p + 1, return to execute steps S104 to S105 until p = (k + 1)J + 1, then execute step S107; S107, e k(m) 1. e k(m) 2、…、e k(m) J Adding them together, we get the (k+m)th subvector f. k+m ; S108, update k = k + M, return to execute steps S103 to S107 until k = (N + 1) × M, then execute step S109; S109, the result of multiplying the first matrix A and the second matrix B is formed by all the sub-vectors.
2. The matrix parallel computing system based on local storage according to claim 1, characterized in that, The step of sending the (k+1)th to (k+M)th first row vectors in the first matrix A to the M calculation submodules respectively includes: Send the (k+m)th first row vector in the first matrix A to the mth calculation submodule.
3. The matrix parallel computing system based on local storage according to claim 1, characterized in that, q = g(p), including: When p mod J = 0, q = J; When p mod J ≠ 0, q = p mod J.
4. The matrix parallel computing system based on local storage according to claim 1, characterized in that, When the computer program is executed by the processor, in addition to implementing step S103, the following steps are also implemented: Load the (k+M+1)th to (k+2M)th first row vectors from the first matrix A into the memory.
5. The matrix parallel computing system based on local storage according to claim 1, characterized in that, The memory is a DDR memory.
6. The matrix parallel computing system based on local storage according to claim 1, characterized in that, The step of storing the second matrix B to the local storage includes: The second matrix B is stored in the cache, and then stored from the cache to the local storage.
7. The matrix parallel computing system based on local storage according to claim 1, characterized in that, The time interval between two adjacent preset time points is a preset period.
8. The matrix parallel computing system based on local storage according to claim 1, characterized in that, The result of multiplying the first matrix A and the second matrix B by all sub-vectors includes: Concatenate all subvectors row by row, and use the concatenated result as the product of the first matrix A and the second matrix B.