GPU batch matrix multiplication accelerator and processing method thereof

Abstract

The invention discloses a GPU (Graphics Processing Unit) batch matrix multiplication accelerator, which comprises a main branch circuit consisting of a working group manager, an instruction decoding processor, a high-speed storage unit, a shared memory unit, a computing unit and a DMA (Direct Memory Access), and is used for carrying out batch order optimization on matrix data to obtain matrix fragments and loading the matrix fragments into the computing unit; a bypass branch circuit is composed of an instruction decoding processor, a high-speed storage unit, a shared memory unit, a calculation unit and a DMA, when batch order optimization is not carried out, kernel function calculation is directly carried out on matrix data, and a matrix calculation result is obtained; according to the method, the utilization efficiency of the computing units can be effectively improved, the load balance of the task load of each computing unit is achieved, the computing density during operation is improved, and higher instruction parallelism, thread parallelism and memory access parallelism are achieved, so that the computing power of hardware is fully exerted, and the purpose of computing acceleration is achieved.

Description

technical field [0001] The invention relates to the research field of high-performance computing, in particular to a GPU batch matrix multiplication accelerator and a processing method thereof. Background technique [0002] Basic Linear Algebra Subprograms (Basic Linear Algebra Subprograms, BLAS) is an interface standard for a series of basic linear algebra operation functions, which are widely used in various fields of scientific computing and industry. Some higher-level languages ​​and computing libraries are also implemented by calling the BLAS interface (R language, Matlab, Numpy, Lapack, etc.). Nowadays, with the development of BLAS, various BLAS library implementations based on different platforms and hardware architectures have appeared, such as CuBLAS, RocBLAS, MKL, MAGMA, OpenBLAS, etc., and have played a crucial role in the development of modern science and industry. [0003] Classical BLAS application scenarios tend to have better performance for large inputs (la...

AI Technical Summary

Problems solved by technology

[0006] However, in variable-size matrix calculation operations, simultaneous calls of different kernel functions between different calculation instances will lead to irregularities inside the hardware during calculation; Due to the Round-robin scheduling algorithm, the workload of each computing unit is unbalanced when the computing unit is loaded, resulting in uneven distribution of GPU computing resources, which in turn has a great impact on computing parallelism and computing density. damage

[0007] In the prior art, there are the following problems: when the existing GPU-based batch matrix multiplication calculation library solves variable-size large-batch matrix multiplication, the combined effect of the variable-size matrix input distribution and the work group scheduling algorithm will lead to Serious load imbalance and internal irregularities among the computing units affect the execution efficiency of the computing pipeline, making the GPU unable to make good use of the parallel capability of the hardware, thereby reducing the calculation rate of the final algorithm

Images