Hardware accelerator capable of configuring sparse attention mechanism

Abstract

The invention discloses a hardware accelerator capable of configuring a sparse attention mechanism. The hardware accelerator comprises a sample dense matrix multiplication module, a mask block packaging module and a configurable sparse matrix multiplication module, wherein the sample dense matrix multiplication module adopts a hardware structure of a systolic array; the mask block packaging module comprises a column number counter, a row activation unit counter and a buffer area; the configurable sparse matrix multiplication module comprises a configurable arithmetic unit PE, a register array and a divider, and the configurable arithmetic unit is separated from the register array. According to the method, the sparse mode of the fractional matrix is efficiently and dynamically determined according to the characteristics of the input matrix, the high circulation amount can still be kept under the high sparseness, and the operation of the sparse attention mechanism can be efficiently and dynamically accelerated.

Description

technical field [0001] The invention relates to an artificial intelligence application hardware accelerator, in particular to a hardware accelerator capable of configuring a sparse attention mechanism, which is a multi-stage systolic array hardware accelerator that can be configured for the sparse attention mechanism. Background technique [0002] Artificial neural networks based on attention mechanism have played an important role in machine learning in recent years. Literature [1] (A.Vaswani, N.Shazeer, N.Parmar, J.Uszkoreit, L.Jones, A.N.Gomez, Kaiser, and I. Polosukhin, "Attention is all you need," in Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp.6000–6010.) document the attention mechanism. Among them, the Transformer structure uses the attention mechanism as its basic component, and has excellent performance in various artificial intelligence tasks, such as language models in the field of natural language processi...

AI Technical Summary

Problems solved by technology

However, this work only designed a few dot product operators at the hardware structure level, and did not have good hardware acceleration capabilities.

Images