Hardware accelerator supporting dynamic and static sparse attention mechanisms

Abstract

The application discloses a kind of support dynamic sparse and static sparse hardware accelerator, comprising: data segmentation and reordering module is used to the input data is segmented to adapt to the size of spatial accelerator hardware;Spatial accelerator module is used to support the operation of sparse attention mechanism;Weight summation module is used to support the segmentation of hybrid attention mode, the result obtained by the calculation of segmented input data is merged to obtain the final output;Pattern matching module includes matrix multiplication operation module, double tuning sequencer and sliding window comparator, for hybrid attention mode matching.The application simultaneously supports static sparse method and dynamic sparse method, and can carry out efficient static and dynamic sparse attention calculation.Saves at least 6.1 times of calculation amount, while maintaining the accuracy of output result.

Images