Deep learning accelerator for accelerating BERT neural network operation

Abstract

The invention belongs to the technical field of integrated circuits, and particularly relates to a deep learning accelerator for accelerating BERT neural network operation. The accelerator comprises:three matrix multiply arrays used for calculating multiply-accumulate operation; a Softmax and dot product calculation unit that is used for calculating a Softmax probability function and carrying outdot multiplication on branch network output to obtain an output characteristic value; three feature memories that are used for storing input and output feature values; two weight memories; and a controller and an on-chip off-chip interface, wherein the controller is used for controlling the data in the off-chip DRAM to interact with the on-chip data. A branch network structure in the neural network is optimized, so that the storage space of intermediate data is effectively reduced, the off-chip on-chip data interaction frequency is reduced, and the power consumption is reduced; meanwhile, reconfigurable data interconnection between the storage unit and the calculation unit is configured, so that the branch network structure calculation requirement in the BERT neural network is met, and the method can be used for end-to-end neural network calculation.

Description

technical field [0001] The invention belongs to the technical field of integrated circuits, and in particular relates to a deep learning accelerator for accelerating BERT neural network operations. Background technique [0002] With the development of the BERT neural network, it is widely used in natural language processing tasks such as automatic question answering, text translation, text classification, and speech recognition. The BERT neural network includes a multi-head attention (Multi-head attention) neural network part with a branch structure and an end-to-end feedforward prediction neural network part. The former is used to extract the associated features between contexts, and the latter predicts the next moment of speech or text output based on the extracted features. [0003] Current computing platforms CPU / GPU or traditional custom-designed neural network accelerators can only accelerate the processing of end-to-end neural networks. For a neural network with a b...

AI Technical Summary

Problems solved by technology

The output eigenvalues ​​of each branch must be temporarily stored in a specific storage space when they do not participate in post-processing, wasting storage units

Images