大規模生成人工知能ハードウェアのストリーミング演算のための重みメモリマッピング方法およびシステム

Abstract

The present invention provides a weight memory mapping method and system for streaming computations of large-scale generative artificial intelligence hardware. A weight memory mapping system according to one embodiment includes a weight memory for storing a weight matrix for a pre-trained artificial intelligence model, an input register for storing a plurality of input data, a first hardware arithmetic unit that processes matrix multiplication between the plurality of input data and the weight matrix and reuses the partial sum of the matrix operation to calculate a final sum on a lane-by-lane basis while the matrix multiplication is in progress, and a second hardware arithmetic unit that uses the final sum to pre-process the next matrix multiplication while the matrix multiplication is in progress.

Images