Dataflow accelerator architecture for general matrix-matrix multiplication and tensor computation in deep learning

Abstract

A tensor computation dataflow accelerator semiconductor circuit is disclosed. The data flow accelerator includes a DRAM bank and a peripheral array of multiply-and-add units disposed adjacent to the DRAM bank. The peripheral array of multiply-and-add units are configured to form a pipelined dataflow chain in which partial output data from one multiply-and-add unit from among the array of multiply-and-add units is fed into another multiply-and-add unit from among the array of multiply-and-add units for data accumulation. Near-DRAM-processing dataflow (NDP-DF) accelerator unit dies may be stacked atop a base die. The base die may be disposed on a passive silicon interposer adjacent to a processor or a controller. The NDP-DF accelerator units may process partial matrix output data in parallel. The partial matrix output data may be propagated in a forward or backward direction. The tensor computation dataflow accelerator may perform a partial matrix transposition.

Description

RELATED APPLICATION DATA[0001]This application claims the benefit of U.S. Patent Application Ser. No. 62 / 777,046, filed Dec. 7, 2018, which is hereby incorporated by reference.BACKGROUND[0002]The present inventive concepts relate to deep learning, and more particularly, to a dataflow accelerator architecture for general matrix-matrix multiplication and tensor computation in deep learning.[0003]Deep neural networks are considered as a promising approach to realizing artificial intelligence, and have demonstrated their effectiveness in a number of applications. Training deep neural network requires both high precision and wide dynamic range, which demand efficient floating point operations. Tensor computation, which includes the majority of floating point operations and contributes the most time in training deep neural networks, is a key primitive operation for acceleration. Compute-centric accelerators for tensor computation suffer from a “memory wall” issue, since computation perfor...

AI Technical Summary

Benefits of technology

The patent is about a new type of chip that speeds up the process of performing tensor computations. The chip has a special design that allows data to be processed in a more efficient way. By combining two types of units, the chip can quickly accumulate data and perform a type of computation called a partial matrix transposition. This can be useful for applications like machine learning and image processing. Essentially, this chip makes it faster and more efficient to do complicated calculations.

Problems solved by technology

Compute-centric accelerators for tensor computation suffer from a “memory wall” issue, since computation performance scales much faster than memory bandwidth and latency, and off-chip data movement consumes two orders greater magnitude of energy than a floating point operation.

However, conventional PIM approaches mainly explore deep learning inference applications, which may tolerate reduced precision, but may also be incapable of complex floating point training tasks.

However, the number of ALUs is strictly confined due to area budget, and NDP approaches also lose a significant amount of internal bandwidth compared with compute-centric architectures.

These shortcomings make NDP architectures less effective in floating point performance compared with compute-centric approaches.

Moreover, simply adding floating points units to satisfy the computational demands of tensor processing results in significant and unacceptable area overhead in a DRAM die.

In addition, emerging non-volatile memory based accelerators suffer from poor write endurance and long write latency, which are unsuitable for write-intensive deep learning training tasks.

Furthermore, static random-access memory (SRAM)-based accelerators do not have enough on-chip memory capacity to store all of the model parameters and intermediate results needed for deep learning training.

Images