Dataflow accelerator architecture for general matrix-matrix multiplication and tensor computation in deep learning

Abstract

A tensor computation dataflow accelerator semiconductor circuit is disclosed. The data flow accelerator includes a DRAM bank and a peripheral array of multiply-and-add units disposed adjacent to the DRAM bank. The peripheral array of multiply-and-add units are configured to form a pipelined dataflow chain in which partial output data from one multiply-and-add unit from among the array of multiply-and-add units is fed into another multiply-and-add unit from among the array of multiply-and-add units for data accumulation. Near-DRAM-processing dataflow (NDP-DF) accelerator unit dies may be stacked atop a base die. The base die may be disposed on a passive silicon interposer adjacent to a processor or a controller. The NDP-DF accelerator units may process partial matrix output data in parallel. The partial matrix output data may be propagated in a forward or backward direction. The tensor computation dataflow accelerator may perform a partial matrix transposition.

Description

RELATED APPLICATION DATA[0001]This application claims the benefit of U.S. Patent Application Ser. No. 62 / 777,046, filed Dec. 7, 2018, which is hereby incorporated by reference.BACKGROUND[0002]The present inventive concepts relate to deep learning, and more particularly, to a dataflow accelerator architecture for general matrix-matrix multiplication and tensor computation in deep learning.[0003]Deep neural networks are considered as a promising approach to realizing artificial intelligence, and have demonstrated their effectiveness in a number of applications. Training deep neural network requires both high precision and wide dynamic range, which demand efficient floating point operations. Tensor computation, which includes the majority of floating point operations and contributes the most time in training deep neural networks, is a key primitive operation for acceleration. Compute-centric accelerators for tensor computation suffer from a “memory wall” issue, since computation perfor...

AI Technical Summary

Problems solved by technology

Compute-centric accelerators for tensor computation suffer from a “memory wall” issue, since computation performance scales much faster than memory bandwidth and latency, and off-chip data movement consumes two orders greater magnitude of energy than a floating point operation.

However, conventional PIM approaches mainly explore deep learning inference applications, which may tolerate reduced precision, but may also be incapable of complex floating point training tasks.

However, the number of ALUs is strictly confined due to area budget, and NDP approaches also lose a significant amount of internal bandwidth compared with compute-centric architectures.

These shortcomings

Images