In-network and in-memory dual sparsity computing accelerator

Abstract

A Network-On-Chip (NoC)-based accelerator is disclosed for performing in-network and in-memory sparse tensor computations between an activation tensor and a weight tensor. The accelerator comprises a plurality of NoC nodes organized in a first two-dimensional (2D) network. Each NoC node includes a plurality of kernels organized in a second 2D network, and each kernel contains a plurality of cores organized in a third 2D network. The weight tensor is evenly distributed among the NoC nodes in the first 2D network. Within each NoC node, the kernels are configured to perform output-channel gathering on the weight tensor, while the cores within each kernel perform input-channel gathering. These gathering processes generate a condensed weight tensor that is evenly distributed among the NoC nodes. The activation tensor is broadcasted to the NoC nodes, enabling parallel sparse tensor computation using the evenly distributed and condensed weight tensor.

Images