Two-Dimensional Compute-in-Memory Mesh for DNN Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compute-in-memory (CIM) accelerator systems for deep neural networks (DNNs) face inefficiencies due to inadequate attention to non-MAC operations, on-chip interconnects, and data-transfer, leading to a gap between peak and sustained performance and energy-efficiency, particularly for workloads like language and speech networks.
Innovation Solution
A heterogeneous and programmable CIM accelerator architecture with a 2D mesh of CIM analog tiles and specialized compute cores for MAC operations, combined with on-chip interconnects and microcontrollers, supports a wide range of DNN workloads by efficiently performing both MAC and auxiliary operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If spatial-multiplexing across multiple word- or bit-lines is invoked to implement multi-bit weights, then multi-bit weight precision is achieved, but area and energy are traded off
Solution Approach 1:
The patent transitions from binary/few-bit digital memory to analog memory, utilizing the continuous conductance range of emerging non-volatile memories (PCM, RRAM) to store multi-bit weights. This dimensional change from discrete digital states to continuous analog states enables high-precision weight storage without increasing spatial multiplexing complexity
Solution Approach 2:
The patent changes the physical parameter used for weight storage from discrete binary/few-bit states to continuous analog conductance states. This parameter change allows weights to be represented with higher precision naturally, eliminating the need for spatial-multiplexing techniques that trade area for precision
2Adaptability or versatility
If conventional SIMD digital compute-cores are used for auxiliary operations, then diverse DNN workloads are supported, but communication time and energy consumption increase
Solution Approach 1:
The patent merges the CIM analog tiles and digital compute-cores into a unified heterogeneous architecture where both MAC operations and auxiliary operations can be performed on-chip. This integration eliminates the need for external communication between separate CIM and digital processing units, reducing communication time and energy
Solution Approach 2:
The patent introduces on-chip interconnects as intermediaries that enable efficient data transfer between CIM analog tiles and digital compute-cores. These interconnects serve as dedicated communication pathways that reduce the time and energy overhead compared to using conventional external interfaces
3Productivity
If heterogeneous compute cores and memory cores are integrated on-chip, then system-level utilization is optimized, but device complexity increases
Solution Approach 1:
The patent segments the accelerator into distinct functional units: CIM analog tiles for MAC operations, digital compute-cores for auxiliary operations, and memory cores for intermediate signal storage. This segmentation allows each component to be optimized independently while working together as an integrated system, managing complexity through modular design
Solution Approach 2:
The patent designs the heterogeneous compute cores and memory cores to perform multiple functions. The compute-cores can handle various auxiliary operations (activations, pooling, normalization), and the memory cores can serve as scratchpad memory for different types of intermediate signals, reducing the need for dedicated specialized units for each function
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The architecture achieves high energy-efficiency and speed by optimizing system-level utilization, supporting diverse DNN workloads including CNNs, LSTMs, and Transformers, with reduced communication time and energy consumption.
Implementation Method 1
Some emerging non-volatile memories, such as PCM and RRAM, exhibit a broad and continuous range of analog conductance states, offering a path towards high-density weight-storage.
Data Source
AI summary
Embodiments disclosed herein include a compute in-memory (CIM) accelerator architecture for deep neural network (DNN). The CIM accelerator architecture may include a first analog fabric engine having a plurality of compute in-memory (CIM) analog tiles. Each CIM analog tile may be configured to store a matrix of weight operands producing a vector of outputs from a vector of inputs, and perform in-memory computations. The first analog fabric may also include a plurality of compute cores. Each CIM analog tile and each compute core may include a microcontroller configured to execute a set of instructions. The first analog fabric may also include on-chip interconnects communicatively connecting all CIM analog tiles in the plurality of CIM analog tile to the compute cores.