Two-Dimensional Mesh Compute-in-Memory Architecture for DNN Data Exchange
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compute-in-memory (CIM) accelerator systems for deep neural networks (DNNs) face inefficiencies due to inadequate attention to non-MAC operations, on-chip interconnects, data-transfer, and power management, particularly for workloads beyond Convolutional Neural Networks (CNNs), and lack support for diverse DNN workloads like language and speech networks.
Innovation Solution
A heterogeneous and programmable CIM accelerator architecture with spatially-distributed CIM memory-array tiles and specialized compute-cores for auxiliary digital computation, utilizing a two-dimensional mesh for efficient data exchange, supporting a wide range of DNN workloads including CNNs, LSTM, and Transformers, with power-gating mechanisms to optimize energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If spatial-multiplexing across multiple word- or bit-lines is invoked to implement multi-bit weights, then weight storage capacity is improved, but area and energy are increased
Solution Approach 1:
The patent transitions from binary/few-bit storage to analog conductance states, adding a dimensional aspect to weight representation. By exploiting the continuous range of analog conductance states in emerging non-volatile memories, the system can store multi-bit weights directly in a single memory cell without requiring spatial multiplexing across multiple bit-lines, thereby improving storage capacity while reducing area overhead
Solution Approach 2:
The patent changes the physical parameter used for weight storage from discrete binary states to continuous analog conductance states. This parameter change enables direct representation of multi-bit weights through varying conductance levels, eliminating the need for spatial multiplexing and reducing the area required for weight storage while maintaining high storage capacity
2Adaptability or versatility
If conventional SIMD digital compute-cores are used for auxiliary operations, then computational flexibility is improved, but system complexity and communication overhead are increased
Solution Approach 1:
The patent segments the compute-in-memory architecture into specialized functional units, each optimized for specific auxiliary operations (e.g., separate units for activation functions, normalization, pooling). This segmentation allows each unit to be simple and efficient while collectively providing the flexibility of a full SIMD core, reducing overall system complexity while maintaining adaptability
Solution Approach 2:
The patent designs auxiliary compute units with multi-functional capabilities that can handle various types of non-MAC operations (activation functions, normalization, pooling, etc.) using unified hardware structures. This universality reduces the number of specialized components needed, thereby simplifying system architecture while maintaining computational flexibility across different DNN workloads
3Productivity
If on-chip scratchpad memory is provided for intermediate signals, then computation speed is improved, but area consumption is increased
Solution Approach 1:
The patent merges the scratchpad memory functionality with the compute units by integrating small on-chip memory buffers directly within or adjacent to the auxiliary compute cores. This merging allows intermediate signals to be stored locally without requiring separate large scratchpad memory arrays, thereby improving computation speed through reduced access latency while minimizing area consumption through compact integrated memory structures
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The architecture achieves high energy-efficiency and speed by optimizing system-level utilization, supporting diverse DNN workloads through efficient data transport and power management, enabling functional equivalence in DNN accuracy with reduced communication penalties.
Implementation Method 1
utilizing a two-dimensional mesh for efficient data exchange
Implementation Method 2
power-gating mechanisms to optimize energy efficiency
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments disclosed herein include a compute in-memory (CIM) accelerator architecture for deep neural network (DNN). The CIM accelerator architecture may include a first analog fabric engine having a plurality of compute in-memory (CIM) analog tiles. Each CIM analog tile may be configured to store a matrix of weight operands producing a vector of outputs from a vector of inputs, and perform in-memory computations. The first analog fabric may also include a plurality of compute cores. Each CIM analog tile and each compute core may include a microcontroller configured to execute a set of instructions. The first analog fabric may also include on-chip interconnects communicatively connecting all CIM analog tiles in the plurality of CIM analog tile to the compute cores.