In-Memory MAC Architecture With Swapped Charge-Storage Banks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current In-Memory Computing (IMC) technologies for AI applications face power efficiency issues due to latencies and high power overheads associated with converting digital byte streams to analog for MAC operations, particularly in analog IMC implementations that rely on digital-to-analog converters.
Innovation Solution
An IMC architecture with a MAC core featuring an array of multilevel non-volatile memory cells, shared bit-lines, and charge-storage banks, along with analog-to-digital converters, that sequentially shifts and accumulates bit-line currents, and concurrently converts charges to improve efficiency and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If digital-to-analog converters (DACs) are used to convert input data byte streams to analog for MAC operations, then analog IMC solutions can be implemented, but significant power overheads and latencies are introduced
Solution Approach 1:
The patent removes the DAC component from the IMC architecture entirely. Instead of converting digital input data to analog signals, the system directly uses digital input data to control memory cell operations, performing MAC computations in the analog domain without requiring digital-to-analog conversion. This extraction of the DAC eliminates the associated power overhead while preserving analog IMC functionality.
Solution Approach 2:
The patent segments the IMC operation into distinct phases: digital input registration, analog weight multiplication in memory cells, and analog-to-digital conversion of results. By separating the input data path from the analog computation path, the system avoids converting input data to analog while still leveraging analog memory cells for efficient MAC operations.
2Adaptability or versatility
If DACs are used to convert digital byte streams to analog prior to accessing memory rows, then MAC operations can be performed in analog, but significant latencies are introduced
Solution Approach 1:
The patent performs preliminary action by pre-loading weight values into the memory cells before the MAC operation begins. The input digital data is registered in advance in shift registers, and memory cells are pre-configured with weight values. When the MAC operation starts, the pre-prepared digital input data directly controls the analog multiplication in memory cells without requiring real-time DAC conversion, thereby eliminating conversion latency.
Solution Approach 2:
The patent replaces the mechanical DAC conversion process with a direct digital-to-analog control mechanism. Instead of using a DAC to generate analog signals from digital input, the system uses digital input signals to directly control the switching and operation of analog memory cells, substituting the DAC mechanism with a more efficient digital control approach.
3Speed
If traditional IMC architectures are used, then data processing speed can be improved, but power efficiency deteriorates due to continuous data movement between memory and processing elements
Solution Approach 1:
The patent merges the memory storage function and the MAC computation function into a single integrated structure. Memory cells simultaneously store weight values and perform analog multiplication operations. The digital input data directly controls these in-memory computations, eliminating the need to move data between separate memory and processing elements, thereby achieving both high speed and power efficiency.
Solution Approach 2:
The memory cells in the patent serve multiple functions: they store weight values, perform analog multiplication, and generate output currents. This multi-functionality allows the same hardware structure to handle both data storage and computation, eliminating redundant data movement and improving power efficiency while maintaining high processing speed.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enhances power efficiency and processing speed by eliminating the need for additional power scaling and reducing charge leakage, resulting in higher operations per second per watt compared to prior art IMC architectures.
Implementation Method 1
Each memory cell includes a multilevel, non-volatile memory (NVM) device... each memory cell stores a weight value in an analog format
Implementation Method 2
activate the NVM devices based on a state of the bit, and to produce a weighted bit-line current from each activated NVM device proportional to a product of the bit and a weight stored in the NVM device
Implementation Method 3
A plurality of first charge-storage banks... configured receive a sum of weighted bit-line currents and to accumulate for each bit of the input bytes charge produced by the sum of weighted bit-line currents
Implementation Method 4
plurality of second charge-storage banks coupled to a number of analog-to-digital converters (ADCs), each of the second charge-storage banks configured to concurrent with the shifting and accumulating, to provide scaled voltages for each bit of previously received second input bytes to the ADC for conversion into an output byte
Data Source
AI summary
In-memory computing architectures and methods of performing multiply-and-accumulate operations are provided. The method includes sequentially shifting bits of first input bytes into each row in an array of memory cells arranged in rows and columns. Each memory cell is activated based on the bit to produce a bit-line current from each activated memory cell in a column on a shared bit-line proportional to a product of the bit and a weight stored therein. Charges produced by a sum of the bit-line currents in a column are accumulated in first charge-storage banks coupled to a shared bit-line in each of the columns. Concurrently, charges from second input bytes accumulated in second charge-storage banks previously coupled to the columns are sequentially converted into output bytes. The charge-storage banks are exchanged after the first input bytes have been accumulated and the charges from the second input bytes converted. The method then repeats.