In-Memory MAC Accumulation for Multi-Bit AI Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI systems face an IO bottleneck and inefficient multiply-and-accumulation (MAC) operations due to their architecture, leading to reduced efficiency and accuracy, especially when performing MAC operations with multi-bit inputs and weights.
Innovation Solution
An In-Memory-Computing (IMC) memory device with a memory array, multiplication circuit, digital accumulating circuit, analog accumulating circuit, and decision unit that performs bitwise multiplication, analog or digital accumulating, or hybrid accumulating based on the MAC operation result to optimize MAC operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional process-centric architecture is used for AI MAC operations, then system architecture is simple, but IO bottleneck occurs and efficiency is low
Solution Approach 1:
The patent merges the MAC operation functionality directly into the memory array by integrating multiplication circuits and accumulating circuits within the memory structure itself. This combines storage and computation functions, eliminating the need for separate ALUs and reducing data movement between memory and processing units, thereby resolving the IO bottleneck while achieving high MAC operation efficiency.
Solution Approach 2:
The patent replaces the traditional von Neumann architecture's sequential processing mechanism with parallel in-memory computation. Instead of moving data through complex ALU mechanisms, the system performs bitwise multiplication and accumulation directly within memory cells using current-based operations, substituting mechanical data movement with parallel computational operations.
2Measurement precision
If multi-bit inputs and weights are used for high accuracy MAC, then computation accuracy is improved, but IO bottleneck worsens and efficiency decreases
Solution Approach 1:
The patent segments multi-bit inputs and weights into individual bits, performing bitwise multiplication separately for each bit position. This segmentation allows parallel processing of multiple bits simultaneously within the memory array, maintaining high accuracy through bit-level precision while improving efficiency by eliminating sequential processing bottlenecks.
Solution Approach 2:
The patent transitions from sequential scalar MAC operations to parallel bitwise operations across multiple dimensions. By performing multiplication and accumulation operations simultaneously for multiple bit positions within the memory array, the system achieves both high accuracy through bit-level precision and high efficiency through parallelism.
3Loss of energy
If non-volatile IMC is used to reduce data movement, then data movement is reduced, but operation speed and accuracy must be met to improve performance
Solution Approach 1:
The patent implements self-service computation where the memory array performs MAC operations autonomously using its own stored weights and input data. The multiplication circuits and accumulating circuits within the memory cells enable the system to compute results directly without requiring external processing units, thereby reducing data movement while maintaining high operation speed through in-memory parallel computation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The IMC memory device improves MAC operation speed and accuracy by reducing memory cell usage and eliminating the IO bottleneck, achieving higher performance and efficiency compared to traditional AI systems.
Implementation Method 1
in performing bitwise multiplication, the memory cells generate a plurality of memory cell currents
Implementation Method 2
an analog accumulating circuit coupled to the memory array for performing an analog accumulating on the memory cell currents to generate a first MAC operation result
Implementation Method 3
a digital accumulating circuit coupled to the multiplication circuit for performing a digital accumulating on the multiplication results
Data Source
AI summary
A memory device and an operation method thereof are provided. The memory device includes: a memory array including a plurality of memory cells for storing a plurality of weights; a multiplication circuit for performing bitwise multiplication on a plurality of input data and the weights to generate a plurality of multiplication results, wherein in performing bitwise multiplication, the memory cells generate a plurality of memory cell currents; a digital accumulating circuit for performing a digital accumulating on the multiplication results; an analog accumulating circuit for performing an analog accumulating on the memory cell currents to generate a first MAC operation result; and a decision unit for deciding whether to perform the analog accumulating; the digital accumulating or a hybrid accumulating, wherein in performing the hybrid accumulating, whether the digital accumulating circuit is triggered is based on the first MAC operation result.