CIM Memory Array Charge-Sharing for Faster Neural MAC
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The transfer of data elements between a processor and memory becomes a major bottleneck for AI computations in neural networks, making it impractical to store large and deep neural networks in processor cache, and existing compute-in-memory (CIM) devices are limited in efficiency.
Innovation Solution
A CIM memory device with a controller, write and input buffers, a CIM array, computing circuit, and analog-to-digital converter (ADC) performs MAC operations at memory cell level, reducing data transfer latency by storing weight and activation data in bit cells and using capacitor circuits for charge-sharing to calculate weighted mean results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data elements are transferred between processor and memory for AI computations, then computations can be performed, but data transfer becomes a major bottleneck and time consumption increases
Solution Approach 1:
The patent merges computing functionality directly into the memory structure by implementing compute-in-memory (CIM) operations. The memory device performs multiply-accumulate operations directly on stored weight and activation data, combining data storage and computation in a single integrated system. This eliminates the need to transfer data between separate processor and memory components, thereby resolving the data transfer bottleneck while maintaining high computation speed.
2Loss of time
If existing compute-in-memory devices are used to process dot product multiplications, then computation time is reduced, but device efficiency is limited
Solution Approach 1:
The patent segments the computation process into fine-grained operations at the bit-cell level within the memory array. Each bit cell can independently perform multiply-accumulate operations on individual bits of weight and activation data. This segmentation enables parallel processing across multiple bit cells simultaneously, significantly reducing computation time while improving overall device efficiency through distributed computation architecture.
Solution Approach 2:
The patent changes the operational parameters of the memory device by enabling it to perform compute-in-memory operations directly. The memory system transitions from passive data storage to active computation by modifying control signals and operational modes. This parameter change allows the memory to execute MAC operations locally on stored data, reducing computation time and enhancing device efficiency without requiring external processor intervention.
3Productivity
If large and deep neural networks are stored in processor cache, then computations can be performed efficiently, but it becomes impractical due to memory capacity constraints
Solution Approach 1:
The patent makes the memory device universal by enabling it to perform both data storage and computation functions. The same memory structure that stores weight and activation data also executes the multiply-accumulate operations directly. This multi-functionality eliminates the need for separate processor cache memory, allowing large and deep neural networks to be stored and processed in the memory itself, thereby resolving the capacity constraint while maintaining computation efficiency.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The CIM device enhances computation speed by performing MAC operations directly in memory, reducing latency and improving computational efficiency for neural networks.
Implementation Method 1
capacitor circuits to store data values
Implementation Method 2
performing accumulation operations to the multiplication results and storing the accumulation results to the capacitor circuits through the bit lines
Data Source
AI summary
A memory device is provided. The memory device includes a compute-in-memory (CIM) array, capacitor circuit pairs, a first switch circuit and an analog-to-digital converter. The CIM array includes bit cells arranged in columns, in which the CIM array generates, in response to an input vector and a stored vector in the bit cells, accumulation results. The capacitor circuit pairs receive the accumulation results through bit lines, in which portions, in one of the columns, of the bit cells and a corresponding one of capacitor circuit pairs are coupled to a corresponding bit line of the bit lines. The first switch circuit is coupled to the capacitor circuit pairs is switched to generate, based on the accumulation results, weight mean results in one capacitor circuit in each of the plurality of capacitor circuit pairs. The analog-to-digital converter generates, according to the plurality of weight mean results, a multiply-and-accumulate result.