CIM Memory Array With Capacitor Averaging for Faster MAC
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The transfer of data elements between a processor and memory becomes a major bottleneck for AI computations in neural networks, making it impractical to store large and deep neural network data in processor cache, and existing compute-in-memory (CIM) devices are limited in efficiency.
Innovation Solution
A CIM memory device with a controller, write and input buffers, a CIM array, computing circuit, and analog-to-digital converter (ADC) performs MAC operations directly at the memory cell level, enabling faster computations by storing weight and activation data and generating digital representations of results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data elements are transferred between processor and memory for neural network computations, then computation can be performed with stored data, but data transfer becomes a major bottleneck reducing computation efficiency
Solution Approach 1:
The patent merges the memory array with computing circuits by integrating MAC operation circuits directly within the memory structure. Weight data is stored in memory cells while computing circuits perform multiply-accumulate operations on this stored data, combining storage and computation functions into a single integrated system to eliminate data transfer between separate memory and processor components
Solution Approach 2:
The patent introduces computing circuits as intermediary components between the memory cells and the external processor. These computing circuits receive input data, perform MAC operations with weight data stored in memory cells, and output results without requiring continuous data transfer to external processors, thus mediating the computation process within the memory device itself
2Speed
If compute-in-memory devices perform MAC operations at memory cell level, then computation speed is improved, but device complexity increases
Solution Approach 1:
The memory device is designed with universal computing circuits that can perform MAC operations on multiple weight data elements stored in memory cells. The same computing circuit structure handles different computations by receiving different input data vectors, making the device multi-functional without requiring separate dedicated circuits for each computation type
Solution Approach 2:
The computing circuits are segmented into multiple independent units that can operate in parallel on different sets of weight data. Each computing circuit unit handles specific memory cell columns or rows, allowing parallel MAC operations across multiple data elements simultaneously, which improves computation speed while distributing the complexity across modular units
Data Source
AI summary
A memory device is provided. The memory device includes a compute-in-memory (CIM) array, capacitor circuit pairs, a first switch circuit and an analog-to-digital converter. The CIM array includes bit cells arranged in columns, in which the CIM array generates, in response to an input vector and a stored vector in the bit cells, accumulation results. The capacitor circuit pairs receive the accumulation results through bit lines, in which portions, in one of the columns, of the bit cells and a corresponding one of capacitor circuit pairs are coupled to a corresponding bit line of the bit lines. The first switch circuit is coupled to the capacitor circuit pairs is switched to generate, based on the accumulation results, weight mean results in one capacitor circuit in each of the plurality of capacitor circuit pairs. The analog-to-digital converter generates, according to the plurality of weight mean results, a multiply-and-accumulate result.