Compute-in-Memory Array Layout for Faster MAC Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional compute-in-memory (CIM) arrays face limitations in MAC result throughput, power consumption, and write/refresh cycle time due to the need for multiple columns per input value and the use of integrator and ADC circuits that consume substrate area, as well as potential disturb effects during programming.
Innovation Solution
A CIM array design where weight values are stored in rows of nonvolatile memory cells, allowing input values to be applied via bit lines, with source lines dedicated to each row, and MAC results are summed and converted on these lines, using a nonvolatile memory array with programmable threshold voltages to enable faster updates and reduced disturb risks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple columns per input value are used in conventional CIM arrays, then weight storage capacity is improved, but device complexity and area consumption increase
Solution Approach 1:
The patent inverts the conventional approach by storing multiple weight values for a single input value in the same column across different rows, rather than using multiple columns. Specifically, weight values W1, W2, and W3 for input X1 are stored in row 1, row 2, and row 3 respectively of the same column pair (0,1), allowing MAC results to be accumulated across rows during sequential processing.
Solution Approach 2:
The patent transitions from a column-based storage organization to a row-based storage organization. Instead of distributing weight values across multiple columns horizontally, the invention stacks weight values vertically across multiple rows within the same column, effectively changing the dimension of data organization from horizontal to vertical.
2Measurement precision
If integrator and ADC circuits are used in conventional CIM arrays, then MAC result conversion is improved, but substrate area consumption increases
Solution Approach 1:
The patent extracts and removes the integrator and ADC circuits from the CIM array architecture. Instead of including these conversion circuits within the array substrate, the invention outputs analog MAC results directly from the memory cells to external conversion circuits, thereby eliminating the substrate area overhead while maintaining conversion functionality.
Solution Approach 2:
The patent introduces bit line pairs as intermediaries to transfer MAC results from memory cells to external conversion circuits. The bit lines serve as conductive pathways that carry the analog signals without requiring on-chip integration or conversion, enabling area-efficient design while preserving signal integrity.
3Manufacturing precision
If multiple row programming is performed in conventional CIM arrays, then weight update completeness is improved, but write/refresh cycle time increases
Solution Approach 1:
The patent segments the weight storage function across multiple row-selectable memory cells within the same column. Each memory cell in a column can be independently programmed via its corresponding row, allowing parallel weight updates across different rows. This segmentation enables complete weight updates to be performed simultaneously across multiple rows rather than sequentially.
Solution Approach 2:
The patent performs preliminary programming of weight values into memory cells before MAC operations are executed. Weight values are pre-loaded into the memory array during idle periods or between inference tasks, so that when MAC operations are needed, the weights are already in place and no additional programming time is required during the computation phase.
4Quantity of substance
If multiple columns per input value are used in conventional CIM arrays, then weight storage capacity is improved, but power consumption increases
Solution Approach 1:
The patent merges multiple weight values into the same column by storing them in different rows of that column. Instead of duplicating column structures to store multiple weights, the invention combines the storage function into a single column that can hold multiple weights through row differentiation, thereby reducing the total number of column pairs required and lowering overall power consumption.
Solution Approach 2:
The patent makes each column pair multi-functional by enabling it to store multiple weight values for different input values. A single column pair (0,1) can sequentially store and process weights W1, W2, W3 for different inputs X1, X2, X3 by selecting different rows, making the column structure universal rather than dedicated to a single weight, thus reducing the total column count and power usage.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The proposed design enhances MAC result throughput, reduces power consumption, and improves write/refresh efficiency by storing weight values in rows, enabling faster updates with lower disturb risks and minimizing the need for multiple row programming.
Implementation Method 1
a current can be generated on a bit line in response to an applied voltage, the current varying according to a programmed state of the memory cell
Implementation Method 2
Currents for selected rows can be accumulated as analog MAC values on a source line common to each row
Implementation Method 3
Analog MAC values from each row can be converted into digital values with ADC circuits
Data Source
AI summary
A method can include, for each row of a nonvolatile memory (NVM) cell array, generating a multiply-accumulate (MAC) result for the row by applying input values on bit lines. Each MAC result comprising a summation of an analog current or voltage that is a function of each input value modified by a corresponding weight value stored by the NVM cells of the row. By operation of at least one multiplexer, one of the rows can be connected to an analog-to-digital converter (ADC) circuit to convert the analog current or voltage of the row into a digital MAC value. A storage element of each NVM cell can be configured to store a weight value that can vary between no less than three different values. Corresponding devices and systems are also disclosed.