Refactoring MAC Operations Using NVM Crossbar Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine-learning (ML) applications using Non-Volatile Memory (NVM)-based crossbar architectures for multiply-and-accumulate (MAC) operations are energy-intensive due to limitations in bit-precision and the need for precise programming of NVM elements, which is challenging for large neural networks, especially in power-constrained devices.
Innovation Solution
The solution involves refactoring MAC operations by using a summing array and a multiplying array with NVM elements programmed to high resistance or low resistance states, allowing for simplified programming and reduced energy consumption by initially summing input signals and then multiplying them by conductance levels, thereby reducing the complexity of weight programming and energy usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If NVM elements are programmed to precise multi-bit conductance levels to represent weight parameters, then the accuracy of MAC operations is improved, but the programming complexity and energy consumption increase significantly
Solution Approach 1:
The patent segments the weight programming process into two distinct phases: first programming NVM elements to coarse conductance levels using simple crossbar operations, then refining the conductance values through iterative adjustment. This segmentation reduces the initial programming complexity while achieving the required precision for accurate MAC operations.
Solution Approach 2:
The patent performs preliminary coarse programming of NVM elements to approximate weight values before final precision adjustment. This preliminary action reduces the burden of direct precise programming by establishing a close initial approximation that requires minimal refinement, thereby reducing overall programming complexity and energy consumption.
2Measurement precision
If NVM elements are programmed to precise multi-bit conductance levels, then the accuracy of MAC operations is improved, but the energy consumption increases
Solution Approach 1:
The programming energy consumption is segmented into two phases: low-energy coarse programming to establish approximate weight values, followed by higher-energy fine-tuning to achieve precise conductance levels. This segmentation reduces total energy consumption by minimizing the duration and intensity of high-energy programming operations.
Solution Approach 2:
The patent performs preliminary coarse programming using low-energy operations to establish approximate weight values before final precision adjustment. This preliminary action reduces the energy required for achieving precise conductance levels by reducing the magnitude of adjustments needed in the final programming stage.
3Measurement precision
If explicit multipliers and adders are used in the digital domain to implement MAC operations, then the computational accuracy is improved, but the energy consumption increases
Solution Approach 1:
The patent replaces digital multipliers and adders with an analog crossbar array that performs MAC operations using Ohm's law and Kirchhoff's current law. Currents through NVM elements with conductance values representing weights are summed at column lines, naturally performing multiplication and accumulation in parallel without requiring explicit digital computational units, thereby reducing energy consumption while maintaining computational accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces the programming steps and energy consumption in MAC operations, enabling more efficient processing of large neural networks without the need for precise programming of NVM elements, thus improving performance and scalability in power-constrained devices.
Implementation Method 1
The mixed-signal approach using NVM-bit cells relies upon Ohm's law to implement multiply operations by taking advantage of the resistive nature of emerging NVM technologies
Implementation Method 2
Currents from multiple bit cells are added in parallel to implement an accumulated sum. Thus, a combination of Ohm's law and Kirchoff's current law implements multiple MAC operations in parallel
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
A method and apparatus for performing refactored multiply-and-accumulate operations is provided. A summing array includes a plurality of non-volatile memory elements arranged in columns. Each non-volatile memory element in the summing array is programmed to a high resistance state or a low resistance state based on weights of a neural network. The summing array is configured to generate a summed signal for each column based, at least in part, on a plurality of input signals. A multiplying array is coupled to the summing array, and includes a plurality of non- volatile memory elements. Each non-volatile memory element in the multiplying array is programmed to a different conductance level based on the weights of the neural network. The multiplying array is configured to generate an output signal based, at least in part, on the summed signals from the summing array.