Counter-Based RPU Crosspoints for Symmetric ANN Weight Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing crosspoint devices in artificial neural networks (ANNs) face challenges such as high device-to-device variability, asymmetry in resistance switching, and inefficient training processes, which hinder power efficiency and training speed.
Innovation Solution
Implementing counter-based resistive processing units (RPUs) with adjustable conductance, utilizing digital counters and resistive circuits to represent weights, enabling symmetric weight updates and parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional crosspoint devices are used in ANNs, then device simplicity is maintained, but device-to-device variability increases and training efficiency decreases
Solution Approach 1:
The crosspoint device is segmented into multiple resistive circuits (e.g., 8 circuits per crosspoint), each controlled by a dedicated counter. This segmentation allows independent control of each resistive circuit's conductance state, enabling precise weight representation and reducing device-to-device variability by distributing the weight control across multiple controllable units rather than relying on a single analog device.
Solution Approach 2:
Counters are introduced as intermediary digital control elements between the input signals and the resistive circuits. These counters receive digital weight values and translate them into precise conductance states of the resistive circuits, acting as mediators that eliminate the need for direct analog control and reduce variability inherent in analog crosspoint devices.
2Speed
If conventional resistive switching is used, then update speed is maintained, but asymmetry in resistance switching increases
Solution Approach 1:
The counter-based control mechanism enables self-service symmetric updates by automatically managing both increment and decrement operations through digital counting. When a weight update is required, the counter can be incremented or decremented by the same amount, providing symmetric and consistent weight adjustments without the asymmetry inherent in conventional resistive switching operations.
3Productivity
If parallel processing is implemented, then training speed improves, but power consumption increases
Solution Approach 1:
The training process utilizes periodic action by implementing separate forward propagation and backward propagation phases. During forward propagation, inputs are processed through the network. During backward propagation, weight updates are performed based on error gradients. This periodic alternation between computation and update phases enables efficient parallel processing while managing power consumption by activating update mechanisms only when necessary.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution enhances ANN performance by optimizing power consumption, training speed, and efficiency, allowing for broader practical applications.
Implementation Method 1
each resistive circuit is associated with a respective single bit counter from the single bit counters. The resistive circuits are activated or deactivated according to a state of the associated single bit counter, and an electrical conductance of the resistor device is adjusted based at least in part on the resistive circuits that are activated.
Data Source
AI summary
Technical solutions are described for storing weight in a crosspoint device of a resistive processing unit (RPU) array. An example system includes a crosspoint array, wherein each array node represents a connection between neurons of the neural network, and wherein each node stores a weight assigned to the node. The crosspoint array includes a crosspoint device at each node. The crosspoint device includes a counter that has multiple single bit counters, and states of the counters represent the weight to be stored at the crosspoint device. Further, the crosspoint device includes a resistor device that has multiple resistive circuits, and each resistive circuit is associated with a respective counter from the counters. The resistive circuits are activated or deactivated according to a state of the associated counter, and an electrical conductance of the resistor device is adjusted based at least in part on the resistive circuits that are activated.