Counter-Based RPU Crosspoint Weights for Symmetric ANN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing crosspoint devices in artificial neural networks (ANNs) face challenges such as significant device-to-device variability and asymmetry in 'set' and 'reset' operations, which hinder efficient training and power management, leading to suboptimal training speed and efficiency.
Innovation Solution
The implementation of counter-based crosspoint devices with digital counters and resistive circuits that adjust conductance to represent weights, enabling symmetric up-counting and down-counting and robustness through CMOS technology, facilitating stochastic parallel updates and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional crosspoint devices are used in ANN, then device simplicity is maintained, but device-to-device variability increases and training efficiency deteriorates
Solution Approach 1:
The crosspoint device is segmented into multiple functional components: a digital counter unit with multiple flip-flops, a weight storage unit, and a conductance adjustment unit with multiple resistive circuits. Each component performs a specific function, and their coordinated operation achieves precise weight control while reducing device-to-device variability.
Solution Approach 2:
A digital counter unit is introduced as an intermediary between the control signal and the resistive circuits. The counter receives update signals, stores weight values in flip-flops, and controls the activation state of corresponding resistive circuits based on counter values, thereby mediating the weight adjustment process and improving training efficiency.
2Productivity
If asymmetric set and reset operations are used in crosspoint devices, then device operation simplicity is maintained, but training efficiency deteriorates due to asymmetric update operations
Solution Approach 1:
The patent intentionally introduces asymmetry in the operational mechanism to achieve symmetry in performance. The counter-based architecture with bidirectional update capability (increment and decrement operations) allows symmetric weight updates in both positive and negative directions, eliminating the asymmetry problem of traditional set/reset operations while improving training speed.
3Productivity
If digital counter-based crosspoint devices are implemented, then training efficiency is improved, but device complexity increases
Solution Approach 1:
The digital counter unit serves multiple functions: it stores weight values in flip-flops, generates control signals for resistive circuit activation, and enables bidirectional weight updates. This multi-functionality reduces the need for separate components, thereby improving training efficiency while limiting the increase in overall device complexity.
Solution Approach 2:
The weight storage unit (flip-flops) is nested within the counter unit, and the resistive circuits are nested within the conductance adjustment unit. This nested structure allows compact integration of multiple functions within a unified architecture, improving training efficiency while controlling device complexity through space-efficient design.
4Use of energy by moving object
If conventional weight storage methods are used, then power consumption is higher due to charge-loss issues, but device simplicity is maintained
Solution Approach 1:
The digital counter unit with flip-flops autonomously maintains weight values without requiring external refresh operations. The flip-flops inherently retain their state (0 or 1) as long as power is supplied, eliminating charge-loss issues and reducing power consumption compared to conventional capacitor-based storage that requires continuous refreshing.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the training speed and efficiency of ANNs by minimizing device-to-device variability, eliminating charge-loss issues, and optimizing power usage, while allowing for broader applications and improved performance.
Implementation Method 1
adjusting electrical conductance of a resistor device of the crosspoint device. The resistor device includes a set of resistive circuits, and each resistive circuit is associated with a respective single bit counter from the set of single bit counters. The electrical conductance of the resistor device is adjusted by activating or deactivating each resistive circuit according to a state of the associated single bit counter.
Data Source
AI summary
Technical solutions are described for storing weight in a crosspoint device of a resistive processing unit (RPU) array. An example method includes setting a state of each single bit counter from a set of single bit counters in the crosspoint device, the states of the single bit counters representing the weight to be stored at the crosspoint device. The method further includes adjusting electrical conductance of a resistor device of the crosspoint device. The resistor device includes a set of resistive circuits, each resistive circuit associated with a respective single bit counter from the set of single bit counters, the electrical conductance adjusted by activating or deactivating each resistive circuit according to a state of the associated single bit counter.