Linear Resistive Processing Unit Circuit for Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training deep neural networks (DNNs) is computationally intensive and requires massive resources, with limitations in scaling due to device specifications in non-volatile memory cells, and existing solutions are hindered by non-ideal device characteristics like asymmetrical set and reset operations.
Innovation Solution
A CMOS-based resistive processing unit (RPU) circuit with series-connected current mirrors and a capacitor, allowing for highly linear and symmetric weight updates through voltage pulse control, enabling efficient neural network training by local data storage and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If non-volatile memory cells are used for neural network training, then data storage and processing capability is improved, but device specifications limit acceleration performance
Solution Approach 1:
The patent changes the operating parameters of the memory cell by applying voltage pulses with specific amplitudes and durations to control the learning rate. By adjusting these temporal parameters, the system achieves desired acceleration performance while working within the intrinsic limitations of the non-volatile memory device specifications.
Solution Approach 2:
The patent employs periodic voltage pulsing to the memory cell, where multiple pulses with different amplitudes and durations are applied sequentially. This periodic action enables precise control of the weight update process and achieves high acceleration factors by efficiently utilizing the memory cell's switching characteristics.
2Ease of operation
If asymmetrical set and reset operations are used in NVM cells, then memory application requirements are met, but neural network training accuracy deteriorates
Solution Approach 1:
The patent implements feedback mechanisms through read operations that monitor the memory cell state during write operations. By measuring the actual weight changes and comparing them with desired values, the system adjusts subsequent pulse parameters to compensate for asymmetrical characteristics, thereby achieving linear and symmetric weight updates despite the inherent device asymmetry.
Solution Approach 2:
The patent dynamically adjusts voltage pulse parameters (amplitude, duration, timing) based on the operational mode required. Different parameter sets are applied for set and reset operations to compensate for device asymmetry, transforming the inherently asymmetrical NVM behavior into effectively symmetric weight updates suitable for neural network training.
3Productivity
If learning rate is increased to reduce training time, then productivity is improved, but accuracy deteriorates
Solution Approach 1:
The patent uses periodic voltage pulsing with varying amplitudes and durations to implement adaptive learning rates. By controlling the number, timing, and characteristics of pulses applied to the memory cell, the system achieves both fast convergence (high productivity) and high accuracy through optimized weight updates, effectively decoupling the traditional trade-off between training speed and accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The RPU circuit achieves significant acceleration in DNN training with reduced time complexity and improved accuracy, independent of array size, by controlling learning rates without increasing operational time or sacrificing accuracy.
Implementation Method 1
a capacitor connected with the at least two current mirrors, the capacitor providing a weight based on a charge level of the capacitor
Implementation Method 2
The capacitor is charged or discharged by one of the at least two current mirrors
Data Source
AI summary
A processing unit, including a first circuit, and a first circuit element connected to the first circuit. The first circuit element is at least charged by the first circuit.