Neural Network Resistive Processing Unit Conductance Drift Mitigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network weights implemented using resistive processing units (RPUs) experience conductance drift over time due to structural relaxation and defect annealing, leading to accuracy loss and increased energy consumption.
Innovation Solution
A correction factor is applied to neuron inputs in the neural network based on the conductance drift coefficient and elapsed time since the RPU weights were programmed, compensating for conductance drift, and quantization of weight values is used to make them less sensitive to drift, with periodic re-quantization to track and adjust for drift.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If RPU weights are used in neural networks, then processing speed and energy efficiency are improved, but conductance drift occurs over time causing accuracy loss
Solution Approach 1:
The patent applies a correction factor to neuron inputs based on the conductance drift coefficient and elapsed time before calculations are performed. This preliminary correction compensates for the expected drift that will occur since the RPU weights were programmed, thereby maintaining accuracy while using RPU for fast processing
Solution Approach 2:
The patent changes the parameter of neuron inputs by applying a correction factor that is a function of time and drift coefficient. This transforms the input parameters to account for the temporal drift characteristics of RPU conductance, resolving the contradiction between using RPU for speed and maintaining accuracy
2Use of energy by moving object
If RPU weights are used in neural networks, then energy consumption is reduced, but conductance drift leads to accuracy degradation
Solution Approach 1:
The correction factor is calculated and applied in advance based on the drift coefficient and elapsed time, compensating for conductance drift before it significantly impacts accuracy. This allows the system to maintain reliability while benefiting from the low energy consumption of RPU
Solution Approach 2:
The system uses the drift coefficient as a feedback parameter to continuously adjust the correction factor applied to neuron inputs. This feedback mechanism ensures accuracy is maintained despite the passive, low-energy nature of RPU that cannot actively compensate for drift
3Reliability
If correction factors are applied to compensate for conductance drift, then accuracy is maintained, but additional computational steps are required
Solution Approach 1:
The correction factor is applied locally at the neuron input stage rather than requiring global recalibration of the entire RPU array. This localized correction maintains accuracy while minimizing the additional computational complexity by focusing the correction where it is most needed
Data Source
AI summary
Methods and systems for performing calculations with a neural network include determining a conductance drift coefficient for resistive processing unit (RPU) weights in a neural network. A correction factor is applied to neuron inputs in the neural network in accordance with the drift coefficient and a time that has elapsed since the RPU weights were programmed. A calculation is performed with the neural network. The correction factor compensates for conductance drift.