Memristor Neural Network Training for Error Mitigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memristor memory-based computing-in-memory neural networks face accuracy decreases due to manufacturing process errors and dynamic errors, which limit their computational acceleration capabilities for complex algorithms.
Innovation Solution
A neural network training method that decomposes memristor conductance values into target, global, local, and dynamic components, models these errors as Gaussian random variables, and uses Bayesian neural network training with variational inference to obtain a statistical representation of weights, effectively mitigating the impact of process and dynamic errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memristor memory is used for computing-in-memory neural network operations, then computational acceleration and energy efficiency are improved, but manufacturing process errors and dynamic errors cause accuracy degradation
Solution Approach 1:
The patent applies preliminary action by performing error characterization and statistical modeling before actual neural network inference. The method pre-characterizes process errors and dynamic errors to build error models that are then used during inference to compensate for expected inaccuracies, thereby maintaining high accuracy while utilizing memristor's computational acceleration capabilities
Solution Approach 2:
The patent introduces statistical models and error characterization frameworks as intermediaries between the memristor hardware and the neural network software. These intermediate layers translate physical error characteristics into computable corrections, allowing the system to account for manufacturing and dynamic errors without sacrificing the inherent speed advantages of in-memory computing
2Quantity of substance
If memristor conductance values are used to represent neural network weights, then storage density and computational parallelism are improved, but device parameter deviations reduce weight precision
Solution Approach 1:
The patent applies parameter changes by transforming the representation of neural network weights from deterministic values to statistical distributions. Instead of storing single precise conductance values, the system stores mean values and variance parameters that characterize the expected distribution of conductance values under process variations, enabling the system to maintain high storage density while accounting for manufacturing imprecision
Solution Approach 2:
The patent converts the harmful effect of device parameter deviations into a beneficial statistical model. By characterizing manufacturing variations and dynamic errors as predictable statistical distributions rather than random noise, the system transforms what would be accuracy-degrading imperfections into manageable parameters that can be compensated for through probabilistic inference methods
3Ease of operation
If conventional neural network training methods are used on memristor hardware, then implementation simplicity is maintained, but accuracy degradation due to errors increases
Solution Approach 1:
The patent applies segmentation by dividing the neural network training and inference process into distinct phases: error characterization, statistical modeling, and corrected inference. This segmentation allows each phase to be optimized independently while maintaining overall system simplicity, enabling the integration of error compensation without requiring complete redesign of the neural network implementation
Data Source
AI summary
The present invention discloses a neural network training method for a memristor memory for memristor errors, which is mainly used for solving the problem of decrease in inference accuracy of a neural network based on the memristor memory due to a process error and a dynamic error. The method comprises the following steps: performing modeling on a conductance value of a memristor under the influence of the process error and the dynamic error, and performing conversion to obtain a distribution of corresponding neural network weights; constructing a prior distribution of the weights by using the weight distribution obtained after modeling, and performing Bayesian neural network training based on variational inference to obtain a variational posterior distribution of the weights; and converting a mean value of the variational posterior of the weights into a target conductance value of the memristor memory.