Neuromorphic Memory Device Weight Error Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Non-volatile memory (NVM) based neuromorphic systems face issues such as weight fluctuations, program errors, and retention loss, which reduce the accuracy of Artificial Neural Networks (ANNs) due to errors in weight storage and retrieval.
Innovation Solution
A memory device and operation method that involve reading weight matrices, recalculating and updating mean and variance values of neuron values, and performing batch normalization to improve ANN accuracy by correcting weight errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If NVM is used to implement ANN, then weight storage is achieved, but weight fluctuations and program errors reduce accuracy
Solution Approach 1:
The patent implements a feedback mechanism where the read-out weight is compared with the actual stored weight, and the difference (error) is used to generate a compensation value. This compensation value is fed back to correct the read-out weight, forming a closed-loop system that continuously improves accuracy by eliminating the discrepancy between stored and read weights.
Solution Approach 2:
The patent changes the parameter representation by introducing a compensation value that adjusts the read-out weight. Instead of directly using the fluctuating read-out weight, the system transforms it by adding the compensation value, thereby changing the parameter state from inaccurate to accurate while maintaining the underlying NVM storage mechanism.
2Reliability
If NVM is used to implement ANN, then weight storage is achieved, but retention loss occurs over time
Solution Approach 1:
The patent performs preliminary action by pre-calculating and storing the compensation value before the retention loss fully manifests. The compensation value is generated based on the initial accurate weight and is stored alongside or associated with the weight data, ready to be applied when read-out occurs, thus preemptively counteracting the retention loss that will occur over time.
3Device complexity
If conventional weight storage is used, then simplicity is maintained, but accuracy is greatly reduced in front layers
Solution Approach 1:
The patent segments the weight storage and retrieval process into two independent components: the actual stored weight and the compensation value. This segmentation allows each component to serve a specific function—the stored weight maintains the original data while the compensation value independently corrects errors—thereby improving accuracy without significantly increasing overall system complexity.
Data Source
AI summary
Provided is an operation method for a memory device, the memory device being used for implementing an Artificial Neural Network (ANN). The operation method includes: reading from the memory device a weight matrix of a current layer of a plurality of layers of the ANN to extract a plurality of neuro values; determining whether to perform calibration; when it is determined to perform calibration, recalculating and updating a mean value and a variance value of the neuro values; and performing batch normalization based on the mean value and the variance value of the neuro values.