Neural Network Error Detection at Analog Memory Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large neural networks face efficiency and accuracy issues due to write reliability problems in analog memory, leading to increased training time and reduced performance, as conventional stochastic error injection methods are inefficient and device-dependent.
Innovation Solution
Implementing per-layer error detection using a residue-based mechanism that allows immediate retraining of layers with error thresholds, reducing training epochs and overhead, and employing a trainable error detection unit to improve accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If analog memory is used to implement neural network, then energy consumption is reduced and efficiency is improved, but write reliability deteriorates due to device variation and noise
Solution Approach 1:
The patent applies preliminary action by performing error detection at the layer level during training, before errors can propagate and cause significant accuracy degradation. The system proactively identifies write errors in analog memory using residue computation and comparison, allowing immediate retraining of affected layers without waiting for backpropagation to reveal errors at the output level.
Solution Approach 2:
The patent introduces an intermediary error detection mechanism that sits between the analog memory and the neural network computation. This intermediary layer uses residue-based verification to detect write errors without disrupting the energy-efficient analog computation, acting as a mediator that preserves both the low energy consumption and the reliability of the system.
2Reliability
If conventional stochastic error injection is used, then error robustness is improved, but training time increases and device-dependency worsens
Solution Approach 1:
Instead of waiting until the end of training to evaluate error robustness, the patent performs preliminary error detection at each layer during the training process. This allows the system to identify and correct errors early, reducing the total training time compared to conventional methods that only detect errors after complete training epochs.
Solution Approach 2:
The error detection unit is designed to be trainable and adaptive, allowing the system to self-adjust to device-specific characteristics. The residue-based mechanism automatically adapts to the particular analog memory device being used, eliminating device-dependency while maintaining error robustness without requiring external calibration or manual tuning.
3Productivity
If per-layer error detection is implemented, then training time is reduced and accuracy is improved, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the error detection process into layer-level segments rather than performing a single end-to-end error check. Each layer has its own error detection unit that independently verifies computations, which reduces the overall training time by parallelizing error detection across layers while keeping each individual detection unit relatively simple.
Solution Approach 2:
The error detection mechanism uses residue copying and comparison - creating a simplified copy of the computation in the residue domain and comparing it with the original. This copying approach enables efficient error detection without requiring complex verification logic, as the residue-based copy can be computed and compared using simple arithmetic operations.
Data Source
AI summary
A processing system performs error detection at each of a plurality of layers of a neural network, such as a neural network implemented at a computational analog memory. By performing error detection at the layer level, the processing system is able to account for write errors when updating neural network weights, without waiting for backpropagation based on an output of the neural network. The processing system thereby reduces the amount of time needed to train the network, both by reducing the number of training epochs, and by reducing the length of the individual training epochs.