Analog MAC-Aware Neural Network Training for Noisy Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face inefficiencies in power consumption and inference time due to the intrinsic electrical noise in analog multiply-and-accumulation circuits, as they are typically trained on clean datasets without accounting for noise, leading to suboptimal weight parameters during inference.
Innovation Solution
Injecting noise into the training process of neural networks to emulate the intrinsic noise in analog multiply-and-accumulation circuits, integrating it into the loss function to determine weight parameters that account for this noise, thereby optimizing the neural network's performance in noisy environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural networks are trained on clean datasets without noise, then training convergence is easier and faster, but the weight parameters are suboptimal for inference on noisy analog hardware
Solution Approach 1:
The patent applies preliminary action by injecting noise into the training dataset before training begins. This pre-conditioning of the training data with noise allows the neural network to learn weight parameters that are robust to noise, so when deployed on noisy analog hardware, the inference accuracy is improved without requiring longer training times to converge
2Measurement precision
If analog multiply-and-accumulation circuits operate at higher precision, then computational accuracy is improved, but power consumption increases
Solution Approach 1:
The patent applies parameter changes by optimizing the precision parameters of the analog multiply-and-accumulation circuits. Instead of using high precision throughout, the system adjusts precision levels dynamically or selectively in different parts of the network, achieving acceptable accuracy while significantly reducing power consumption
3Reliability
If all weight parameters are maintained with full precision, then computational accuracy is maintained, but computational resources and power are wasted
Solution Approach 1:
The patent applies local quality by assigning different precision levels to different weight parameters based on their importance and impact on output accuracy. Less critical weights are represented with lower precision, reducing overall computational resource usage and power consumption, while critical weights maintain higher precision to preserve accuracy
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments described herein are directed to training techniques to reduce the power consumption and decrease the inference time of an NN. For example, during training, an estimate of power consumed by AMACs of a hardware accelerator on which the NN executes during inferencing is determined. The estimate is based at least on the non-zero midterms generated by the AMACs and the precision thereof. A loss function of the NN is modified such that it formulates the non-zero midterms and the precision thereof. The training forces the modified loss function to generate a sparse bit representation of the weights of the NN and to reduce the precision of the AMACs. Noise may also be injected at the output of nodes of the NN that emulates noise generated at an output of the AMACs. This enables the weights to account for the intrinsic noise that is experienced by the AMACs during inference.