Neural Network Parameter Training for Device Mismatch Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network accelerators, particularly neuromorphic NN accelerators, suffer from device mismatch-induced random perturbations of configuration parameters, leading to reduced task performance and challenges in large-scale deployment due to time- and energy-consuming chip-specific training methods and increased power consumption from calibration circuitry.
Innovation Solution
A neural network configuration parameter training method that incorporates a robustness loss function to adapt to generalized device mismatch, involving steps to initialize and update parameters while considering a causal relationship between original and attacked configurations, optimizing for both performance and robustness without additional hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If chip-specific training is performed for each mixed-signal NN chip to solve device mismatch, then task performance is improved, but training time and energy consumption increase significantly
Solution Approach 1:
The patent applies preliminary action by performing robustness training in advance during the model training phase. The method simulates device mismatch attacks and updates configuration parameters beforehand to compensate for expected variations. This preliminary robustness enhancement eliminates the need for time-consuming chip-specific training later, as the model is pre-adapted to handle device variations across different chips.
2Reliability
If calibration circuitry is added to ensure standard behavior of neurons and synapses, then device mismatch is compensated, but power consumption and chip area increase
Solution Approach 1:
The patent replaces the mechanical/hardware-based calibration circuitry with a software/algorithm-based solution. Instead of adding physical circuits to measure and adjust configuration parameters, the method uses simulated attacks during training to update configuration parameters computationally. This substitution eliminates the need for additional calibration hardware, reducing both power consumption and chip area while achieving the same goal of compensating for device mismatch.
3Reliability
If calibration circuitry is added to ensure standard behavior, then device mismatch is handled, but chip area and manufacturing cost increase
Solution Approach 1:
The patent replaces the mechanical/hardware-based calibration circuitry with a software/algorithm-based solution. Instead of adding physical circuits to measure and adjust configuration parameters, the method uses simulated attacks during training to update configuration parameters computationally. This substitution eliminates the need for additional calibration hardware, reducing both power consumption and chip area while achieving the same goal of compensating for device mismatch.
4Device complexity
If subset of configuration parameters is attacked and optimized in each iteration, then computational complexity is reduced, but systematic worst-case attack simulation becomes difficult
Solution Approach 1:
The patent applies partial action by attacking and optimizing only a subset of configuration parameters in each iteration rather than all parameters. This reduces computational complexity while still achieving systematic worst-case attack simulation through multiple iterations. The method strategically selects subsets of parameters to attack, ensuring comprehensive coverage of potential mismatch scenarios across iterations without requiring exhaustive optimization of all parameters simultaneously.
Data Source
AI summary
A neural network (NN) configuration parameter training and deployment method and apparatus are disclosed. The method and the apparatus include searching for simulated attacked NN configuration parameters on a basis of NN configuration parameters, so that the attacked NN configuration parameters move in a direction of maximal divergence from an NN output result corresponding to the NN configuration parameters; taking a difference in an NN output result between the current NN configuration parameters and the attacked NN configuration parameters as a robustness loss function which serves as a part of a total loss function; and finally, optimizing the NN configuration parameters on a basis of the total loss function. Especially for sub-threshold and mixed-signal circuits with ultra-low power consumption, the solution can solve a problem of perturbations of configuration parameters caused by device mismatch, and achieve the technical effect of low-cost and high-efficiency deployment of parameters of NN accelerators.