Neuromorphic Device Resilience to Formation Failures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training complex deep neural network (DNN) models is time-consuming and computationally intensive, requiring significant resources, and existing hardware acceleration techniques are inefficient in terms of power usage and computation resources.
Innovation Solution
A neuromorphic device architecture utilizing a 2D crossbar array of resistive processing unit (RPU) cells, which can perform all cycles of the backpropagation algorithm in parallel, reducing power consumption and computation resources, and a method to train inference models by introducing random defects to weights to account for potential resistive device failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional hardware acceleration techniques (CPU/GPU) are used for DNN training, then training can be performed with existing technology, but training time is excessive and power consumption is high
Solution Approach 1:
The patent replaces conventional digital computing hardware (CPU/GPU) with a neuromorphic device that mimics biological neural networks. This substitution enables parallel processing of forward propagation, backward propagation, and weight update operations simultaneously, achieving orders of magnitude speedup in DNN training while reducing power consumption.
Solution Approach 2:
The patent transitions from sequential processing in conventional hardware to spatial parallel processing in neuromorphic hardware. By mapping neural network operations onto a two-dimensional crossbar array of resistive processing units, the system performs multiple computation cycles (forward, backward, weight update) concurrently across different spatial locations, fundamentally changing the computational dimensionality.
2Productivity
If neuromorphic devices are used to accelerate DNN training, then training speed increases and power consumption decreases, but resistive devices may suffer from formation failures
Solution Approach 1:
The patent applies preliminary action by introducing artificial defects into the weight matrix during the training phase before deployment. By pre-training the network with simulated resistive device failures, the system learns to compensate for these defects, making the trained model robust and resilient when deployed on actual neuromorphic hardware with potential formation failures.
Solution Approach 2:
The patent implements preliminary anti-action by counteracting the potential harmful effect of formation failures through defect injection during training. By deliberately adding defects to the weight matrix beforehand, the system prepares countermeasures that neutralize the impact of actual device failures during inference, ensuring reliable operation despite hardware imperfections.
3Reliability
If random defects are added to weights during training, then the model becomes resilient to formation failures, but additional computation is required during the training process
Solution Approach 1:
The patent applies parameter changes by modifying the weight matrix parameters during training to include random defects. This transformation of the weight parameters from ideal values to defect-containing values enables the network to learn robust representations that tolerate hardware failures. The defect injection process systematically alters weight parameters according to a predefined defect model, integrating reliability training into the standard training pipeline.
Data Source
AI summary
A neuromorphic device includes a plurality of first control lines, a plurality of second control lines and a matrix of resistive processing unit cells. Each resistive processing unit cell is electrically connected with one of the first control lines and one of the second control lines. A given resistive processing unit cell includes a first resistive device and a second resistive device. The first resistive device is a positively weighted resistive device and the second resistive device is a negatively weighted resistive device.