RRAM Weight Update Precision via Digital Intermediary Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hardware neural networks (HNNs) using Resistive Random Access Memory (RRAM) synapses face limitations due to limited precision, asymmetric, and nonlinear weight updates, which affect the accuracy and efficiency of weight adjustments in software-defined neural networks.
Innovation Solution
Implementing a hardware neural network device with a crossbar RRAM array and nonlinear activation functions, where weight updates are adjusted by applying voltage pulses based on pulse numbers calculated from input signals and backpropagation values, effectively shifting and scaling weight sums to improve precision and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weight updates are stored directly in RRAM synapses based on pulse counts, then hardware implementation efficiency is improved, but weight precision and accuracy deteriorate due to limited precision and asymmetric nonlinear updates
Solution Approach 1:
The patent introduces digital memory cells as intermediary storage between the RRAM synapses and the control logic. These digital memory cells store the weight update values with full precision before they are applied to the RRAM synapses, acting as a buffer that decouples the precision requirements from the hardware implementation constraints.
Solution Approach 2:
The system dynamically adjusts the weight update process by using different precision levels at different stages: high precision in digital memory for calculation and storage, then controlled conversion to the RRAM synapse representation. This dynamic approach allows the system to maintain accuracy while adapting to the hardware constraints of RRAM devices.
2Device complexity
If asymmetric nonlinear weight updates are applied in RRAM synapses, then hardware neural network operation is simplified, but classification accuracy and immunity to asymmetric nonlinearity deteriorate
Solution Approach 1:
The patent implements feedback mechanisms where the system monitors the weight update process and adjusts the pulse generation and application accordingly. This feedback loop compensates for the asymmetric nonlinear characteristics of RRAM by dynamically adjusting the update strategy based on observed performance and error patterns.
Solution Approach 2:
The system changes operational parameters such as pulse amplitude, duration, and timing based on the current state of the RRAM synapses and the desired weight update. By dynamically adjusting these parameters, the system can compensate for asymmetric nonlinear effects and maintain classification accuracy despite the simplified hardware operation.
3Ease of manufacture
If bounded weight states are used in RRAM synapses, then device manufacturing and operation become easier, but weight update precision and range are limited
Solution Approach 1:
The patent segments the weight representation into multiple RRAM synapses, each handling a portion of the weight value. This segmentation allows the system to overcome the bounded state limitation of individual RRAM devices by distributing the weight information across multiple devices, thereby achieving higher effective precision while maintaining ease of manufacture at the device level.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the classification accuracy and immunity to asymmetric nonlinearity, improving the overall performance of hardware neural networks by allowing precise and efficient weight adjustments.
Implementation Method 1
The weight state of an RRAM-based (Resistive Random Access Memory, RRAM) synapse has limited precision and is bounded. Furthermore, the weight update is asymmetric and nonlinear.
Data Source
AI summary
A method includes: generating a first sum value at least by a first resistor; generating a first shifted sum value based on the first sum value and a nonlinear function; generating a pulse number based on the first shifted sum value; and changing the first resistor based on the pulse number to adjust the first sum value.