Synaptic Weight Transfer with Polarity Inversion for NVM Asymmetry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial Neural Networks (ANNs) face performance degradation due to nonlinearity and asymmetry in conductance responses of Non-Volatile Memory (NVM) elements, leading to reduced accuracy in tasks like MNIST digit recognition, as large synaptic weights are fragile and difficult to maintain.
Innovation Solution
The method involves representing synaptic weights as a weighted current flow from multiple conductance-pairs of varying significance, where a portion of the weight from a lower-significance pair is transferred to a higher-significance pair, and the polarity of the lower-significance pair is inverted during training cycles to mitigate fixed device asymmetry, ensuring the total synaptic weight remains substantially unchanged.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If signed analog conductance-pairs are used to represent synaptic weights, then high-density neuromorphic implementation is achieved, but fixed device asymmetry causes accuracy degradation
Solution Approach 1:
The synaptic weight is segmented into multiple conductance-pairs of varying significance (higher-significance and lower-significance pairs). This segmentation allows the system to distribute the weight representation across multiple devices, reducing the impact of asymmetry in any single device while maintaining high computational density.
Solution Approach 2:
The polarity of the lower-significance conductance-pair is periodically inverted during training cycles. This parameter change (polarity inversion) mitigates the cumulative effect of fixed device asymmetry by alternating the direction of asymmetry errors, causing them to cancel out over time and improving overall weight representation accuracy.
2Manufacturing precision
If conductance-pairs of varying significance are used to represent synaptic weights, then device asymmetry effects are reduced, but additional weight transfer operations are required
Solution Approach 1:
The polarity inversion of the lower-significance conductance-pair is performed periodically at specific intervals during training cycles, rather than continuously. This periodic action reduces the complexity of control operations while still achieving the benefit of asymmetry cancellation, as the system only needs to perform polarity swaps at designated transfer intervals.
Solution Approach 2:
Weight transfer operations are performed in advance at scheduled intervals to prepare the conductance-pairs for the next training cycle. This preliminary action ensures that the weight representation is optimized before training resumes, reducing the need for complex real-time adjustments during active training.
3Manufacturing precision
If polarity inversion is applied to lower-significance conductance-pairs, then fixed device asymmetry is canceled out, but training cycle structure becomes more complex
Solution Approach 1:
The polarity inversion is integrated into the periodic training cycle structure, where weight transfer and polarity inversion occur at predetermined intervals. This approach embeds the complexity into a regular, predictable pattern that is easier to manage than continuous or ad-hoc adjustments, as the system follows a consistent rhythm of training, transfer, and inversion.
Solution Approach 2:
The polarity inversion operation is merged with the existing weight transfer process, so that both operations are performed together during the same training cycle interval. This merging reduces the overall structural complexity by combining multiple operations into a single coordinated event, rather than treating them as separate, independent processes.
Data Source
AI summary
Artificial neural networks (ANNs) are a distributed computing model in which computation is accomplished with many simple processing units, called neurons, with data embodied by the connections between neurons, called synapses, and by the strength of these connections, the synaptic weights. An attractive implementation of ANNs uses the conductance of non-volatile memory (NVM) elements to record the synaptic weight, with the important multiply—accumulate step performed in place, at the data. In this application, the non-idealities in the response of the NVM such as nonlinearity, saturation, stochasticity and asymmetry in response to programming pulses lead to reduced network performance compared to an ideal network implementation. A method is shown that improves performance by periodically inverting the polarity of less-significant signed analog conductance-pairs within synaptic weights that are distributed across multiple conductances of varying significance, upon transfer of weight information between less-significant signed analog conductance-pairs to more-significant analog conductance-pairs.