Synaptic Weight Representation Using Signed Analog Conductance-Pairs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial Neural Networks (ANNs) face performance degradation due to nonlinearity and asymmetry in conductance responses of non-volatile memory (NVM) elements, particularly in phase-change memory (PCM) devices, leading to reduced accuracy in tasks like MNIST digit recognition, as large synaptic weights are fragile and difficult to maintain.
Innovation Solution
The method involves representing synaptic weights using multiple conductance-pairs of varying significance, where a portion of the weight is periodically transferred from a lower-significance pair to a higher-significance pair during training, allowing the network to maintain stability and accuracy by mitigating the effects of nonlinearity and asymmetry in conductance responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single conductance-pair is used to represent synaptic weight, then device complexity is reduced, but large synaptic weights become fragile and accuracy deteriorates due to nonlinearity and asymmetry in conductance responses
Solution Approach 1:
The synaptic weight representation is segmented into multiple conductance-pairs with different significance levels (e.g., most significant pair and least significant pair). Each pair contributes differently to the overall weight value, allowing the system to represent a wider dynamic range while mitigating the effects of nonlinearity and asymmetry in individual conductance responses. The most significant pair handles large weight values while the least significant pair fine-tunes smaller adjustments.
2Manufacturing precision
If multiple conductance-pairs of varying significance are used, then synaptic weight accuracy and dynamic range are improved, but device complexity increases
Solution Approach 1:
Different conductance-pairs are assigned different significance levels and functional roles. The most significant conductance-pair is optimized for representing large weight values and coarse adjustments, while the least significant conductance-pair is optimized for small weight values and fine adjustments. This local differentiation allows each pair to operate in its optimal range, improving overall accuracy without requiring all pairs to have identical complexity.
3Reliability
If conductance values are periodically transferred from lower-significance to higher-significance pairs, then large weight values are protected from unintended loss, but training time increases
Solution Approach 1:
The system performs periodic transfer of conductance values from lower-significance pairs to higher-significance pairs at predetermined intervals during training. This periodic action serves two purposes: it consolidates accumulated weight changes into the most significant pair (protecting against unintended loss due to nonlinearity), and it maintains training progress by ensuring accurate weight representation. The periodic nature balances the overhead of transfer operations with the benefit of improved weight stability.
Data Source
AI summary
Artificial neural networks (ANNs) are a distributed computing model in which computation is accomplished with many simple processing units, called neurons, with data embodied by the connections between neurons, called synapses and by the strength of these connections, the synaptic weights. An attractive implementation of ANNs uses the conductance of non-volatile memory (NVM) elements to record the synaptic weight, with the important multiply—accumulate step performed in place, at the data. In this application, the non-idealities in the response of the NVM such as nonlinearity, saturation, stochasticity and asymmetry in response to programming pulses lead to reduced network performance compared to an ideal network implementation. A method is shown that improves performance by distributing the synaptic weight across multiple conductances of varying significance, implementing carry operations between less-significant signed analog conductance-pairs to more-significant analog conductance-pairs.