Synaptic Array Device Moving Average Gradient Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Non-symmetric update characteristics in synaptic devices, such as Resistive RAM, Phase Change Memory, and Ferroelectric RAM, lead to inaccurate weight storage and deteriorated neural network learning performance due to varying conductance updates during weight increments and decrements.
Innovation Solution
A synaptic array device and artificial neural network learning method that calculates the average value of accumulated gradients and uses this average value as an offset to simplify the update process, reducing errors and ensuring learning performance closer to digital implementations by employing a moving average algorithm.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If synaptic devices are used to store weight values, then learning performance can be improved through hardware implementation, but weight computation errors occur due to non-symmetric update characteristics
Solution Approach 1:
The patent introduces a symmetry point compensation mechanism as an intermediary between the non-symmetric synaptic device and the weight storage requirement. By calculating and compensating for the symmetry point offset, the system mediates the non-symmetric device behavior to achieve accurate weight representation, resolving the contradiction between hardware productivity and computational precision
Solution Approach 2:
The patent dynamically adjusts the symmetry point parameter based on accumulated gradient values. By changing the reference parameter (symmetry point) according to device characteristics and training progress, the system adapts to non-symmetric behavior while maintaining weight computation accuracy, enabling both high productivity and precision
2Adaptability or versatility
If the symmetry point is not set to 0 in non-ideal devices, then device operational flexibility is maintained, but weight computation errors occur
Solution Approach 1:
The patent implements feedback by continuously monitoring gradient values and using them to update the symmetry point compensation parameter. This feedback loop allows the system to maintain operational flexibility with non-zero symmetry points while correcting weight computation errors through accumulated gradient information, resolving the contradiction between adaptability and precision
3Productivity
If analog array devices are used for synaptic arrays, then hardware efficiency is improved, but errors accumulate due to non-symmetric update characteristics
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing the symmetry point compensation parameter before weight updates occur. This preliminary compensation action prevents error accumulation during training, maintaining both hardware efficiency and learning performance stability by addressing non-symmetric issues before they propagate
Data Source
AI summary
A synaptic array device according to one embodiment of the present disclosure comprises a first synaptic array representing weight values, a second synaptic array receiving the error gradient of the weights of the first synaptic array and representing gradient values refined in row units, and a third synaptic array receiving the gradient values refined in row units from the second synaptic array and passing the portion of the received gradient values exceeding a threshold to the first synaptic array, wherein the third synaptic array derives a moving average value by averaging accumulated values of the gradient values received from the second synaptic array and passes the derived moving average value to the second synaptic array.