Neural Network Device Sigmoidal STDP Weight Transition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing neural network devices using stochastic spike timing dependent plasticity (STDP) with binary synaptic weights face challenges in memory retention, as the stored patterns are easily overwritten during additional learning, leading to deteriorated memory properties.
Innovation Solution
A neural network device employing a stochastic action part with a cumulative probability distribution that exhibits a sigmoidal function behavior, moderating the weight transition to protect existing memories, thereby improving memory retention by using a series-connected switch or stochastic counter configuration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If stochastic STDP with binary synaptic weights is used for learning, then learning efficiency is improved, but memory retention deteriorates as stored patterns are easily overwritten during additional learning
Solution Approach 1:
The patent changes the parameter of cumulative probability distribution from exponential function to sigmoidal function. This parameter change modifies the weight transition characteristics, making weight changes more gradual and controlled, thereby preventing existing memories from being easily overwritten while maintaining learning efficiency
Solution Approach 2:
The patent introduces a dynamic control mechanism where the cumulative probability of weight transition is modulated by the sigmoidal function based on the number of signal input times. This dynamic adjustment allows the system to adapt weight transition probability during learning, protecting stored patterns during additional learning while enabling effective learning
2Adaptability or versatility
If continuous weight changes by STDP are used, then learning capability is maintained, but memory properties deteriorate with increased forgetting number
Solution Approach 1:
The patent changes the weight representation from continuous values to binary values (0 or 1), and modifies the cumulative probability distribution from exponential to sigmoidal function. This parameter change reduces information loss by making weight transitions more controlled and less prone to forgetting during additional learning
Solution Approach 2:
The patent uses binary synaptic weights instead of continuous weights, which simplifies the memory structure and reduces the complexity of weight management. The binary nature of weights makes the system more robust against forgetting while maintaining learning capability through the sigmoidal-modulated stochastic transitions
Data Source
AI summary
According to one embodiment, there is provided a neural network device including a neuron, a conversion part, a transmission part, a control part and a holding part. The conversion part converts a spike signal to a synapse current according to weight. The transmission part transmits the converted synapse current to the neuron. The control part determines transition of a state of the weight. The holding part holds the weight as a discrete state according to the determined transition of the state. The holding part includes an action part that stochastically operates based on a signal input from the control part to cause transition of the state of the weight. A cumulative probability of actions of the action part changes in a sigmoidal shape with respect to number of signal input times.