Spiking Neural Network Training With Competitive RSTDP Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural networks using reward modulated spike time dependent plasticity (RSTDP) tend to favor frequent stimuli over infrequent ones, leading to poor performance when encountering less common stimuli due to weight saturation and biased training.
Innovation Solution
Implementing competitive reward modulated spike time dependent plasticity (cRSTDP) with weight normalization, where the sum of synaptic input and output weights is maintained constant, and using a zero-sum reward training method to prevent weight saturation and ensure equal learning from both common and rare experiences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If RSTDP is used to train the neural network, then learning from frequent stimuli is improved, but performance on infrequent stimuli deteriorates due to weight saturation
Solution Approach 1:
The patent changes the parameter constraints on synaptic weights by enforcing normalization conditions (constant input sum and constant output sum) on the weight matrices. This prevents weights from saturating at maximum values and allows the network to maintain sensitivity to both frequent and infrequent stimuli throughout training.
Solution Approach 2:
The patent implements a reward signal that provides feedback to the learning process. The reward modulates the STDP plasticity rules, allowing the network to learn from both frequent and infrequent stimuli by reinforcing correct responses regardless of stimulus frequency, thereby improving reliability on rare events.
2Productivity
If training rewards favor frequent stimuli, then learning speed on common inputs is improved, but adaptability to rare stimuli deteriorates
Solution Approach 1:
The patent modifies the reward structure by normalizing it to have zero mean, which changes how rewards are distributed during training. This prevents systematic bias toward frequent stimuli and allows the network to adapt equally well to both common and rare inputs, improving overall adaptability.
Solution Approach 2:
The patent replaces the traditional biased reward mechanism with a normalized reward system that has different statistical properties (zero mean). This substitution changes the learning dynamics to be more balanced across different stimulus frequencies, enhancing adaptability to rare events while maintaining learning speed on common inputs.
Data Source
AI summary
A method of training an artificial neural network having a series of layers and at least one weight matrix encoding connection weights between neurons in successive layers. The method includes receiving, at an input layer of the series of layers, at least one input, generating, at an output layer of the series of layers, at least one output based on the at least one input, generating a reward based on a comparison of between the at least one output and a desired output, and modifying the connection weights based on the reward. Modifying the connection weights includes maintaining a sum of synaptic input weights to each neuron to be substantially constant and maintaining a sum of synaptic output weights from each neuron to be substantially constant.