Input Scaling for Spiking Neural Network Stability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial spiking neural networks face instability and reduced sensitivity due to overwhelming spiking inputs from numerous connections, leading to burst spiking and the need for complex connection parameter manipulation to prevent network instabilities.
Innovation Solution
The method involves scaling individual inputs using a transformation function, such as a concave function like logarithm or power law, to produce scaled inputs that are combined and used to update the node state, allowing the network to generate a response only when the updated state breaches a threshold, thereby managing connection efficacy based on time intervals and input magnitudes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple spiking inputs are received from numerous connections, then the network can process more information, but the neuron dynamic process becomes overwhelmed causing burst spiking and reduced sensitivity
Solution Approach 1:
The patent applies parameter changes by transforming the input magnitude range using a concave function. This transformation modifies the parameters of the input signals to compress the dynamic range, allowing the neuron to handle inputs from numerous connections without becoming overwhelmed. The transformation function changes the relationship between input magnitude and neuronal response, preventing burst spiking while maintaining stability.
2Quantity of substance
If multiple spiking inputs are received from numerous connections, then the network can process more information, but the neuron sensitivity to individual inputs is reduced
Solution Approach 1:
The concave transformation function changes the parameter mapping between input magnitude and neuronal response. By applying this transformation, small differences in input magnitude are amplified in the transformed space, thereby enhancing the neuron's sensitivity to individual inputs even when receiving signals from many connections. This allows precise discrimination of input strength while handling high connection counts.
Solution Approach 2:
The patent introduces a transformation dimension by mapping inputs from one magnitude range to another through a concave function. This dimensional transformation allows the system to preserve sensitivity information that would otherwise be lost in the original magnitude space, enabling the neuron to distinguish individual input contributions despite the large number of connections.
3Reliability
If connection parameters are manipulated to prevent network instabilities, then network stability is improved, but the device complexity increases
Solution Approach 1:
The patent extracts the instability problem from the connection parameter manipulation and relocates it to the input transformation stage. By applying the concave transformation function to inputs before they reach the neuron dynamic process, the system prevents burst spiking and instability without requiring complex manipulation of connection parameters. This separates the stability control function from the connection weight adjustments.
Data Source
AI summary
Apparatus and methods for processing inputs by one or more neurons of a network. The neuron(s) may generate spikes based on receipt of multiple inputs. Latency of spike generation may be determined based on an input magnitude. Inputs may be scaled using for example a non-linear concave transform. Scaling may increase neuron sensitivity to lower magnitude inputs, thereby improving latency encoding of small amplitude inputs. The transformation function may be configured compatible with existing non-scaling neuron processes and used as a plug-in to existing neuron models. Use of input scaling may allow for an improved network operation and reduce task simulation time.