ANN to SNN Parameter Conversion via Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Converting an analog neural network (ANN) to a spiking neural network (SNN) is challenging due to the singularity of SNN operation, making it difficult to train and achieve recognition accuracy, while ANN has well-established training schemes and performance improvement methods.
Innovation Solution
A method to convert ANN to SNN by normalizing first parameters based on a reference activation, determining second parameters, and using a spiking mechanism where current membrane potential is calculated from previous potential, reception signal, and bias, with no threshold for output layer neurons, and a pooling layer transmitting spikes from the neuron with the greatest firing rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If SNN is trained directly using conventional training schemes, then training can be performed, but recognition accuracy is poor due to singularity of SNN operation
Solution Approach 1:
The patent applies preliminary action by first training an ANN using conventional training schemes to obtain optimal parameters, then converting these parameters to SNN parameters. This preliminary training of ANN provides a foundation for achieving high recognition accuracy in SNN without directly training the SNN, which would be difficult due to operational singularity.
Solution Approach 2:
The patent uses ANN as an intermediary system to achieve the desired outcome for SNN. By training ANN with conventional methods and then converting its parameters to SNN parameters, the ANN serves as a mediator that bridges the gap between conventional training schemes and SNN operation, enabling high accuracy without direct SNN training.
2Reliability
If ANN parameters are directly used for SNN, then conversion is simple, but recognition accuracy deteriorates due to parameter distribution mismatch
Solution Approach 1:
The patent applies parameter changes by transforming ANN parameters into SNN parameters through normalization and scaling operations. The conversion process modifies weight and bias parameters to account for differences in activation functions and operational characteristics between ANN and SNN, ensuring that the transformed parameters maintain optimal performance for SNN recognition tasks.
3Manufacturing precision
If reference activation is set to maximum activation, then normalization is simple, but information loss occurs due to singularity
Solution Approach 1:
The patent applies local quality by using different reference activation values for different layers of the neural network. Instead of using a single maximum activation value for all layers, the method selects appropriate reference values specific to each layer's activation distribution, preserving local characteristics and avoiding information loss while maintaining normalization precision.
Data Source
AI summary
A neural network conversion method and a recognition apparatus that implements the method are provided. A method of converting an analog neural network (ANN) to a spiking neural network (SNN) normalizes first parameters of a trained ANN based on a reference activation that is set to be proximate to a maximum activation of artificial neurons included in the ANN, and determines second parameters of an SNN based on the normalized first parameters.