Conversion-Aware SNN Training Reduces Data Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ANN-to-SNN conversion techniques suffer from data loss due to the difference in data representation methods between analog artificial neural networks (ANNs) and spiking neural networks (SNNs), leading to increased hardware burden and energy inefficiency.
Innovation Solution
A conversion aware training method and system that generates an SNN model by simulating a spiking neural network using activation functions like ReLU, Clip, and Time to First Spike (TTFS) on an analog ANN model, correcting parameters and weights to minimize data loss and improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional ANN-to-SNN conversion techniques are used, then the conversion process is simple, but data loss occurs due to difference in data representation methods
Solution Approach 1:
The patent applies preliminary action by performing conversion-aware training that simulates SNN behavior during the ANN training phase. This allows the model to adapt to SNN data representation methods before actual conversion, minimizing data loss during the ANN-to-SNN transition while maintaining a manageable conversion process.
Solution Approach 2:
The patent introduces an intermediary simulation process that acts as a bridge between ANN and SNN. By simulating SNN characteristics within the ANN training framework, the system enables gradual adaptation to SNN data representation without direct, loss-prone conversion.
2Loss of information
If ANN training model is directly applied to SNN, then training process is efficient, but data loss occurs during conversion
Solution Approach 1:
The conversion-aware training performs preliminary adaptation by simulating SNN behavior during ANN training. This preliminary action minimizes data loss during conversion while maintaining training efficiency by avoiding the need for complete retraining after conversion.
Solution Approach 2:
The patent implements feedback by using simulation results to guide the training process. The simulated SNN behavior provides feedback signals that help adjust ANN weights and parameters, reducing data loss during conversion while maintaining efficient training through iterative optimization.
3Use of energy by moving object
If SNN operation is implemented, then low-power hardware operation is achieved, but training accuracy decreases due to data loss
Solution Approach 1:
The patent applies preliminary action by performing conversion-aware training that simulates SNN behavior during the ANN training phase. This allows the model to adapt to SNN data representation methods before actual conversion, minimizing data loss during the ANN-to-SNN transition while maintaining a manageable conversion process.
4Loss of information
If conversion aware training with simulation is applied, then data loss is minimized, but training time and computational resources increase
Solution Approach 1:
The patent applies partial action by implementing conversion-aware training selectively during the ANN training phase rather than requiring complete retraining after conversion. This partial approach minimizes data loss while avoiding the excessive time cost of full retraining, achieving an optimal balance between accuracy and efficiency.
Data Source
AI summary
Disclosed are a spiking neural network training method based the conversion aware training and a system thereof. The spiking neural network training method includes an ANN generation operation of generating an analog artificial neural network (ANN) model and inputting variable data, a conversion aware training operation of simulating a spiking neural network (SNN) model by using one or more activation functions with respect to the analog ANN model, and an SNN generation operation of generating the SNN model by correcting parameters and weights of layers based on a result of the simulation.