Spiking Neuron Network Training Threshold Adjustment

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Spiking neuron networks (SNNs) face challenges during training due to vanishing or exploding gradients when using the backpropagation rule, making it difficult to effectively train and process data.

Innovation Solution

The method involves adjusting the spiking firing threshold of a target layer in the SNN based on confidence during the forward propagation stage of training, using a backpropagation rule, to adaptively manage spiking firing sparsity and improve training efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If backpropagation rule is used for training SNN, then training can be performed, but vanishing or exploding gradients occur making training difficult

Engineering Contradiction:

Improvetraining feasibilityVSAvoidtraining stability

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The patent dynamically adjusts the spiking firing threshold parameter during the forward propagation stage based on confidence values from the previous time step. This parameter change prevents gradient vanishing or exploding by adapting the threshold to maintain appropriate spiking activity levels, thereby resolving the training stability issue while preserving training feasibility

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent implements a feedback mechanism where the confidence value from the previous time step is used to adjust the spiking firing threshold in the current time step. This feedback loop allows the network to adaptively regulate its spiking activity and maintain stable gradients throughout training, solving the reliability problem

Inventive Principle:

Principle #23Feedback

2Reliability

If spiking firing threshold is adjusted dynamically, then gradient stability improves, but calculation complexity increases

Engineering Contradiction:

Improvegradient stabilityVSAvoidcalculation complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent calculates the confidence value in advance during the forward propagation stage before the backward propagation stage begins. This preliminary action allows the spiking firing threshold to be pre-adjusted based on predicted spiking activity, avoiding the need for complex iterative adjustments during training and reducing overall calculation complexity while maintaining gradient stability

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11941526B2Methods, electronic devices, and computer-readable media for training, and processing data through, a spiking neuron network

Publication Date: 2024.03.26 LYNXI TECH CO LTD

AI summary

Provided is a method for training spiking neuron network, including: inputting a spiking signal corresponding to a training sample into a spiking neuron network to be trained; and training the spiking neuron network based on a back propagation rule, wherein a forward propagation stage of training includes: adjusting a spiking firing threshold corresponding to a target layer in the spiking neuron network according to confidence corresponding to the target layer. Also provided are a method for processing data based on a spiking neuron network, an electronic device, and a non-transitory computer-readable medium.