Image Recognition Method of Deep Spiking Neural Network Based on Dynamic Threshold Neuron

Abstract

The image recognition method of the deep spiking neural network based on dynamic threshold neurons disclosed by the present invention belongs to the fields of brain-inspired computing and deep learning. Aiming at the problems of low pulse transmission rate, low firing rate and loss of conversion accuracy when implementing deep SNN by the existing DNN-to-SNN conversion technology. The implementation method of the present invention is as follows: First, train the DNN, obtain the weights and save them; then, based on the DNN-to-SNN conversion method, convert the activation function ReLU in the DNN into an IF neuron with a dynamic threshold in the SNN, normalize the DNN weights and map them to the SNN; finally, simulate and run the SNN, and calculate the threshold of each neuron at each time step. This method is applicable to artificial intelligence fields such as image classification and recognition, target recognition and tracking, can accelerate the pulse transmission rate, improve the pulse firing rate, accelerate information transmission, and at the same time reduce the conversion loss.

Description

Technical Field

[0001] The present invention relates to an image recognition method for a deep spiking neural network based on a dynamic threshold neuron model, and belongs to the fields of brain-inspired computing and deep learning. Background Art

[0002] Deep Neural Networks (DNN) based on highly simplified brain dynamics models, as powerful computing tools, have achieved remarkable results in many artificial intelligence fields such as image recognition, target recognition and tracking, speech recognition, and machine translation. However, problems such as extremely high computing requirements and high power consumption have greatly restricted the application scope of DNN.

[0003] Spiking Neural Networks (SNN) are essentially different from DNN. SNN works in an event-driven manner, and neurons transmit information through discrete spikes rather than continuous values. It has a high degree of biological imitation, can process dynamic data of spatio-temporal patterns, and has great potential and application prospects in application scenarios with high requirements for real-time performance and energy efficiency. However, the existing SNN learning algorithms have low training efficiency and inferior effects compared with deep learning models, resulting in low practicality of spiking neural networks.

[0004] Currently, the conversion technology of the pulse form of deep learning models (DNN-to-SNN) can avoid the problem of difficult direct training of spiking networks, and at the same time combines the advantages of easy training and high performance of deep learning models with the advantages of high real-time performance and high energy efficiency of spiking models, expanding the application scope of these two learning models. Although the DNN-to-SNN conversion technology based on frequency encoding achieves performance comparable to that of DNN, for deep SNN, there is still a certain conversion loss, and there are problems such as slow pulse backward propagation speed and low spike firing rate. Summary of the Invention

[0005] Aiming at the problems of low pulse transmission rate, low spike firing rate, and loss of conversion accuracy when implementing deep SNN with the existing DNN-to-SNN conversion technology, the purpose of the present invention is to provide an image recognition method for a deep spiking neural network based on dynamic threshold neurons, which can accelerate the pulse transmission rate, improve the spike firing rate, and reduce the conversion loss.

[0006] The purpose of the present invention is achieved by the following technical solutions:

[0007] The image recognition method of the deep spiking neural network based on dynamic threshold neurons disclosed by the present invention first trains a DNN, obtains and saves the weights; then, based on the DNN-to-SNN conversion method, converts the ReLU activation function in the DNN into an IF neuron with a dynamic threshold in the SNN, normalizes the DNN weights, and maps them to the SNN; finally, simulates and runs the SNN. This neuron model sets the threshold of each neuron at each time step. The neuron no longer needs to accumulate to reach a fixed threshold to emit a spike, and the high-level neurons can emit spikes earlier. Therefore, it can accelerate the spike transmission rate, increase the spike emission rate, and at the same time reduce the conversion loss.

[0008] The image recognition method of the deep spiking neural network based on dynamic threshold neurons disclosed by the present invention includes the following steps:

[0009] Step 1: Train the DNN, obtain and save the weights;

[0010] Use the backpropagation algorithm to train the DNN to make it have high accuracy;

[0011] Step 2: Based on the DNN-to-SNN conversion method, convert the ReLU activation function in the DNN into an IF neuron with a dynamic threshold in the SNN, normalize the DNN weights, and map them to the SNN, that is, establish the SNN.

[0012] Use the DNN-to-SNN conversion method to equivalently convert the ReLU function in the DNN into an IF neuron with a dynamic threshold in the SNN, thereby converting the DNN into an SNN with a similar structure. Use a data-based normalization method to normalize the DNN weight values, and map the normalized weight values to the SNN;

[0013] The specific conversion process is as follows:

[0014] The ReLU units of the DNN and the neurons of the SNN are in one-to-one correspondence. For the L-layer network, W l (l ∈ {1,..., L}) represents the connection weight matrix between the l-1 layer and the l layer units, and b l is the corresponding bias term, and the number of units in each layer is M l . The ReLU activation of the i-th neuron in the l-th layer is expressed as follows:

[0015]

[0016] Starting from a 0 = x, where x is the input, and the normalized input makes each x i ∈ [0, 1]. The membrane potential state equation of the corresponding IF spiking neuron can be expressed as:

[0017]

[0018] W ij is the weight of the j-th synapse of neuron i in the l-th layer, and δ(t) is the Dirac function. contains the pulse emission time of the j-th presynaptic neuron. The continuous-time representation of the IF neuron model is discretized into a time step of 1 ms, and the neuron threshold is initialized to V 0 . The weight W is normalized using the data-based normalization method of Equation (3), and the normalized weight value is mapped to the SNN. l

[0019]

[0020] where λ l is the maximum activation value of the neurons in layer l.

[0021] Step 3: Simulate and run the SNN. This neuron model sets the threshold of each neuron at each time step. Neuron pulse generation no longer requires accumulation to reach a fixed threshold. Higher-layer neurons can generate pulses earlier, thus accelerating the pulse transmission rate, increasing the pulse generation rate, and reducing the conversion loss at the same time.

[0022] The change process of the neuron membrane potential is as follows:

[0023] The membrane potential V i l (t) accumulates the input current at each time step :

[0024]

[0025] is the step function, indicating that a pulse appears in the neuron at time t:

[0026]

[0027] where is the threshold of neuron i at time t, and the threshold dynamics formula is as follows:

[0028]

[0029] α is the threshold change factor, indicating the speed of membrane potential depolarization, α ∈ [0, 1], and V 0 is the initial threshold of the neuron.

[0030] V i l (t) is the membrane potential of neuron i in layer l at time t. The spiking neuron accumulates the input Until the membrane potential V i l (t) exceeds the threshold At this time, a pulse is generated, and the neuron membrane potential is immediately reset:

[0031]

[0032] Among them, V thr is generally set to 1.

[0033] Beneficial effects:

[0034] 1. The image recognition method of the deep spiking neural network based on dynamic threshold neurons disclosed in the present invention adopts a dynamic threshold IF neuron model. Compared with the traditional fixed threshold IF model, the post-spike propagation rate is faster, the spike firing rate is higher, the higher-level neurons fire spikes earlier, and the information transmission is faster, which can improve the accuracy of the SNN in image classification tasks. Brief description of the drawings

[0035] Figure 1 Schematic diagram of the dynamic threshold neuron model disclosed in the present invention;

[0036] Figure 2 Flowchart of the image recognition method of the deep spiking neural network based on dynamic threshold neurons disclosed in the present invention;

[0037] Figure 3 Accuracy-latency graph of the traditional IF neuron model under the CIFAR-10 dataset resnet18 network structure;

[0038] Figure 4 Accuracy-latency graph of the present invention under the CIFAR-10 dataset resnet18 network structure. Detailed implementation manners

[0039] The present invention will be described in detail below in conjunction with the drawings and embodiments. At the same time, the technical problems solved by the technical solution of the present invention and the beneficial effects are also described. It should be noted that the described embodiments are only for facilitating the understanding of the present invention and do not limit it in any way.

[0040] A dynamic threshold neuron model, as shown in the attached Figure 1 figure. The neuron model integrates the input spikes at each moment and sets the threshold at this moment according to the change in the neuron membrane potential compared with the previous moment. This change is equal to the weighted sum of the spikes input to the neuron at this moment. The threshold is inversely proportional to this change. When the membrane potential at this moment exceeds the calculated threshold, a spike is emitted, and the membrane potential is randomly reset.

[0041] The image recognition method of the deep spiking neural network based on dynamic threshold neurons disclosed in this embodiment uses the Pytorch deep learning framework to perform DNN training in the environment of Intel(R) Core(TM) i7-8700 CPU 3.20GHz and NVIDIA GdForce GTX 2080Ti GPU. The overall process is as shown in the appendix Figure 2 and specifically includes the following steps:

[0042] Step 1: Train the DNN, obtain the weights and save them;

[0043] Taking the resnet18 network structure and CIFAR-10 dataset as an example, the network uses average pooling, is trained using the backpropagation algorithm, and the accuracy is 94.570%.

[0044] Step 2: Based on the DNN-to-SNN conversion method, convert the ReLU activation function in the DNN into an IF neuron with a dynamic threshold in the SNN, normalize the DNN weights, and map them to the SNN, that is, establish the SNN.

[0045] Using the DNN-to-SNN conversion method, the ReLU activation function in the model in step (1) is equivalent to an IF neuron with a dynamic threshold. For the average pooling in the model, it is converted into spatial downsampling. The network weights trained in step (1) are normalized using a data-based normalization method. 0.4% of the training set is loaded as the data for the normalization model, and 99.9% of the maximum activation value is used as the normalization value λ. The normalized weight values are mapped to the SNN with the same network structure to obtain the SNN network with the resnet18 structure. The threshold V of the dynamic threshold IF neuron 0 is initialized to 1.

[0046] Step 3: Simulate and run the SNN. This neuron model sets the threshold of each neuron at each time step. Neurons no longer need to accumulate to reach a fixed threshold to emit pulses. Higher-level neurons can emit pulses earlier, so it can accelerate the pulse transmission rate, increase the pulse emission rate, and at the same time reduce the conversion loss.

[0047] The simulation time step is set to 400, and the threshold change factor α is set to 1. At each time step, the threshold of each neuron is dynamically calculated according to different input values. The neuron accumulates the membrane potential until it exceeds this dynamic threshold, then emits a pulse, and the membrane potential is immediately reset. As shown in the appendix Figure 3 The conversion accuracy of the traditional IF neuron model is 94.141%, as shown in the appendix Figure 4As shown, the conversion accuracy of the dynamic threshold IF neuron model of the present invention is 94.342%, the accuracy is improved by 0.2%, the pulse emission rate is increased by 1.25%, and the first firing pulse time of the last layer of IF neurons is shortened by 80%, accelerating information transmission.

[0048] The above specific description further details the purpose, technical solution and beneficial effects of the invention. It should be understood that the above is only a specific embodiment of the present invention and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. Image recognition method of deep spiking neural network based on dynamic threshold neurons, characterized in that, it includes the following steps: Step 1: First, train the DNN, obtain the weights and save them; Step 2: Based on the DNN-to-SNN conversion method, convert the ReLU activation function in the DNN into an IF neuron with a dynamic threshold in the SNN, normalize the DNN weights, and map them to the SNN, that is, establish the SNN; Step 3: Simulate and run the SNN. This neuron model sets the threshold of each neuron at each time step. Neurons no longer need to accumulate to reach a fixed threshold to emit spikes. Higher-level neurons can emit spikes earlier. Therefore, it can increase the spike emission rate, accelerate the spike transmission rate, and at the same time reduce the conversion loss; The change process of the neuron membrane potential is as follows: Membrane potential At each time step for the input current Accumulate: is a step function, indicating that a pulse appears at time t: Among them, is the threshold of neuron i at time t, and the threshold dynamics formula is as follows: α is the threshold change factor, representing the speed of membrane potential depolarization rate, α ∈ [0, 1], and V 0 is the initial threshold of the neuron; is the membrane potential of the $i$-th neuron in the $l$-th layer at time $t$. The spiking neuron accumulates the input until the membrane potential exceeds the threshold At this time, a spike is generated, and the membrane potential of the neuron is immediately reset: Among them, V thr is set to 1.

2. The image recognition method of deep spiking neural network based on dynamic threshold neurons according to claim 1, characterized in that, the implementation method of step 2 is: Use the DNN-to-SNN conversion method to equivalently convert the ReLU function in the DNN into an IF neuron with a dynamic threshold in the SNN, thereby converting the DNN into an SNN with a similar structure. Use a data-based normalization method to normalize the DNN weight values, and map the normalized weight values to the SNN; The specific conversion process is as follows: The ReLU units of the DNN and the SNN neurons are in one-to-one correspondence. For an L-layer network, W l (l ∈ {1,..., L}) represents the connection weight matrix between the units of layer l-1 and layer l, and b l is the corresponding bias term. The number of units in each layer is M l ; ReLU activation of neuron i in the l-th layer The expression is as follows: Starting from a 0 = x, where x is the input, and the normalized input makes each x i ∈ [0, 1]; The corresponding IF pulsed neuron has its membrane potential state equation expressed as: W ij is the weight of the j-th synapse of the i-th neuron in the l-th layer, and δ(t) is the Dirac function. contains the spike emission time of the j-th presynaptic neuron; the continuous-time representation of the IF neuron model is discretized into a time step of 1 ms, and the neuron threshold is initialized to V 0 ; Normalize the weight W using the data-based normalization method of Equation (3) l and map the normalized weight value to the SNN; Among them, λ l is the maximum activation value of the neurons in layer l.

Images