Ultra-Low Latency Spiking Neural Network Based on Space-Time Information and Learning Method

Abstract

The present invention relates to a neural network and a learning method, and specifically to an ultra-low latency pulse neural network and a learning method based on space-time information, in order to solve the technical problem that it is difficult to balance the processing accuracy and the processing time information capability for the training of a pulse neural network carrying a long pulse sequence. The pulse neural network is a multi-layer perceptron or convolutional neural network based on pulse neurons, which adopts a multi-threshold LIF model and includes an input layer, multiple intermediate hidden layers and an output layer in sequence. The method steps include: constructing a pulse neural network of a multi-threshold LIF model; initializing the pulse neuron weights using a normal distribution; calculating the forward propagation of the pulse neural network; calculating the loss function to determine whether the error meets the requirements. If so, the learning is completed; if not, the pulse neural network error is calculated for back propagation and the weights are updated until the error meets the requirements.

Description

Technical Field

[0001] The present invention relates to a neural network and a learning method, and in particular to an ultra-low delay pulse neural network and a learning method based on space-time information. Background Art

[0002] Spiking Neural Networks (SNNs) are brain-inspired neural network models that fully exploit spatiotemporal information and exhibit event-driven characteristics. Unlike traditional artificial intelligence networks, which are composed of static, continuous neuron models, SNNs are constructed from more biologically-inspired neuron models and can process discrete spike events. These characteristics give SNNs enormous potential in terms of computational and power efficiency.

[0003] However, for current spiking neural networks, the time information carried by a pulse sequence consisting of dozens or even hundreds of pulses is required to train results with the same level of accuracy as traditional artificial intelligence networks (ANNs). These long pulse sequences not only bring power consumption problems to spiking neural networks, but the large amount of static power consumption introduced also seriously limits the potential of spiking neural networks in terms of power consumption. The current solution is mainly to convert the trained artificial intelligence network into the corresponding spiking neural network. This method can make the obtained spiking neural network and artificial intelligence network have similar accuracy, but it abandons the ability of spiking neural networks to process time information, thereby limiting the advantages of spiking neural networks in space-time information processing. Another method is to directly train the spiking neural network and shorten the pulse sequence to dozens or even dozens of pulse lengths through means such as pulse sequence compression, but this method has the problem of serious accuracy degradation at high compression rates. Summary of the Invention

[0004] The purpose of the present invention is to solve the technical problem that it is difficult to balance processing accuracy and processing time information capability in the training of pulse neural networks carrying long pulse sequences, and to propose an ultra-low delay pulse neural network and learning method based on space-time information, which can effectively shorten the delay of the pulse neural network while ensuring processing accuracy.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] An ultra-low-latency spiking neural network based on spatiotemporal information, which is special in that: a multi-layer perceptron or convolutional neural network based on spiking neurons is constructed using a multi-threshold LIF model, including an input layer, multiple intermediate hidden layers, and an output layer arranged in sequence;

[0007] The input layer is used to transform neural network information, using a normalization method to convert the input information into a real value of 0-1, and pass it as input current to the next layer of the network;

[0008] The synaptic model in the pulse neuron model of the middle hidden layer adopts a zero-order model;

[0009] The output layer converts the labels of the learning samples into neurons that only the neurons corresponding to the target numbers can generate pulses, and the other neurons do not generate pulses.

[0010] Furthermore, the multi-threshold LIF model is:

[0011]

[0012]

[0013]

[0014] in:

[0015] I[t] is the synaptic input current sent by the previous layer of spiking neural network to the i-th spiking neuron;

[0016] i represents the i-th spiking neuron of the spiking neural network;

[0017] j represents the jth spiking neuron in the previous layer of spiking neural network;

[0018] N is the number of spiking neurons in the previous layer of spiking neural network;

[0019] ω ij is the weight between the jth spiking neuron in the previous spiking neural network and the i-th spiking neuron in the current spiking neural network;

[0020] s j [t] represents the output pulse of the j-th spiking neuron in the previous layer of spiking neural network at time t;

[0021] u i [t] is the membrane voltage of the i-th spiking neuron at time t;

[0022] u i [t-1] is the membrane voltage of the i-th spiking neuron at the previous moment;

[0023] u reset [t] is the reset voltage of the spiking neuron at time t;

[0024] τ m is the membrane voltage time constant of the spiking neuron;

[0025] s i[t] is the output pulse of the i-th spiking neuron at time t;

[0026] S max is the upper limit of the output pulse of the spiking neuron;

[0027] V th is the threshold voltage of the spiking neuron.

[0028] In order to simplify the calculation, the synaptic model in the spike neuron model of the middle hidden layer adopts a zero-order model, that is, I[t] in the model is calculated using a zero-order model.

[0029] At the same time, the present invention also proposes an ultra-low latency spike neural network learning method based on space-time information, which is special in that it includes the following steps:

[0030] Step 1: Replace the traditional single-threshold LIF model with a multi-threshold LIF model, and construct a spiking neural network based on the multi-threshold LIF model;

[0031] Step 2: Based on the multi-threshold LIF model, the spike neuron weights are initialized using normal distribution according to formula (1):

[0032]

[0033] in:

[0034] l represents the lth layer of the spiking neural network;

[0035] W (l) is the weight of the l-th layer of the spiking neural network;

[0036] V th is the threshold voltage of the spiking neuron;

[0037] N l-1 is the number of spiking neurons in the previous layer of spiking neural network;

[0038] Step 3: Calculate the forward propagation of the spiking neural network based on the multi-threshold LIF model according to the spiking neuron weights to obtain the output pulses of the spiking neurons;

[0039] Step 4: Calculate the spiking neural network loss function to determine whether the error meets the error requirements:

[0040] 4.1) According to the output pulse of the spiking neuron, the spiking neural network at t is calculated by formula (2): k The loss function at the moment:

[0041]

[0042] in:

[0043] i represents the i-th spiking neuron of the spiking neural network;

[0044] L[t k ] is the pulse neural network at t k Error in time;

[0045] No is the number of spiking neurons in the output layer of the spiking neural network;

[0046] y i [t k ] is in t k The output pulse expected at each moment;

[0047] s i [t k ] is in t k The actual output pulse at the moment;

[0048] 4.2) Judgment error L[t k ] Whether the error requirement is met: If yes, the spiking neural network learning is completed; if not, go to step 5;

[0049] Step 5: Calculate the error back propagation of the spiking neural network based on the multi-threshold LIF model according to the spiking neuron weights and errors, update the spiking neuron weights, and then return to step 3.

[0050] Furthermore, in step 1, the multi-threshold LIF model is specifically represented by formulas (3), (4), and (5):

[0051]

[0052]

[0053]

[0054] Formula (3) is used to calculate the synaptic input current of the previous layer of the pulse neural network;

[0055] in:

[0056] I[t] is the synaptic input current sent by the previous layer of spiking neural network to the i-th spiking neuron;

[0057] i represents the i-th spiking neuron of the spiking neural network;

[0058] j represents the jth spiking neuron in the previous layer of spiking neural network;

[0059] N is the number of spiking neurons in the previous layer of spiking neural network;

[0060] ω ijis the weight between the jth spiking neuron in the previous spiking neural network and the ith spiking neuron in the current spiking neural network, ω ij is the matrix W (l) Elements in

[0061] s j [t] represents the output pulse of the j-th spiking neuron in the previous layer of spiking neural network at time t;

[0062] Formula (4) is used to calculate the membrane voltage of the update pulse neuron;

[0063] in:

[0064] u i [t] is the membrane voltage of the i-th spiking neuron at time t;

[0065] u i [t-1] is the membrane voltage of the i-th spiking neuron at the previous moment;

[0066] u reset [t] is the reset voltage of the spiking neuron at time t;

[0067] τ m is the membrane voltage time constant of the spiking neuron;

[0068] Formula (5) is used to calculate the obtained output pulse:

[0069] in:

[0070] s i [t] is the output pulse of the i-th spiking neuron at time t;

[0071] S max is the upper limit of the output pulse of the spiking neuron;

[0072] V th is the threshold voltage of the spiking neuron.

[0073] Furthermore, in step 3, the forward propagation calculation process is specifically as follows:

[0074] First, the synaptic current generated by the neurons in the previous layer is calculated according to formula (3);

[0075] Secondly, according to formula (4) and the membrane voltage value of the spiking neuron at the previous moment, the membrane voltage of the updated spiking neuron is calculated;

[0076] Finally, through formula (5) and threshold voltage V th The comparison determines whether a pulse is generated, and the resulting output pulse is calculated.

[0077] Furthermore, in step 5, the error back propagation calculation process is specifically as follows:

[0078] 5.1) Calculate the partial derivative of membrane voltage with respect to weight according to formula (6):

[0079]

[0080] in:

[0081] s (l) [t k ] is the lth layer of the pulse neural network at t k Output pulse at time;

[0082] s (l-1) [t k ] is the pulse neural network of the previous layer at t k Output pulse at time;

[0083] u (l) [t k ] is the lth layer of the pulse neural network at t k Membrane voltage at the moment;

[0084] u (l-1) [t k ] is the pulse neural network of the previous layer at t k Membrane voltage at the moment;

[0085] is the l-th layer of the spiking neural network at t k The partial derivative of the output pulse at time t on the membrane voltage;

[0086] is the l-th layer of the spiking neural network at t k The partial derivative of the membrane voltage with respect to the weight at time t;

[0087] is the l-th layer of the spiking neural network at t k-1 The partial derivative of the membrane voltage with respect to the weight at time t;

[0088] 5.2) Calculate the partial derivative of the error with respect to the weight according to formulas (10) and (11):

[0089]

[0090]

[0091] in:

[0092] W (l+1) is the weight of the next layer of spiking neural network;

[0093] δ (l) [t k] is the lth layer of the pulse neural network at t k The partial derivative of the moment error with respect to the membrane voltage;

[0094] δ (l+1) [t k ] is the next layer of pulse neural network at t k The partial derivative of the moment error with respect to the membrane voltage;

[0095] 5.3) Update weight W using Adam algorithm (l) .

[0096] By performing back propagation calculations on the pulse neural network errors, the weights are updated and the errors are gradually reduced, thus ensuring that the pulse neural network achieves high accuracy and low latency.

[0097] Further, in step 5.1), the It is replaced by the approximate equivalent function f1(u), f2(u) or f3(u), specifically formula (7), (8) and (9):

[0098]

[0099]

[0100]

[0101] in:

[0102] u is the membrane voltage of the spiking neuron;

[0103] a H is the first approximate equivalent function parameter, which controls the height of the equivalent function curve;

[0104] a W is the second approximate equivalent function parameter, which controls the width of the equivalent function curve.

[0105] The use of approximate equivalent functions f1(u), f2(u) or f3(u) solves the problem that the discrete process cannot be calculated when the pulse occurs during the error back propagation process. The three approximate equivalent functions can all achieve good output accuracy.

[0106] Furthermore, in step 1, the multilayer perceptron or convolutional neural network based on pulse neurons is implemented using the Pytorch architecture, with a simulation step size of 1ms and a training time of 2 steps; the input information includes pixel and audio information, and the number of input information is defined as the batch number N Batch ;

[0107] In step 5.3), the weight update uses the Adam algorithm as the optimizer, and uses one or more of the datasets MNIST, FashionMNIST, and CIFAR10. The learning rates corresponding to the datasets MNIST, FashionMNIST, and CIFAR10 are η1, η2, and η3, respectively.

[0108] Furthermore, in step 3 and step 5:

[0109] The membrane voltage time constant τ m is 10ms; the threshold voltage V th 10mV;

[0110] The first approximate equivalent function parameter α H The second approximate equivalent function parameter α is 1; W is 20;

[0111] The output pulse upper limit S max is 15;

[0112] The batch number N Batch is 128;

[0113] The learning rates η1, η2, and η3 of the datasets MNIST, FashionMNIST, and CIFAR10 are 0.005, 0.005, and 0.0005, respectively.

[0114] Furthermore, the data set adopts CIFAR10, the training cycle of the pulse neural network is 100, and the approximate equivalent function is selected as f2(u).

[0115] Compared with the prior art, the present invention has the following beneficial effects:

[0116] 1) The present invention adopts an ultra-low-latency spiking neural network learning method based on space-time information, replaces the traditional single-threshold LIF model with a multi-threshold LIF model, and constructs a multi-layer perceptron (MLP) or convolutional neural network based on spiking neurons. By calculating the forward propagation and error back propagation of the spiking neural network based on the multi-threshold LIF model, the error of the spiking neural network meets the requirements, effectively shortening the delay of the spiking neural network while ensuring processing accuracy.

[0117] 2) The method of the present invention uses a normal distribution to initialize the weights of spiking neurons based on the multi-threshold LIF model, which can improve the output accuracy of the spiking neural network.

[0118] 3) The present invention updates the weights by performing back propagation calculations of the pulse neural network errors, gradually reducing the errors, thereby ensuring that the pulse neural network obtains high accuracy and low latency.

[0119] 4) In the process of calculating the back propagation of the pulse neural network error, the present invention proposes three approximate equivalent functions, which solves the problem that the discrete process cannot be calculated when the pulse occurs during the back propagation of the error. For the problem of , three approximate equivalent functions can all achieve good output accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] Figure 1 Schematic diagram of any two layers of a spiking neural network of a multi-threshold LIF model in the ultra-low latency spiking neural network learning method based on space-time information of the present invention;

[0121] Figure 2 For the present invention Schematic diagram of the approximate equivalent function replacement, f1, f2, f3 are three approximate equivalent functions, the horizontal axis is the membrane voltage (with the threshold voltage V th as units);

[0122] Figure 3 For the CIFAR10 dataset in this invention, different Comparison of the network trained by the approximate equivalent function of ; where the horizontal axis is the number of iterations and the vertical axis is the accuracy;

[0123] Figure 4 for Figure 3 A partial enlarged view of . DETAILED DESCRIPTION

[0124] In order to more clearly illustrate the technical solution of the present invention, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0125] An ultra-low-latency spike neural network learning method based on space-time information includes the following steps:

[0126] Step 1: Replace the traditional single-threshold Leaky Integrate and Fired (LIF) model with a multi-threshold LIF model and construct a multi-layer perceptron (MLP) or convolutional neural network based on spiking neurons. The constructed multi-layer perceptron or convolutional neural network includes an input layer, multiple intermediate hidden layers, and an output layer.

[0127] Step 2: Initialize the neuron weights using normal distribution according to formula (1).

[0128]

[0129] Among them, W (l) is the weight of the l-th layer of the pulse neural network, and the elements in the matrix are denoted as ω ij , V th is the voltage threshold of the spiking neuron, N l-1 is the number of neurons in the previous layer of spiking neural network.

[0130] Step 3: Calculate the forward propagation of the spiking neural network based on the multi-threshold LIF model to obtain the value of the output pulse of the spiking neuron.

[0131] According to the multi-threshold LIF model, the synaptic current generated by the neurons in the previous layer is first calculated, and then the membrane voltage of the spiking neurons is calculated based on the membrane voltage value of the spiking neurons at the previous moment. Finally, the membrane voltage of the spiking neurons is updated by comparing it with the threshold voltage V th The comparison determines whether a pulse is generated, and the value of generating the pulse is calculated.

[0132] Step 4: Calculate the loss function according to formula (2) and determine whether the error meets the requirements.

[0133]

[0134] Among them, L[t k ] is the pulse neural network at t k The error at the moment, No is the number of neurons in the output layer, y i [t k ] and s i [t k ] are respectively in t k The expected pulse and the actual pulse at each moment.

[0135] Step 5: If the error does not meet the requirements, calculate the back propagation of the spiking neural network error based on the multi-threshold LIF model, update the spiking neuron weights, and return to step 3 until the error meets the requirements.

[0136] The above steps of the ultra-low latency spike neural network learning method based on space-time information of the present invention are further explained as follows:

[0137] In step 1, the multi-threshold LIF model is described by formulas (3)-(5):

[0138]

[0139]

[0140]

[0141] Formula (3) describes the calculation of the synaptic input current I[t] of the previous layer, where N is the number of spiking neurons in the previous layer of the spiking neural network, ωij is the weight between neuron j in the previous neural network and neuron i in the current neural network, s j [t] represents the output pulse of the j-th spiking neuron in the previous layer of spiking neural network at time t.

[0142] Formula (4) describes the pulse neuron membrane voltage u i [t] Update calculation, where u i [t-1] represents the membrane voltage value at the previous moment, u reset [t] represents the reset voltage, I[t] is the sum of the synaptic input currents sent by the previous neural network to neuron i, τ m is the time constant of the membrane voltage.

[0143] Formula (5) describes the generation process of the multi-threshold LIF model, where s i [t] is the output pulse of the i-th spiking neuron, S max is the upper limit of the output pulse value, V th is the threshold voltage, u i [t] is the membrane voltage of the neuron at time t.

[0144] The multi-threshold LIF model can be used to construct but is not limited to multi-layer perceptrons (MLPs) or convolutional neural networks based on spiking neurons.

[0145] In step 3, the forward propagation of the spiking neural network based on the multi-threshold LIF model is calculated. First, the synaptic current generated by the neurons in the previous layer is calculated according to formula (3). In the second step, the membrane voltage of the updated spiking neuron is calculated according to formula (4) and the membrane voltage value of the spiking neuron at the previous moment. Finally, according to formula (5), the membrane voltage of the updated spiking neuron is calculated by comparing it with the threshold voltage V. th The comparison determines whether a pulse is generated, and the value of generating the pulse is calculated.

[0146] In step 5, the specific process of error back propagation calculation is:

[0147] 5.1) Calculate the partial derivative of membrane voltage with respect to weight according to formula (6):

[0148]

[0149] in, For t k The deflection of the pulse generated at each moment on the membrane voltage, For t k-1 The partial derivative of the membrane voltage with respect to the weight at time s (l-1) [t k ] is the previous layer of neural network at t k The pulse generated at time τ m is the time constant of the membrane voltage.

[0150] Since the discrete process when the pulse occurs cannot be calculated The present invention proposes three approximately equivalent functions to replace The three approximately equivalent functions f1, f2, and f3 are shown in formulas (7), (8), and (9):

[0151]

[0152]

[0153]

[0154] Where u is the membrane voltage of the spiking neuron, a H and a W These are two parameters of the approximate equivalent function, which control the height and width of the curve respectively.

[0155] 5.2) Calculate the partial derivative of the error with respect to the weight according to formula (10) (11):

[0156]

[0157]

[0158] Among them, W (l+1) is the weight of the next layer of spiking neural network, δ (l+1) [t k ] is the next layer of neural network at t k The partial derivative of the timing error with respect to the membrane voltage.

[0159] 5.3) Update weight W using Adam algorithm (l)

[0160] W (l) Represents the weights of the neural network at layer l. The Adam algorithm uses datasets including MNIST, FashionMNIST, and CIFAR10. MNIST is a handwritten data set, FashionMNIST is a clothing image set, and CIFAR10 is a general object recognition dataset. The corresponding learning rates for these three datasets are denoted as η1, η2, and η3, respectively. Of course, the Adam algorithm is not limited to the three datasets mentioned above; it can also be used for other datasets. Furthermore, the weight update algorithm is not limited to the Adam algorithm; other algorithms can also be used.

[0161] Example

[0162] like Figure 1The figure shows a schematic diagram of any two layers of neural networks in a spiking neural network based on a multi-threshold LIF model. The spiking neural network in this embodiment is a multi-layer perceptron. In other embodiments, it can also be used for a convolutional neural network. The specific steps for constructing a spiking neural network are as follows:

[0163] The main function of the input layer of the pulse neural network is to transform the neural network information. It uses the normalization method to convert the input information such as pixels and audio into real values ​​of 0-1, and passes it as input current to the next layer of the neural network.

[0164] In order to simplify the calculation in the middle hidden layer (i.e., multiple hidden layers after excluding the first input layer and the last output layer), the synaptic model in the pulse neuron model in step 1 adopts a zero-order model. The specific calculation is shown in formula (3).

[0165] In the output layer, the labels of the learning samples are converted so that only the neurons corresponding to the target numbers can generate pulses, and the rest of the neurons do not generate pulses.

[0166] The spiking neural network adopts the ultra-low delay spiking neural network learning method based on space-time information of the present invention for learning and training: the spiking neuron weights are initialized using normal distribution; the forward propagation of the spiking neural network is calculated based on the weights to obtain the output pulse; the t is calculated based on the output pulse and the expected output pulse. k The loss function at the moment is used to determine whether the error meets the error requirements; if not, the back propagation of the pulse neural network error is calculated based on the weights and errors, and the weights are updated; then the forward propagation of the pulse neural network and the loss function are recalculated until the error meets the requirements.

[0167] In a preferred embodiment of the present invention, the Pytorch architecture is used for implementation, the simulation step size is selected to be 1ms, and the training time is set to 2 steps. The Adam algorithm is used as the optimizer to update the weights.

[0168] The parameters involved in the forward propagation and the back propagation of the calculation error in this embodiment are shown in Table 1. Among them, the batch number N Batch It refers to the number of input information read in simultaneously. The Adam algorithm uses data sets including MNIST, FashionMNIST, and CIFAR10, and the corresponding learning rates are denoted as η1, η2, and η3 respectively.

[0169] Table 1. Parameters involved in the forward propagation and error backpropagation calculations

[0170]

[0171] like Figure 2 As shown, The schematic diagram of the approximate equivalent function replacement of The approximate equivalent functions f1, f2, and f3 are one of the core technologies of the present invention. The horizontal axis in the figure is the membrane voltage, and the threshold voltage V th It can be seen that under different membrane voltages, the three functions Approximate equivalent value of .

[0172] like Figure 3 、 Figure 4 As shown in the figure, in a preferred embodiment of the present invention, the CIFAR10 dataset is used to compare the recognition accuracy of spiking neural networks trained for 100 cycles using approximate equivalent functions f1, f2, and f3. As can be seen from the figure, the three approximate equivalent functions f1, f2, and f3 all achieve good accuracy, with f2 being the optimal choice.

[0173] The above is only a description of the preferred embodiment of the present invention, and is not a limitation of the technical solution of the present invention. Any modifications made by those skilled in the art based on the main technical concept of the present invention fall within the technical scope to be protected by the present invention.

Claims

1. An ultra-low latency spiking neural network based on space-time information, characterized by: The spiking neural network is a multi-layer perceptron or convolutional neural network based on spiking neurons constructed using a multi-threshold LIF model, comprising an input layer, multiple intermediate hidden layers, and an output layer arranged in sequence; The input layer is used to transform neural network information, using a normalization method to convert the input information into a real value between 0 and 1, and pass it as input current to the next layer of the network; the input information includes pixel and audio information; The synaptic model in the pulse neuron model of the middle hidden layer adopts a zero-order model; The output layer converts the labels of the learning samples into neurons that only the neurons corresponding to the target numbers can generate pulses, while the rest of the neurons do not generate pulses. The multi-threshold LIF model is: in: I[t] is the synaptic input current sent by the previous layer of spiking neural network to the i-th spiking neuron; i represents the i-th spiking neuron of the spiking neural network; j represents the jth spiking neuron in the previous layer of spiking neural network; N is the number of spiking neurons in the previous layer of spiking neural network; ω ij is the weight between the jth spiking neuron in the previous spiking neural network and the i-th spiking neuron in the current spiking neural network; s j [t] represents the output pulse of the j-th spiking neuron in the previous layer of spiking neural network at time t; u i [t] is the membrane voltage of the i-th spiking neuron at time t; u i [t-1] is the membrane voltage of the i-th spiking neuron at the previous moment; u reset [t] is the reset voltage of the spiking neuron at time t; τ m is the membrane voltage time constant of the spiking neuron; s i [t] is the output pulse of the i-th spiking neuron at time t; S max is the upper limit of the output pulse of the spiking neuron; V th is the threshold voltage of the spiking neuron.

2. An ultra-low-latency spike neural network learning method based on space-time information, characterized in that: The following steps are involved: Step 1: Replace the traditional single-threshold LIF model with a multi-threshold LIF model, and construct a spiking neural network based on the multi-threshold LIF model; the spiking neural network is a multi-layer perceptron or convolutional neural network based on spiking neurons, including an input layer, multiple intermediate hidden layers, and an output layer arranged in sequence; the input layer is used to transform neural network information, using a normalization method to convert the input information into a real value between 0 and 1, and pass it as input current to the next layer of the network; the input information includes pixel and audio information; Step 2: Based on the spiking neural network, the spiking neuron weights are initialized using normal distribution according to formula (1): in: l represents the lth layer of the spiking neural network; W (l) is the weight of the l-th layer of the spiking neural network; V th is the threshold voltage of the spiking neuron; N l-1 is the number of spiking neurons in the previous layer of spiking neural network; Step 3: Calculate the forward propagation of the spiking neural network based on the multi-threshold LIF model according to the spiking neuron weights to obtain the output pulses of the spiking neurons; Step 4: Calculate the spiking neural network loss function to determine whether the error meets the error requirements: 4.1) According to the output pulse of the spiking neuron, the spiking neural network at t is calculated by formula (2): k The loss function at the moment: in: i represents the i-th spiking neuron of the spiking neural network; L[t k ] is the pulse neural network at t k Error in time; No is the number of spiking neurons in the output layer of the spiking neural network; y i [t k ] is in t k The output pulse expected at each moment; s i [t k ] is in t k The actual output pulse at the moment; 4.2) Judgment error L[t k ] Whether the error requirement is met: If yes, the spiking neural network learning is completed; if not, go to step 5; Step 5: Calculate the error back propagation of the spiking neural network based on the multi-threshold LIF model according to the spiking neuron weights and errors, update the spiking neuron weights, and then return to step 3.

3. The ultra-low-latency pulse neural network learning method based on space-time information according to claim 2 is characterized in that: In step 1, the multi-threshold LIF model is specifically: Formula (3) is used to calculate the synaptic input current of the previous layer of the pulse neural network; in: I[t] is the synaptic input current sent by the previous layer of spiking neural network to the i-th spiking neuron; i represents the i-th spiking neuron of the spiking neural network; j represents the jth spiking neuron in the previous layer of spiking neural network; N is the number of spiking neurons in the previous layer of spiking neural network; ω ij is the weight between the jth spiking neuron in the previous spiking neural network and the i-th spiking neuron in the current spiking neural network; s j [t] represents the output pulse of the j-th spiking neuron in the previous layer of spiking neural network at time t; Formula (4) is used to calculate the membrane voltage of the update pulse neuron; in: u i [t] is the membrane voltage of the i-th spiking neuron at time t; u i [t-1] is the membrane voltage of the i-th spiking neuron at the previous moment; u reset [t] is the reset voltage of the spiking neuron at time t; τ m is the membrane voltage time constant of the spiking neuron; Formula (5) is used to calculate the obtained output pulse: in: s i [t] is the output pulse of the i-th spiking neuron at time t; S max is the upper limit of the output pulse of the spiking neuron; V th is the threshold voltage of the spiking neuron.

4. The ultra-low-latency pulse neural network learning method based on space-time information according to claim 3 is characterized in that: In step 3, the forward propagation calculation process is specifically as follows: First, the synaptic current generated by the neurons in the previous layer is calculated according to formula (3); Secondly, according to formula (4) and the membrane voltage value of the spiking neuron at the previous moment, the membrane voltage of the updated spiking neuron is calculated; Finally, through formula (5) and threshold voltage V th The comparison determines whether a pulse is generated, and the resulting output pulse is calculated.

5. The ultra-low-latency pulse neural network learning method based on space-time information according to claim 4 is characterized in that: In step 5, the error back propagation calculation process is specifically as follows: 5.1) Calculate the partial derivative of membrane voltage with respect to weight according to formula (6): in: s (l) [t k ] is the lth layer of the pulse neural network at t k Output pulse at time; s (l-1) [t k ] is the pulse neural network of the previous layer at t k Output pulse at time; u (l) [t k ] is the lth layer of the pulse neural network at t k Membrane voltage at the moment; u (l-1) [t k ] is the pulse neural network of the previous layer at t k Membrane voltage at the moment; is the l-th layer of the spiking neural network at t k The partial derivative of the output pulse at time t on the membrane voltage; is the l-th layer of the spiking neural network at t k The partial derivative of the membrane voltage with respect to the weight at time t; is the l-th layer of the spiking neural network at t k-1 The partial derivative of the membrane voltage with respect to the weight at time t; 5.2) Calculate the partial derivative of the error with respect to the weight according to formulas (10) and (11): in: W (l+1) is the weight of the next layer of spiking neural network; δ (l) [t k ] is the lth layer of the pulse neural network at t k The partial derivative of the moment error with respect to the membrane voltage; δ (l+1) [t k ] is the next layer of pulse neural network at t k The partial derivative of the moment error with respect to the membrane voltage; 5.3) Update weight W using Adam algorithm (l) .

6. The ultra-low-latency pulse neural network learning method based on space-time information according to claim 5 is characterized in that: In step 5.1), the It is replaced by the approximate equivalent function f1(u), f2(u) or f3(u), specifically formula (7), (8) and (9): in: u is the membrane voltage of the spiking neuron; a H is the first approximate equivalent function parameter, which controls the height of the equivalent function curve; a W is the second approximate equivalent function parameter, which controls the width of the equivalent function curve.

7. The ultra-low-latency spike neural network learning method based on space-time information according to any one of claims 2 to 6, characterized in that: In step 1, the multilayer perceptron or convolutional neural network based on pulse neurons is implemented using the Pytorch architecture, with a simulation step size of 1ms and a training time of 2 steps; the number of input information is defined as the batch size N Batch ; In step 5.3), the weight update uses the Adam algorithm as the optimizer, and uses one or more of the datasets MNIST, FashionMNIST, and CIFAR10. The learning rates corresponding to the datasets MNIST, FashionMNIST, and CIFAR10 are η1, η2, and η3, respectively.

8. The ultra-low-latency pulse neural network learning method based on space-time information according to claim 7 is characterized in that: In step 3 and step 5: The membrane voltage time constant τ m 10ms; The threshold voltage V th 10mV; The first approximate equivalent function parameter α H is 1; The second approximate equivalent function parameter α W is 20; The output pulse upper limit S max is 15; The batch number N Batch is 128; The MNIST learning rate η1 is 0.005; FashionMNIST learning rate η2 is 0.005; The learning rate η3 for CIFAR10 is 0.0005.

9. The ultra-low-latency spike neural network learning method based on space-time information according to claim 8, characterized in that: The data set adopts CIFAR10, the training cycle of the spiking neural network is 100, and the approximate equivalent function is selected as f2(u).

Images