Pulse neural network training method and device based on pulse firing rate

Abstract

This invention provides a method and apparatus for training a spiking neural network based on the pulse firing rate. The method includes: determining the pulse firing rate of multiple pulses fired by spiking neurons of a preset spiking neural network model; performing forward and backward propagation based on the pulse firing rate to obtain an updated spiking neural network model; converting the multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model; and performing a weighted summation of the pulse sequences of the multiple simulation time steps based on the reparameterized weights to obtain the input value of the next layer neuron. This invention solves the technical problem of slow training speed in related spiking neural network training methods.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a method and apparatus for training a spiking neural network based on the pulse firing rate. Background Technology

[0002] Spiking neural networks have attracted much attention in recent years due to their low power consumption, event-driven nature, and inherent time-domain information processing capabilities. In particular, they have shown excellent performance when combined with traditional convolutional neural networks and converter models, and many new architectures have been designed on this basis to improve model performance.

[0003] However, the aforementioned spiking neural networks employ direct training algorithms, which often require the additional construction of a computational graph along the time dimension so that gradients can be propagated forward and backward along time. Although this method can fully express the spatiotemporal dynamics of spiking neurons, the additional large computational graph places a huge burden on memory, thus severely slowing down the training speed.

[0004] It is evident that the training methods for spiking neural networks in related technologies suffer from the technical problem of slow training speed. Summary of the Invention

[0005] This invention provides a spiking neural network training method and apparatus based on pulse firing rate, which solves the problem of slow training speed in existing spiking neural network training methods and further improves the performance of spiking neural networks.

[0006] This invention provides a spiking neural network training method based on pulse firing rate, comprising the following steps: determining the pulse firing rate of multiple pulses fired by spiking neurons in a preset spiking neural network model; performing forward and backward propagation based on the pulse firing rate to obtain an updated spiking neural network model; converting the multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model; and performing a weighted summation of the pulse sequences of the multiple simulation time steps based on the reparameterized weights to obtain the input values ​​of the next layer neurons.

[0007] According to the present invention, a method for training a spiking neural network based on the pulse firing rate is provided. The step of determining the pulse firing rate of multiple pulses fired by a spiking neuron of a preset spiking neural network model includes: limiting the number of pulses fired by the spiking neuron to between 0 and the firing intensity of the spiking neuron, and rounding the number of pulses fired to obtain an integer number of pulses; and using the ratio of the integer number of pulses to the firing intensity of the spiking neuron as the pulse firing rate of the spiking neuron.

[0008] According to the present invention, a spiking neural network training method based on pulse firing rate is provided, wherein forward and backward propagation are performed based on the pulse firing rate to obtain an updated spiking neural network model, comprising: constructing a forward propagation relation for the pulse firing rate between layers based on the pulse firing function of the preset spiking neural network model at multiple simulation time steps; constructing a backward propagation relation based on the forward propagation relation; and updating the preset spiking neural network model based on the forward and backward propagation relations to obtain an updated spiking neural network model.

[0009] According to the present invention, a spiking neural network training method based on pulse firing rate is provided, wherein the firing intensity of the spiking neuron is a parameter D, and the step of converting the plurality of pulses into a pulse sequence corresponding to multiple simulation time steps based on the updated spiking neural network model includes: extending the firing intensity parameter D of the spiking neuron to the time dimension; and converting the plurality of pulses at a single time step into a pulse sequence corresponding to the parameter D based on the updated spiking neural network model, wherein the pulse sequence consists only of 0 or 1.

[0010] According to the present invention, a spiking neural network training method based on pulse firing rate is provided, wherein the time step of the updated spiking neural network model is parameter T, and the actual simulation step of the updated spiking neural network model is the product of parameter T and parameter D.

[0011] According to the present invention, a spiking neural network training method based on pulse firing rate is provided. The preset spiking neural network model includes a convolution-based spiking neural network block, a converter-based convolutional neural network block, and a downsampling module. The convolution-based spiking neural network block is used to extract local features; the converter-based convolutional neural network block is used to fuse features in the channel dimension; and the downsampling module is used to expand the number of channel dimensions.

[0012] The present invention also provides a spiking neural network training device based on pulse firing rate, comprising the following modules: a determination module, used to determine the pulse firing rate of multiple pulses fired by spiking neurons of a preset spiking neural network model; an update module, used to perform forward and backward propagation based on the pulse firing rate to obtain an updated spiking neural network model; a conversion module, used to convert the multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model; and a summation module, used to perform weighted summation of the pulse sequences of the multiple simulation time steps based on the reparameterized weights to obtain the input value of the next layer neuron.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the spiking neural network training method based on the pulse firing rate as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the spiking neural network training method based on the pulse firing rate as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the spiking neural network training method based on the pulse firing rate as described above.

[0016] The present invention provides a spiking neural network training method based on pulse firing rate. This method determines the pulse firing rate of multiple pulses fired by spiking neurons in a pre-defined spiking neural network model. Forward and backward propagation are performed based on the pulse firing rate to obtain an updated spiking neural network model. Thus, the pulse sequence is represented by the pulse firing rate of neurons over a period of time, avoiding the storage overhead of complex computational graphs of spiking neurons during forward and backward propagation. This achieves an efficient training algorithm for spiking neural networks. Based on the updated spiking neural network model, multiple pulses are converted into pulse sequences corresponding to multiple simulation time steps. The pulse sequences of multiple simulation time steps are weighted and summed based on the reparameterized weights to obtain the input values ​​of the next layer neurons. Therefore, integer pulses can be equivalently converted into pulse sequences while maintaining the low-power advantage of sparse addition in spiking neural networks, thereby solving the technical problem of slow training speed in related spiking neural network training methods. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the spiking neural network training method based on pulse firing rate provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the training and inference process of the spiking neural network training method based on pulse firing rate provided in the embodiments of the present invention.

[0020] Figure 3This is a network architecture diagram of a spiking neural network based on pulse firing rate provided in an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of the multi-timestep training and multi-timestep inference process provided in the embodiments of the present invention.

[0022] Figure 5 This is a schematic diagram of the structure of a spiking neural network training device based on pulse firing rate provided in an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] Spiking Neural Networks (SNNs) have garnered significant attention in recent years due to their low energy consumption, event-driven nature, and inherent temporal information processing capabilities. Their performance has been particularly impressive when combined with traditional Convolutional Neural Networks (CNNs) and Transformer models, leading to the design of numerous new architectures to further enhance model performance. However, direct training algorithms for these SNNs often require the additional construction of a computational graph along the time dimension to facilitate forward and backward gradient propagation. While this approach effectively represents the spatiotemporal dynamics of spiking neurons, the massive additional computational graph places a heavy burden on memory, severely slowing down training. Furthermore, the training time is directly correlated with the simulation step size; training a 15M SNN with a Transformer architecture and a simulation step size of 4 requires nearly 200 hours, directly impacting the usability of SNNs.

[0026] Furthermore, the practice of using repetitive encoding to construct temporal features for static images in order to leverage the spatiotemporal dynamics of spiking neurons is questionable, because the original static images themselves do not possess temporal features, and repeated inputs would only cause additional overhead in terms of memory and computation.

[0027] Although there are some algorithms that improve the training efficiency of spiking neural networks, none of these algorithms fundamentally address the significance of simulation step size for static input. Consequently, they cannot improve the efficiency of direct training of spiking neural networks from the source, making it difficult to leverage the advantages of spiking neural networks in practical applications.

[0028] To address the issues of high memory consumption and slow training time during the training of spiking neural networks, this invention proposes an efficient spiking neural network training algorithm based on Spike Firing Approximation (SFA).

[0029] refer to Figure 1 , Figure 1 This is a flowchart illustrating the spiking neural network training method based on pulse firing rate provided by the present invention, as shown below. Figure 1 As shown, the method includes the following:

[0030] Step 101: Determine the pulse firing rate of multiple pulses fired by the spiking neurons of the preset spiking neural network model.

[0031] In this embodiment of the invention, during the training phase of the spiking neural network, the spiking neurons of the preset spiking neural network model fire up to D pulses on an additional extended dimension used to characterize the firing intensity (parameter D), and the corresponding firing rate is calculated as the output on this dimension.

[0032] Step 102: Perform forward and backward propagation based on the pulse firing rate to obtain the updated spiking neural network model.

[0033] In this embodiment of the invention, during the training phase of the spiking neural network, the pulse firing rate is used as a feature to characterize the pulse sequence, a transmission relationship of the pulse firing rate between layers is constructed, and the specific processes of forward and backward propagation are constructed based on this relationship to achieve efficient training.

[0034] Here, based on the pulse firing function of the preset spiking neural network model at multiple simulation time steps, an approximate forward propagation relation for the pulse firing rate between layers is constructed; based on the forward propagation relation, a backward propagation relation is constructed; based on the approximate forward propagation relation and the backward propagation relation, the preset spiking neural network model is updated to obtain the updated spiking neural network model.

[0035] It should be noted that the training algorithm described above can be applied not only to single-step input spiking neural network models, but also extended to multi-step input spiking neural network models, enriching the expressive power of spiking neurons through additional extended dimensions.

[0036] Step 103: Based on the updated spiking neural network model, convert multiple pulses into pulse sequences corresponding to multiple simulation time steps.

[0037] In this embodiment of the invention, during the inference stage of the spiking neural network, the above-mentioned method of characterizing by pulse firing rate can be equivalently converted into the pulse sequence fired by the spiking neurons participating in inference, so as to maintain the pure addition and pulse-driven characteristics of the spiking neural network.

[0038] In this embodiment of the invention, integer pulses are converted into pulse sequences corresponding to D simulation time steps by using a soft reset integral-and-fire (SR-IF) neuron model.

[0039] Step 104: Based on the reparameterized weights, perform a weighted summation of the pulse sequences at multiple simulation time steps to obtain the input values ​​of the next layer of neurons.

[0040] In this embodiment of the invention, during the inference stage of the spiking neural network, the neurons of the soft-reset integral ignition neuron model convert integer pulses into pulse sequences of length D, and the pure addition during inference is achieved by reparameterizing the weights. The specific process can be represented by the following formula (1):

[0041]

[0042] Among them, {S l [d]} D SR-IF represents the weighted summation of a purely additive pulse sequence (the input value to the next layer of neurons), D represents the soft-reset integral firing neuron model, and D represents the firing pulse intensity (number of pulses). This represents the reparameterized weights, while S... l-1 [d] is the activation value of the d-th neuron in layer l-1.

[0043] The reparameterized weights can be represented by the following formula (2):

[0044]

[0045] in, The weights are the reparameterized values, where D represents the pulse intensity (number of pulses), and W... l V represents the weight of the l-th layer. th This indicates the threshold for issuance.

[0046] In this embodiment of the invention, the neurons of the soft-reset integral ignition neuron model expand the single-step membrane potential input into a pulse sequence corresponding to the firing intensity (parameter D) time step, and obtain the membrane potential input value of the next layer of neurons after weighted summation.

[0047] This invention proposes a highly efficient spiking neural network training algorithm based on pulse firing rate approximation. Its key feature is that it represents the pulse sequence by the pulse firing rate of neurons over a period of time, avoiding the storage overhead of complex computational graphs of spiking neurons during forward and backward propagation. This achieves a highly efficient training algorithm for spiking neural networks. Furthermore, during inference, this training algorithm can convert the pulse firing rate into a pulse sequence for computation and possesses pulse-driven characteristics. In addition, the above-mentioned efficient training algorithm is task-independent and exhibits excellent performance in various tasks such as image classification, object detection, instance segmentation, and event data classification. It also significantly reduces the sparseness of pulse firing in the network, meeting the requirements of low-power neuromorphic chips.

[0048] According to the present invention, a spiking neural network training method based on pulse firing rate determines the pulse firing rate of multiple pulses fired by spiking neurons in a preset spiking neural network model, including:

[0049] The number of firing pulses of the spiking neuron is limited to between 0 and the firing intensity of the spiking neuron, and the number of firing pulses is rounded down to obtain the integer number of firing pulses;

[0050] The ratio of integer pulses to the firing intensity of the spiking neuron is used as the firing rate of the spiking neuron.

[0051] In this embodiment of the invention, multiple pulses are fired simultaneously based on the input along an additional extended dimension representing the firing intensity, and the pulse firing rate along this dimension is calculated as a representation of the pulse sequence. After establishing the relationship between the pulse firing rates of adjacent layers of the network, forward and backward propagation is performed based on this relationship.

[0052] In this embodiment of the invention, during the forward propagation process, the firing function for firing multiple pulses based on the input is fire. D (·), where D represents the maximum number of pulses emitted in the dimension characterizing the emission intensity, as shown in the following formula (3):

[0053] fire D (.)=round(clip(x,0,D)) (3)

[0054] Here, clip(x,0,D) is the clipping function, which restricts the input x to the range [0,D], and round(·) is the rounding function, which ensures that the number of pulses issued is an integer.

[0055] In this embodiment of the invention, the formula for calculating the pulse firing rate can be expressed by the following formula (4):

[0056]

[0057] Among them, a D The fire rate is represented by D, which represents the maximum number of pulses fired in the dimension characterizing fire intensity. D This represents the pulse firing function.

[0058] It should be noted that during the reverse propagation process, the pulse firing function `fire` in the forward propagation... D The (·) function itself is not differentiable and cannot be trained directly. Therefore, a surrogate gradient function is needed to ensure the learnability of the network. The surrogate gradient function used in the reverse process is the rectangular box function.

[0059] According to the present invention, a spiking neural network training method based on pulse firing rate is provided, which performs forward and backward propagation based on pulse firing rate to obtain an updated spiking neural network model, including:

[0060] Based on the pulse firing function of the pre-defined spiking neural network model at multiple simulation time steps, a forward propagation relationship of the pulse firing rate between layers is constructed.

[0061] Based on the forward transitive relation, construct the backward transitive relation;

[0062] Based on the forward and backward propagation relations, the preset spiking neural network model is updated to obtain the updated spiking neural network model.

[0063] In this embodiment of the invention, based on the pulse firing function of the preset spiking neural network model at multiple simulation time steps, an approximate forward propagation relationship of the interlayer pulse firing rate is constructed, including: based on the iterative formula of the SR-IF neuron at time step D, an approximate forward propagation relationship of the interlayer pulse firing rate can be constructed, specifically referring to the following formula (5):

[0064] Where D represents the distribution intensity, W represents the pulse firing rate of the l-th layer with firing intensity D. l V represents the weight of the l-th layer. th Indicates the issuance threshold, fire D (x) represents the integer pulse firing function.

[0065] Based on the above forward transit relation, the backward transit process can be constructed by referring to the following formula (6):

[0066]

[0067] in, W represents the partial derivative, loss represents the loss function. l This represents the weight of the l-th layer. This represents the pulse firing rate of the l-th layer with firing intensity D.

[0068] It should be noted that since the pulse firing process is not differentiable, a surrogate gradient method is also needed to ensure the training process, specifically for the firing function `fire`. D (x) uses a surrogate gradient function with a rectangular box.

[0069] According to the present invention, a spiking neural network training method based on pulse firing rate is provided, wherein the firing intensity of the spiking neuron is parameter D, and multiple pulses are converted into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model, including:

[0070] Extend the firing intensity parameter D of the spiking neuron to the time dimension;

[0071] Based on the updated spiking neural network model, multiple pulses at a single time step are converted into a pulse sequence corresponding to the simulated time step D, where the pulse sequence consists only of 0 or 1.

[0072] In this embodiment of the invention, the dimension representing the firing intensity is converted into a time dimension, and specific spiking neurons fire pulses based on the input to generate a pulse sequence. Furthermore, by reparameterizing the network weights, the network contains no multiplication operations except for the first coding layer, and only includes pulse-driven sparse addition processes. Specifically, it includes the following two parts:

[0073] During the inference phase, the parameter D representing the firing intensity during training is extended to the time dimension, and the single-time-step input is converted into a pulse sequence over D time steps based on the integral ignition spiking neuron model with soft reset, and the calculated pulse firing rate remains consistent with that during training.

[0074] During the inference phase, the pulse sequence obtained at D time steps after pulse firing is multiplied by the reparameterized time-domain shared weights and summed as the input to the next layer of neurons. Since the pulse sequence consists only of 0s and 1s, the above process only includes pure addition calculation.

[0075] According to the pulse firing rate-based spiking neural network training method provided by the present invention, the time step of the updated spiking neural network model is parameter T, and the actual simulation step of the updated spiking neural network model is the product of parameter T and parameter D.

[0076] refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the training and inference process of a spiking neural network training method based on pulse firing rate provided in an embodiment of the present invention. It includes single-time-step training and multi-time-step inference, spatial forward propagation, temporal forward propagation, and fire... D Represents the integer impulse firing function, and IF stands for Integrate-and-Fire Neuron Model (IF) used to simulate the electrical activity of neurons.

[0077] In this embodiment of the invention, the training and inference process is carried out by the additionally introduced parameter D. The additional extended dimension is only used to characterize the firing intensity by the firing rate and does not have effective spatiotemporal dynamics. Therefore, this training algorithm can be used to replace the traditional binary firing mechanism, thereby applying the training algorithm to a neuron model with spatiotemporal dynamics.

[0078] For example, the traditional methods of attenuating membrane potential or resetting after pulse firing remain unchanged in traditional spiking neurons. During training, the traditional binary pulse firing mechanism is replaced by the spiking neural network training method based on pulse firing rate provided by this invention. This results in richer pulse expression during training, while the actual time step required for spiking neuron simulation remains T, ensuring no additional increase in training cost. During inference, the firing intensity D is combined with the neuron's original simulation step size T, making the total simulation step size required for the inference process T×D.

[0079] In this embodiment of the invention, an SR-IF neuron model is used to convert integer pulses into pulse sequences at corresponding parameter D simulation time steps. The parameter D is combined with the time steps of the neuron's own parameter T, so that the actual simulation step size of the model is the product of parameter T and parameter D. The multiplication and addition operation between the output pulse firing rate and the weight is converted into a pure addition operation through reparameterization.

[0080] According to the present invention, a spiking neural network training method based on pulse firing rate is provided. The preset spiking neural network model includes a convolution-based spiking neural network block, a converter-based convolutional neural network block, and a downsampling module. The convolution-based spiking neural network block is used to extract local features; the converter-based convolutional neural network block is used to fuse features in the channel dimension; and the downsampling module is used to expand the number of channel dimensions.

[0081] refer to Figure 3 , Figure 3 This is a network architecture diagram of a spiking neural network based on pulse firing rate provided in an embodiment of the present invention.

[0082] Among them, the spiking neural network based on pulse firing rate includes: input (static image), downsampling, convolution-based SNN blocks, Transformer-based SNN blocks, and classification projection; the convolution-based SNN blocks include: pulse-depth separable convolutions (including point convolution, normalization, depth convolution, etc.), and channel convolutional layers (including standard convolution, normalization, etc.); the Transformer-based SNN blocks include: pulse-depth separable convolutions (including point convolution, normalization, depth convolution, etc.), efficient pulse-driven self-attention operators (including linear layers, normalization, pulse-driven self-attention operators, etc.), and channel linear layers (including linear layers, normalization, etc.).

[0083] Specifically, the entire network structure consists of convolution-based spiking neural network blocks, Transformer-based convolutional neural network blocks, and a downsampling module. The entire network is divided into four stages: the first two stages are convolutional neural network-based spiking neural network blocks, the last two stages are Transformer-based spiking neural network blocks, and a downsampling module is added before the start of each stage.

[0084] The convolutional neural network-based spiking neural network block consists of a depthwise separable convolution and a channel convolution, used to enhance the model's extraction of local features; the Transformer-based spiking neural network block consists of a spiking-driven self-attention layer and a channel-dimensional linear layer, the former used to extract information from the global scope, and the latter used for channel-dimensional feature fusion; the downsampling module is implemented by a standard convolution, used to reduce the resolution of the feature map and expand the number of channel dimensions.

[0085] This invention provides an efficient spiking neural network training algorithm based on Spike Firing Approximation (SFA). By representing the spike sequence using the spike firing rate, the memory overhead and training time during training become independent of the simulation step size under static input conditions. This accelerates model training speed with the same inference time step and further improves model performance with the same network architecture and training / inference costs. In static input, the introduction of SFA avoids the previous method of repeatedly implementing the encoding of the original input and reduces the overall spike firing rate of the network, thereby further reducing the energy consumption required for computation. This SFA training algorithm can also be extended to multi-step input cases, improving model performance through richer spike firing representations. Besides accelerating training and improving model performance, this training algorithm can still convert integer spikes into equivalent spike sequences during inference, maintaining the low-power advantage of sparse addition in spiking neural networks.

[0086] The SFA training algorithm provided in this embodiment of the invention can also be extended to process multi-step inputs, such as the training and inference process of multi-step neurons after applying the SFA method to a task with dynamic input.

[0087] refer to Figure 4 , Figure 4 This is a schematic diagram of the multi-time-step training and multi-time-step inference process provided in an embodiment of the present invention, which includes processes such as spatial forward propagation (multi-time-step training) and temporal forward propagation (multi-time-step inference). D This represents the integer impulse firing function, and IF represents the integration and firing neuron model.

[0088] During training, the iterative form of the multi-step spiking neuron after applying the SFA method can be referred to the following formulas (7) to (10):

[0089] U l [t]=βH l [t-1]+X l [t],#(7)

[0090] Where l represents the number of layers in the neural network, U l [t] is used to represent the update of the membrane potential, β is used to represent the decay factor, and H l [t-1] is used to represent the membrane potential at the previous time step, X l [t] is used to represent the input at the current time step.

[0091]

[0092] in, This represents the pulse (pulse sequence) corresponding to time step t in the l-th layer of the neural network. D U represents the integer pulse firing function. l [t] represents the membrane potential at time step t.

[0093]

[0094] Hard Reset refers to a hard reset, where H... l [t] represents the update of the membrane potential at time step t, U l [t] represents the current membrane potential, and Hea represents the step function, used to determine whether a pulse has been emitted (via...). (indicated), V reset Indicates reset voltage; Soft Reset indicates soft charge, obtained by adjusting the membrane potential (U0). l Subtract the threshold (V) from [t]) th Multiply by the number of pulses emitted. To achieve this.

[0095]

[0096] in, The pulse firing rate is expressed as the average number of pulses fired over D time steps. It was obtained through practice.

[0097] Here, formula (7) is used to control whether the membrane potential decays after the previous firing time. Based on this, neurons are divided into a leak-integrate-and-fire (LIF) model with decay (0 < β < 1) and an integral-and-fire (IF) model (β = 1). Formula (9) is the reset mechanism adopted after the pulse firing, namely hard reset (HR) and soft reset (SR), where Hea(x) represents the step function, which outputs 1 when the input x is greater than 0, and outputs 0 otherwise. reset This indicates the reset voltage. Based on the above two classification methods, spiking neurons can be divided into four neuron models: HR-LIF, SR-LIF, HR-IF, and SR-IF. When the SFA method is applied to these models, formula (8) can fire up to an integer pulse of D, and the pulse firing rate corresponding to the integer pulse in formula (10) is used as the output of the neuron at each time step.

[0098] In this embodiment of the invention, the iterative form of pure addition calculation in the inference process of the multi-step spiking neuron using the SFA algorithm can be referred to the following formulas (11) to (14):

[0099]

[0100] Among them, X l [t] represents the input at time step (t) of the current layer (1). S represents the reparameterized weights. l-1 [t, d] represents the output of the previous layer at time step (t) (l-1).

[0101] U l [t]=βH l [t-1]+X l [t],#(12)

[0102] Where l represents the number of layers in the neural network, U l [t] is used to represent the update of the membrane potential, β is used to represent the decay factor, and H l [t-1] is used to represent the membrane potential at the previous time step, X l [t] is used to represent the input at the current time step.

[0103] {S l [t, d]} D =SR-IF D (U l [t]), #(13)

[0104] {S l [t, d]} D SR-IF represents the weighted summation of the purely additive pulse sequence at the current time step (t), SR-IF represents the soft-reset integral ignition neuron model, D represents the firing pulse intensity (number of pulses), and U represents the number of pulses fired. l [t] represents the membrane potential at time step (t).

[0105]

[0106] Among them, H l [t] represents the update of the membrane potential at time step t, Hard Reset represents a hard reset, U l [t] represents the current membrane potential, S l [t, 0] represents the output of the zero-fire pulse at time step t of the l-th layer neural network, V reset Indicates reset voltage; Soft Reset indicates soft charge, obtained by adjusting the membrane potential (U0). l Subtract the threshold (V) from [t]) th Multiply by the cumulative number of pulses emitted. To achieve this.

[0107] In this embodiment of the invention, the output of each layer at each time step t is a pulse sequence of length D, which is multiplied by the reparameterized weights and summed to serve as the input value of the next layer of neurons.

[0108] This invention presents the SFA training algorithm, a solution to the slow training speed of direct SNN training. In the implementation scenario, a classification model was trained on the ImageNet-1k dataset using the Meta-SpikeFormer architecture on eight NVIDIA-A100-40GB graphics cards with both Vanilla Training and SFA Training methods, for 200 training epochs. The results show that regardless of whether the parameter count is 55M or 15M, the training time required by the SFA training algorithm is significantly lower than that of the traditional training algorithm. Furthermore, as the inference time step increases, the training time required by the traditional training algorithm is basically proportional to the time step size, while the training time of the SFA training algorithm is only related to the model size. This demonstrates that the SFA training algorithm has a significant benefit in improving the training speed of SNNs.

[0109] The SFA algorithm and E-SpikeFormer architecture (e.g., ...) are gradually applied to a model with 10M parameters in the Meta-SpikeFormer architecture. Figure 3 (As shown in the image) and tested image classification on the ImageNet-1k dataset. The results show that, with the architecture unchanged, using only the SFA training algorithm improves model performance by 6.5% and reduces energy consumption by 38.7% at the same inference time step. Applying this to the E-SpikeFormer architecture does not significantly improve performance; the main improvement is the reduction in power consumption. This indicates that the SFA algorithm effectively improves model performance while introducing sparsity to reduce the power consumption generated by model inference.

[0110] The E-SpikeFormer architecture, after applying the SFA algorithm, was tested for image classification performance on the ImageNet-1k dataset with three parameter scales: 5M, 10M, and 19M. The input time step was T=1, and the maximum number of bursts was D=4. Compared with the Meta-SpikeFormer architecture, it achieved 80.4% performance with the same inference time step and used only 19M parameters, which is a 66.2% reduction in parameters compared to Meta-SpikeFormer. The energy consumption was 5.9mJ, which is an 83.9% reduction in energy consumption compared to Meta-SpikeFormer.

[0111] To verify that the training algorithm not only performs well in image classification but also significantly improves model performance in other tasks, this paper extends the training algorithm to object detection and instance segmentation tasks based on the E-SpikeFormer architecture, and achieves excellent performance in both. This demonstrates that the SFA training algorithm is a universal and excellent algorithm that can improve model performance while reducing training time.

[0112] Besides its application on static datasets with single-step inputs, the SFA training algorithm can also be extended to spiking neurons with multi-step inputs. The firing format of integer pulses gives neurons richer expressive power at each time step, and the D corresponding to the firing intensity can be merged with the T corresponding to the multi-time-step inputs to ensure that the pure additive nature of inference is preserved. This paper validates the algorithm on the HAR-DVS dataset. The introduction of the SFA algorithm improves the model's performance, demonstrating its effectiveness in multi-time-step input scenarios.

[0113] The following describes the pulse firing rate-based spiking neural network training device provided by the present invention. The pulse firing rate-based spiking neural network training device described below can be referred to in correspondence with the pulse firing rate-based spiking neural network training method described above.

[0114] refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a spiking neural network training device based on pulse firing rate provided in an embodiment of the present invention, including a determining module 501, an updating module 502, a conversion module 503, and a summing module 504. Specifically, the determining module 501 is used to determine the pulse firing rate of multiple pulses fired by spiking neurons of a preset spiking neural network model; the updating module 502 is used to perform forward and backward propagation based on the pulse firing rate to obtain an updated spiking neural network model; the conversion module 503 is used to convert multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model; and the summing module 504 is used to perform weighted summation of the pulse sequences of multiple simulation time steps based on the reparameterized weights to obtain the input value of the next layer neuron.

[0115] Specifically, the spiking neural network training device based on pulse firing rate provided by the present invention can implement all the method steps implemented in the above-described spiking neural network training method embodiment based on pulse firing rate, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0116] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640. The processor 610, communication interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logic instructions in the memory 630 to execute a spiking neural network training method based on the pulse firing rate. This method includes: determining the pulse firing rate of multiple pulses fired by spiking neurons in a preset spiking neural network model; performing forward and backward propagation based on the pulse firing rate to obtain an updated spiking neural network model; converting the multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model; and performing a weighted summation of the pulse sequences from the multiple simulation time steps based on the reparameterized weights to obtain the input values ​​of the next layer of neurons.

[0117] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the spiking neural network training method based on the pulse firing rate provided by the above methods. The method includes: determining the pulse firing rate of multiple pulses fired by the spiking neurons of a preset spiking neural network model; performing forward and backward propagation based on the pulse firing rate to obtain an updated spiking neural network model; converting the multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model; and performing a weighted summation of the pulse sequences of multiple simulation time steps based on the reparameterized weights to obtain the input value of the next layer of neurons.

[0119] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a spiking neural network training method based on the pulse firing rate provided by the above methods. The method includes: determining the pulse firing rate of multiple pulses fired by spiking neurons of a preset spiking neural network model; performing forward and backward propagation based on the pulse firing rate to obtain an updated spiking neural network model; converting the multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model; and performing a weighted summation of the pulse sequences of the multiple simulation time steps based on the reparameterized weights to obtain the input value of the next layer of neurons.

[0120] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training an impulse neural network based on a firing rate of impulses, characterized by, include: Determine the firing rate of multiple pulses fired by the spiking neurons of the preset spiking neural network model; Based on the pulse firing rate, forward and backward propagation are performed to obtain the updated spiking neural network model; Based on the updated spiking neural network model, the multiple pulses are converted into pulse sequences corresponding to multiple simulation time steps; The pulse sequences of the multiple simulation time steps are weighted and summed based on the reparameterized weights to obtain the input values ​​of the next layer of neurons. The determination of the pulse firing rate of multiple pulses fired by the spiking neurons of the preset spiking neural network model includes: The number of firing pulses of the spiking neuron is limited to between 0 and the firing intensity of the spiking neuron, and the number of firing pulses is rounded down to obtain an integer number of firing pulses; The ratio of the integer pulse to the firing intensity of the spiking neuron is taken as the firing rate of the spiking neuron. The process of performing forward and backward propagation based on the pulse firing rate to obtain the updated spiking neural network model includes: Based on the pulse firing function of the preset spiking neural network model at multiple simulation time steps, a forward propagation relationship of the pulse firing rate between layers is constructed. Based on the forward transit relation, construct the reverse transit relation; Based on the forward propagation relation and the backward propagation relation, the preset spiking neural network model is updated to obtain the updated spiking neural network model.

2. The spiking neural network training method based on pulse firing rate according to claim 1, characterized in that, The firing intensity of the spiking neuron is parameter D. The process of converting the multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model includes: The firing intensity parameter D of the spiking neuron is extended to the time dimension; Based on the updated spiking neural network model, the multiple pulses at a single time step are converted into a pulse sequence for the simulated time step corresponding to the parameter D, wherein the pulse sequence consists only of 0 or 1.

3. The method for training a spiking neural network based on pulse firing rate according to claim 2, characterized in that, The time step of the updated spiking neural network model is parameter T, and the actual simulation step of the updated spiking neural network model is the product of parameter T and parameter D.

4. The method for training a spiking neural network based on pulse firing rate according to any one of claims 1 to 3, characterized in that, The preset spiking neural network model includes a convolution-based spiking neural network block, a converter-based convolutional neural network block, and a downsampling module. The convolution-based spiking neural network block is used to extract local features; the converter-based convolutional neural network block is used to fuse features in the channel dimension; and the downsampling module is used to expand the number of channel dimensions.

5. A spiking neural network training device based on pulse firing rate, characterized in that, include: The determination module is used to determine the pulse firing rate of multiple pulses fired by the spiking neurons of the preset spiking neural network model; The update module is used to perform forward and backward propagation based on the pulse firing rate to obtain the updated spiking neural network model; The conversion module is used to convert the multiple pulses into pulse sequences corresponding to multiple simulation time steps based on the updated spiking neural network model; The summation module is used to perform weighted summation on the pulse sequences of the multiple simulation time steps based on the reparameterized weights to obtain the input value of the next layer neuron; The determining module is specifically used for: The number of firing pulses of the spiking neuron is limited to between 0 and the firing intensity of the spiking neuron, and the number of firing pulses is rounded down to obtain an integer number of firing pulses; The ratio of the integer pulse to the firing intensity of the spiking neuron is taken as the firing rate of the spiking neuron. The update module is specifically used for: Based on the pulse firing function of the preset spiking neural network model at multiple simulation time steps, a forward propagation relationship of the pulse firing rate between layers is constructed. Based on the forward transit relation, construct the reverse transit relation; Based on the forward propagation relation and the backward propagation relation, the preset spiking neural network model is updated to obtain the updated spiking neural network model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the spiking neural network training method based on the pulse firing rate as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the spiking neural network training method based on the pulse firing rate as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the spiking neural network training method based on the pulse firing rate as described in any one of claims 1 to 4.

Images