Training method, device and equipment of spiking neural network, medium and product

Abstract

The invention discloses a spiking neural network training method, device and equipment, a medium and a product, and relates to the technical field of artificial intelligence, and the method comprises the steps: inputting event flow data into a spiking neural network to obtain a pulse sequence and a pulse distribution rate, calculating pulse counting loss based on an expected pulse sequence of the event flow data and the pulse sequence, the task precision is ensured; the distribution rate is introduced to control the loss, the pulse distribution rate is brought into training constraint, and the distribution rate can be regulated and controlled to a reasonable level; the pulse counting loss and the release rate control loss are fused, so that the synaptic weight of the pulse neural network is updated, and the task precision and release rate regulation and control can be considered in weight optimization. The dual constraint directly brings the distribution rate control into a training target, and the problem that the distribution rate is increased while only the precision is pursued is fundamentally avoided, so that the inter-layer neuron pulse distribution rate is reduced, and the energy consumption advantage of the pulse neural network is brought into full play.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, apparatus, equipment, medium and product for a pulse neural network. Background Art

[0002] With the rapid development of artificial intelligence (AI), neural networks have been widely used in fields such as image recognition and speech processing. However, their high energy consumption has become a key bottleneck hindering their further adoption in resource-constrained scenarios such as mobile terminals and embedded devices. Spiking Neural Networks (SNNs), by simulating the event-driven computing properties of biological neurons, transmit information through sparse spike signals, performing computations only when neurons fire, remaining silent most of the time. These SNNs exhibit significant energy savings when processing spatiotemporal information, making them a key technology for achieving low-energy intelligent computing.

[0003] In spiking neural networks, synaptic weights are closely related to the neuron's spike rate, and improper settings can lead to an increase in the spike rate. However, due to the non-differentiable nature of the spike firing process, traditional backpropagation is difficult to use for synaptic weight training. Mainstream approaches rely on converting artificial neural networks (ANNs) to spiking neural networks (SNNs) or direct training based on proxy gradients. These methods focus on downstream task accuracy, fail to consider the relationship between synaptic weights and spike rate, and do not constrain the spike rate. As a result, the spike rate of neurons in interlayers of trained SNNs is relatively high, making it difficult to realize their energy efficiency advantages and limiting their practical value in low-power scenarios. Summary of the Invention

[0004] The present application provides a training method, apparatus, device, medium and product for a pulse neural network, so as to at least solve the problem in the related art that the pulse neural network training process aims at accuracy, resulting in high energy consumption caused by high pulse firing rate of interlayer neurons.

[0005] The present application provides a training method for a spiking neural network, comprising: inputting event stream data into the spiking neural network to obtain a pulse sequence and a pulse firing rate; calculating a pulse counting loss based on an expected pulse sequence and a pulse sequence corresponding to the event stream data; calculating a firing rate control loss based on the pulse firing rate; calculating a fusion loss based on the pulse counting loss and the firing rate control loss; and updating the synaptic weights of the spiking neural network based on the fusion loss.

[0006] The present application also provides a training device for a spiking neural network, comprising: A data preparation module is used to input event stream data into the spiking neural network to obtain spike trains and spike firing rates; a pulse counting loss calculation module, for calculating the pulse counting loss based on the expected pulse sequence and the pulse sequence corresponding to the event stream data; A firing rate control loss calculation module is used to calculate the firing rate control loss according to the pulse firing rate; A fusion loss calculation module is used to calculate the fusion loss based on the pulse count loss and the firing rate control loss; The weight update module is used to update the synaptic weights of the spiking neural network according to the fusion loss.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of the above-mentioned pulse neural network training method when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the above-mentioned pulse neural network training method are implemented.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned pulse neural network training method when executed by a processor.

[0010] This application first inputs event stream data into a spiking neural network to obtain pulse sequences and pulse firing rates. The pulse count loss is calculated based on the expected pulse sequence and pulse sequence of the event stream data to ensure task accuracy. The firing rate control loss is introduced to incorporate the pulse firing rate into the training constraints, which can regulate the firing rate to a reasonable level. The pulse count loss and firing rate control loss are integrated to update the synaptic weights of the spiking neural network, allowing weight optimization to take into account both task accuracy and firing rate control. This dual constraint directly incorporates firing rate control into the training objective, fundamentally avoiding the problem of pursuing accuracy while allowing the firing rate to increase, thereby reducing the interlayer neuron pulse firing rate and giving full play to the energy consumption advantages of SNNs. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A schematic diagram of a specific hardware architecture on which the execution of a spiking neural network training method provided in an embodiment of the present application relies; Figure 2A flowchart of a spiking neural network training method provided in an embodiment of the present application; Figure 3 Schematic diagram of the firing mechanism of spiking neurons; Figure 4 Schematic diagram of information transmission between spiking neurons; Figure 5 A schematic diagram of the structure of a spiking neural network provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of a pulse counting loss calculation circuit provided in an embodiment of the present application; Figure 7 A schematic diagram of the structure of a distribution rate control loss calculation circuit provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of a fusion loss calculation circuit provided in an embodiment of the present application; Figure 9 A schematic diagram of the structure of a spiking neural network training device provided in an embodiment of the present application; Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0014] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0015] In order to more clearly illustrate the embodiments of the present application, the following briefly introduces the technical terms used in the embodiments: The Dynamic Vision Sensor (DVS) is a new type of visual sensor that mimics the neural mechanisms of the biological retina. Rather than using the frame sampling method of traditional cameras, it asynchronously outputs data only when the brightness of pixels in the scene changes (i.e., events). This allows for extremely high temporal resolution (nanoseconds) and extremely low power consumption.

[0016] ANN is a computational model inspired by biological neural systems. It consists of a large number of interconnected artificial neurons (nodes) forming a network structure. Through weight transfer and nonlinear transformations between multiple layers of nodes, it can achieve features extraction, pattern recognition, and prediction of input data.

[0017] SNNs are third-generation neural network models that more closely resemble the workings of biological neural systems. Neurons transmit information in the form of "spikes," encoding information through the temporal sequence and frequency of spikes. This approach offers lower power consumption and greater parallel processing potential. It's important to note that while ANNs operate on continuous values, SNNs, based on discrete spike events, are more suitable for event-driven, real-time computing.

[0018] A field programmable gate array (FPGA) is an integrated circuit chip that can be programmed and configured using a hardware description language. It contains a large number of configurable logic cells, memory cells, and interconnect resources, allowing users to dynamically define hardware functions based on their needs.

[0019] The Integrate and Fire neuron (IF) is one of the most basic neuron models in spiking neural networks, which is used to simulate the charge accumulation and discharge process of biological neurons.

[0020] In spiking neural networks (SNNs), synaptic weights are quantified values that measure the strength of synaptic connections between neurons. They determine the degree to which a spike from one neuron influences another. Essentially, a synaptic weight is a numerical parameter that reflects the efficiency of synaptic signal transmission. Mathematically, if neuron i is connected to neuron j via a synapse, the synaptic weight can be represented as wij. The magnitude and sign of the weight are significant. A positive value indicates that the spikes from the preceding neuron tend to promote spikes from the succeeding neuron (excitatory synapses), while a negative value indicates that the spikes from the preceding neuron tend to inhibit spikes from the succeeding neuron (inhibitory synapses). During SNN operation, neurons receive signals from multiple other neurons through synapses, and the strength of these signals is determined by the synaptic weights. The larger the weight, the greater the proportion of the signal transmitted by the corresponding synapse in the total input to the target neuron, and the greater the influence on whether the target neuron spikes.

[0021] The core goal of the SNN training process in related technologies is to improve the accuracy of downstream tasks. The relationship between synaptic weights and pulse firing rate is not considered during weight optimization, and the pulse firing rate is not effectively constrained. This results in the trained spiking neural network having high pulse firing rates among interlayer neurons because the weight configuration does not take into account the firing rate regulation. This makes it difficult to fully utilize the low energy consumption advantage of SNNs, and even leads to energy consumption approaching that of traditional neural networks, seriously restricting the practical value of spiking neural networks in low-power scenarios.

[0022] In order to solve some or all of the technical problems mentioned above, the present application introduces the firing rate control loss and integrates it with the pulse count loss, thereby specifically solving the technical problem that the pulse firing rate in the pulse neural network is not effectively constrained and the energy consumption is high. Specifically, the firing rate control loss is calculated according to the pulse firing rate, and the pulse firing rate is incorporated into the training considerations. It can directly constrain the pulse firing rate to avoid excessive response of neurons to input signals due to unreasonable configuration of synaptic weights, thereby effectively controlling the pulse firing rate and maintaining it at a reasonable level; not only focusing on the pulse count loss, but also combining the firing rate control loss to calculate the fusion loss. When updating the synaptic weights, the operation is performed according to the fusion loss, which means that the training process simultaneously considers the difference between the pulse sequence and the expected pulse sequence (i.e., the pulse count loss) and the pulse firing rate. In this way, the pulse firing rate is effectively controlled, and the problem of high pulse firing rate of interlayer neurons is avoided. The pulse neural network can better play its low energy consumption advantage; updating the synaptic weights through the fusion loss makes the weight optimization more in line with the working mechanism of SNN. It not only improves the matching degree between the pulse sequence and the expectation, but also reasonably adjusts the synaptic weights to adapt to the requirements of the pulse firing rate, thereby improving the training quality and performance of the entire pulse neural network, making it more efficient and accurate when processing tasks such as event stream data.

[0023] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0024] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the pulse neural network training method depends, the specific application environment architecture or specific hardware architecture is described here.

[0025] like Figure 1 As shown, Figure 1 A schematic diagram of the specific hardware architecture relied upon for executing a spiking neural network training method provided in an embodiment of the present application. The hardware architecture includes an inter-layer firing rate collection module, a pulse firing rate statistics backpropagation gradient calculation module, a pulse step proxy gradient backpropagation gradient calculation module, and a downstream task loss calculation module. Each module has a clear division of labor: the forward module is responsible for generating data, the loss module defines the optimization objective, the gradient module addresses SNN training adaptability, and weight updates implement iterative optimization.

[0026] Among them, the SNN forward calculation is to load the current SNN weights, perform forward propagation on the input data, simulate the neuron pulse emission process, and output the pulse response of each layer. Generating the original data of downstream task loss and inter-layer firing rate is the prerequisite for reverse update.

[0027] The inter-layer firing rate collection module is inserted between layers during the forward computation of the spiking neural network. It captures the firing state of neurons in each layer of the SNN in real time and calculates the inter-layer firing rate. This requires hardware with a pulse event register to capture neuron firing states in real time. The firing rate constraint target is a preset firing rate control parameter that defines the constraints on the firing rate during SNN training, such as setting an upper limit on the total firing rate. This serves as the basis for calculating the firing rate control loss.

[0028] The downstream task loss calculation module calculates the difference between the forward output of the SNN and the expected result for a specific task. This module requires a high-precision floating-point unit (FPU) to support the calculation of complex loss functions while balancing accuracy and energy efficiency. The downstream task loss is a task-specific loss value used to quantify the quality of the SNN's task performance.

[0029] The spike rate statistics backpropagation gradient calculation module relies on a gradient calculation circuit to calculate the backpropagation gradient of the spike rate control loss based on the difference between the spike rate constraint target and the actual spike rate. Hardware support for hardware acceleration of the gradient of the custom loss function is required.

[0030] Due to the discrete characteristics of pulse emission, the pulse step proxy gradient back propagation gradient calculation module requires hardware implementation of the proxy gradient approximation circuit to convert discrete pulses into continuous and differentiable gradient signals to adapt to the back propagation algorithm.

[0031] The SNN weight update is to integrate the pulse counting loss and the firing rate control loss, and update the SNN weight according to the optimization algorithm.

[0032] The embodiments of the present application provide a training method for a spiking neural network, and the method is described in detail in conjunction with the execution process of the training method for a spiking neural network.

[0033] like Figure 2 As shown, the method includes the following steps S201 to S205: S201. Input event stream data into a pulse neural network to obtain a pulse sequence and a pulse firing rate.

[0034] The event stream data includes multidimensional vectors that record spatial light changes. These can be four-dimensional vectors that record a single spatial light change. The entire event stream data set is a collection of multiple four-dimensional vectors, with the event stream data within the collection arranged in chronological order. The sparsity and sequential nature of the event stream data ensure that neurons respond only to meaningful light changes, suppressing ineffective pulses and reducing the overall firing rate.

[0035] The four-dimensional vector records the following: space (e.g., length and width), time, and event polarity. This fully preserves the spatiotemporal relationship of light changes. Event polarity can be positive or negative, and four-dimensional vectors with different event polarities must be processed separately.

[0036] It should be noted that the mainstream method uses a fixed time frame to record data, and regular sampling is required regardless of whether the scene changes, resulting in a large amount of redundant data transmission and processing, which increases energy consumption. If no events occur in the scene space within a certain period of time, then no task scene will be generated during this period. Event stream data only generates records when the light changes, and no data is generated when there is no event. This reduces the amount of data from the source and reduces transmission and computing energy consumption. Compared with the previous fixed time frame data recording method, the use of event stream data in this application can greatly reduce data transmission.

[0037] In some embodiments, during the execution of step S201, the event stream data is first preprocessed and compressed to obtain event frames; then the event frames are input into a pulse neural network to obtain a pulse sequence and a pulse firing rate.

[0038] The above embodiment first preprocesses the event stream data into event frames, which is essentially a structured integration and compression of the original event stream: on the one hand, compression can eliminate invalid or redundant event information, reduce the amount of input data, and avoid the increase in the firing rate of neurons due to excessive response to irrelevant information; on the other hand, the event frame converts the discrete event stream in time into a structured input form that is more consistent with the SNN processing logic, so that the network can extract key features more efficiently and reduce invalid pulse firing caused by data format mismatch.

[0039] During the preprocessing and compression of event stream data to generate event frames, multidimensional vectors are mapped to corresponding three-dimensional tensors according to spatiotemporal distribution patterns. Specifically, event data is mapped to corresponding three-dimensional tensors (two-dimensional space + time) according to spatiotemporal distribution patterns. Data of two different polarities are generally processed separately, equivalent to having two channels in the event frame. The three-dimensional tensor integrates spatial coordinates and timestamps into structured data, allowing neurons to naturally capture spatiotemporal patterns and reduce ineffective spike firing.

[0040] The compressed event frame reduces the redundancy of the input data, directly reduces the processing volume of the SNN, suppresses the trend of excessive pulse firing rate from the input source, and forms a synergistic relationship with the subsequent firing rate control loss to further strengthen energy consumption control, making it easier to bring into play the energy consumption advantage of the SNN; secondly, the structured event frame improves the adaptability of data and the network, helps the SNN learn the spatiotemporal characteristics of events more accurately, reduces the feature extraction error caused by data confusion, and maintains or even improves the accuracy of downstream tasks while ensuring energy consumption reduction.

[0041] Optionally, a multi-threaded approach can be used to compress the multidimensional vector into a preset number of event frames. Mapping with a fixed number of events can ensure information validity. Setting fixed event frames for event stream data can enhance the parallelism of the process. For example, compressing all data into 10 event frames allows multi-threading to quickly generate 10 two-channel event frames each time an input sample is processed. Alternatively, a fixed time window approach can be used for mapping.

[0042] Multithreading to generate a fixed number of event frames (e.g., 10 frames) in parallel reduces processing time. For example, eight threads can reduce latency to one-eighth that of a single thread, breaking through real-time bottlenecks. A mapping strategy for a fixed number of events dynamically adjusts the time window to ensure that each frame contains sufficient information, avoiding fluctuations in the firing rate caused by data sparsity or density. This, combined with the firing rate control loss, optimizes energy consumption.

[0043] The spiking neural network is composed of a multi-layer structure. Each layer includes an operation layer and a spiking neuron. The operation layer represents the membrane voltage calculation under topological structures such as convolution, pooling, and full connection. The operation of the spiking neuron will further update the output pulse and membrane voltage according to the threshold. The firing mechanism of the spiking neuron is as follows: Figure 3 As shown, the circle u represents the neuron's membrane potential. The three axonal inputs it receives are pulse signals from other neurons or external stimuli. These inputs are transmitted through synapses, causing the neuron's membrane potential u to accumulate, corresponding to information accumulation, simulating the charge / potential accumulation process in biological neurons after receiving signals. The neuron's membrane potential u is transmitted to the threshold module thr, which continuously monitors whether the neuron's membrane potential u exceeds a preset threshold. If the neuron's membrane potential u does not reach the threshold, it remains in the integration state, awaiting new input accumulation. When the neuron's membrane potential u exceeds the threshold thr, the neuron triggers a spike, transmitting the neural code with discrete pulses, simulating the characteristic of biological neurons that fire when their potential reaches the threshold. After the spike is fired, the membrane potential u is updated, typically by a rapid reset, such as to zero or to the basal potential, returning the neuron to its initial state, ready to receive the next round of input and repeat the cycle of integration, judgment, and firing.

[0044] Information transmission between two spiking neurons is as follows Figure 4As shown in the figure, the SNN inference process is first divided into 10 simulation time steps. A single neuron can spike a maximum of 10 times during a cycle, one spike per time step. The rate encoding rule measures information as the number of spikes / preset number of time steps. This is a quantitative way of transmitting information between neurons. For example, 7 spikes correspond to an encoding value of 0.7. The spikes fired by the first neuron are transmitted as input to the second neuron. If 7 spikes are transmitted, this essentially encodes the information 0.7 to the second neuron using 7 spike events. Due to the event-driven nature of SNNs, each spike received by the second neuron triggers a computation, such as membrane potential integration and threshold determination. Receiving 7 spikes results in 7 computations, each of which consumes energy. To reduce SNN energy consumption, the key is to control the number of spikes received by the second neuron. For example, reducing the number of spikes transmitted can reduce the encoding value; alternatively, optimizing the time step distribution can keep the number of computations low. This achieves the application goals of less computation and lower energy consumption, which is consistent with the design requirements of efficient and low-power spiking neural networks.

[0045] Optionally, SNN adopts classic convolutional pooling and fully connected structures. Specifically, the following symbols can be used to represent the relevant network structure (64C3-P2) 5 -128-128-128, where 64C3 represents a convolution operation with a 3×3 kernel and 64 output channels, P2 represents a 2×2 average pooling layer, and 128 represents a fully connected layer with 128 neurons.

[0046] like Figure 5 As shown in the figure, SNN includes an N-layer structure, which takes the event stream data as the input of SNN and obtains the pulse sequence and pulse firing rate corresponding to the output of the current synaptic weight of SNN.

[0047] In some embodiments, inputting event stream data into a spiking neural network to obtain a spike rate includes: inputting the event stream data into the spiking neural network for forward propagation, and counting the spike rates of each layer of the spiking neural network. Specifically, during the forward propagation, the spike rate is calculated based on the spikes emitted by each layer.

[0048] For example, the pulse firing rate of layer l The following formula (1): (1) N in formula (1) l represents the number of spiking neurons in layer l, and T represents the number of simulation time steps of SNN. Because the number of pulses emitted includes the time step, the denominator needs to be multiplied by the two, and the numerator It represents the total number of neurons released by this layer during the entire SNN reasoning process.

[0049] The above-mentioned method for calculating the spike rate provides a standardized spike rate metric by integrating the number of neurons, simulation time steps, and total spike counts. This allows the training process to monitor and constrain the activity level of each layer in real time to avoid over-spiking. By using the spike rate as an explicit optimization target, the spike rate calculated by formula (1) directly participates in the construction of the spike rate control loss, allowing the model to reduce invalid spikes while ensuring accuracy. In addition, formula (1) aggregates discrete spike events into a continuous spike rate indicator, providing a smooth optimization target for the proxy gradient, allowing the gradient to be effectively propagated to the synaptic weights, thereby improving training efficiency.

[0050] It should be noted that the calculation process of the pulse rate is non-differentiable, so this application designs a corresponding proxy gradient to perform gradient reversal. During the gradient reversal process, the subsequent gradient of each layer needs to be propagated to the previous layer with the help of the proxy gradient. For details, please refer to the detailed description of the embodiment in the subsequent step S205.

[0051] S202: Calculate the pulse counting loss according to the expected pulse sequence and the pulse sequence corresponding to the event stream data.

[0052] Among them, the expected pulse sequence corresponding to the event stream data is the true label corresponding to the downstream task of the pulse neural network, and the pulse sequence is the actual result output by the forward calculation of the pulse neural network.

[0053] In some embodiments, for each output layer neuron of the pulse neural network, the actual cumulative number of pulses of the pulse sequence within a preset time is calculated; a difference operation is performed on the number of expected pulse sequences and the actual cumulative number of pulses to obtain a pulse difference; the pulse difference is divided by the preset time, and then squared to obtain a pulse difference square term; the pulse difference square terms corresponding to each output layer neuron are added to obtain the sum of the pulse difference square terms; the sum of the pulse difference square terms is divided by the total number of output layer neurons to obtain a pulse counting loss.

[0054] Optionally, calculate the pulse counting loss L according to formula (2) sc : (2) The spike count loss is used to measure the difference between the spike sequence output by the SNN and the expected spike sequence. In formula (2), N out It represents the number of downstream tasks, which is equal to the number of neurons in the output layer of the SNN. T is the preset time. represents the output pulse train of the neuron corresponding to category i; is the number of expected pulse trains corresponding to class i.

[0055] The above embodiment quantifies the difference between the actual and expected number of pulses, forcing the model to learn a pulse encoding method that conforms to biological laws, thereby reducing invalid firing while ensuring accuracy; the pulse difference is converted into a continuously differentiable square loss function, and effective optimization is achieved through the proxy gradient technology; the pulse counting loss implicitly constrains the firing rate through the number of expected pulse sequences. If the actual number of firings far exceeds expectations, the loss function will drive the weight adjustment to reduce the firing rate, which complements the firing rate control loss.

[0056] In some embodiments, the expected pulse sequence and the pulse sequence corresponding to the event stream data are input into a pulse count loss calculation circuit to obtain the pulse count loss. The pulse count loss calculation circuit may include an asymmetric multi-branch adder tree, a subtractor, a demultiplexer, and a multiplier. The asymmetric multi-branch adder tree includes five groups of dual-input adders at the bottom layer, two groups of dual-input adders and one independent adder at the middle layer, and one group of dual-input adders at the top layer.

[0057] like Figure 6 The pulse counting loss calculation circuit shown, is the pulse emission signal of a single neuron in 10 time steps. Use 4 layers of adders (5 at the bottom, 2 in the middle, and 1 at the top) to accumulate 10 inputs and calculate The actual total output of the addition tree is The difference is calculated through a subtractor; the difference enters the multiplier and adder modules to implement the operation of dividing by the time step to obtain the average; the demultiplexer copies the averaged difference into two identical values in preparation for the square operation; the two identical values are input into the multiplier to implement the square operation, and the pulse count loss of a single neuron is obtained.

[0058] The calculation of pulse counting loss is realized in hardware, and the mathematical operations in the formula (addition, subtraction, averaging, and squaring) are converted into connections of hardware units such as adders, subtractors, and multipliers. This allows the pulse counting loss of the pulse neural network to be efficiently calculated in hardware, supporting the training and optimization logic of the SNN.

[0059] The above step S202 directly constrains the pulse emission pattern of the neuron by comparing the difference between the actual pulse sequence and the expected pulse sequence, ensuring that the network learns the correct spatiotemporal coding pattern.

[0060] S203. Calculate the pulse emission rate control loss according to the pulse emission rate.

[0061] In some embodiments, the pulse firing rate is constrained; when the pulse firing rate is less than or equal to a firing rate threshold, the gradient corresponding to the pulse firing rate is used as the firing rate control loss. The firing rate threshold is a pre-set target for the pulse firing rate constraint. Specifically, after collecting the overall forward propagation firing rate, all pulse firing rates are summed, and then a firing rate constraint target is constructed to ensure that the pulse firing rate approaches the constraint target. Explicitly monitoring and constraining the pulse firing rate prevents excessive firing rates due to unreasonable synaptic weights, fundamentally resolving energy consumption issues.

[0062] In some embodiments, the pulse firing rates of each layer of the spiking neural network are accumulated to obtain the sum of the total firing rates; and a difference operation is performed between the sum of the total firing rates and a preset parameter to obtain the firing rate control loss.

[0063] Optionally, the emission rate control loss is shown in formula (3): (3) In formula (3), ρ is a configurable parameter that represents the target firing rate and can be set to 0.1. This allows the total firing rate to be directly incorporated into the loss function, forcing the SNN to control the firing rate within a reasonable range, fundamentally solving the problem of excessive energy consumption.

[0064] The above embodiment optimizes the sum of the firing rates of each layer, prompting the network to automatically balance the activity levels between layers and improve the overall energy efficiency ratio; it converts discrete pulse events into continuous firing rate indicators, provides a smooth optimization target for the proxy gradient, and improves training stability.

[0065] Optionally, the pulse firing rate is input into a firing rate control loss calculation circuit to obtain the firing rate control loss. The firing rate control loss calculation circuit comprises a symmetrical binary adder tree, a subtractor, a demultiplexer, and a multiplier. The symmetrical binary adder tree includes four groups of dual-input adders at the bottom layer, two groups of dual-input adders in the middle layer, and one group of dual-input adders at the top layer.

[0066] like Figure 7As shown in the figure, the firing rate control loss calculation circuit has eight inputs, f1 to f8, corresponding to the eight layers of spike rates. These are statistically derived spike rates of neurons in each layer and serve as the raw inputs for the circuit calculation. Four adders in the bottom layer accumulate inputs in pairs to achieve group summation. Two adders in the middle layer perform a secondary accumulation of the results from the bottom adders to compress the data size. Finally, a single adder in the top layer aggregates all group sums to obtain the sum of the eight firing rates. The seven adders here use an adder tree structure to efficiently perform multi-input accumulation. The sum of the adder tree outputs is subtracted from the parameter ρ via a subtractor to obtain the intermediate key quantity for the loss calculation. This difference enters a demultiplexer and is copied into two identical values to prepare for the subsequent squaring operation. The two identical differences are input to a multiplier for multiplication, ultimately outputting the firing rate control loss.

[0067] The firing rate control loss calculation circuit implements the firing rate control loss calculation in hardware. Specifically, it transforms abstract statistical and computational algorithms into connections between hardware modules such as adders and subtractors, enabling the firing rate control loss calculation to run efficiently within the hardware circuit and supporting the firing rate control function of spiking neural networks.

[0068] S204: Calculate the fusion loss based on the pulse counting loss and the firing rate control loss.

[0069] In some embodiments, an exponential transformation is performed on the firing rate control loss to obtain an exponential transformation result, which is then summed with the pulse counting loss to obtain a fusion loss. The pulse counting loss and the firing rate control loss are fused to form a unified optimization objective, enabling the network to maintain task accuracy while also taking into account the constraints of the pulse firing rate, achieving a balance between accuracy and energy consumption.

[0070] Optionally, according to formula (4), the firing rate control loss and the pulse counting loss are fused to obtain the fusion loss L all : (4) The above embodiment nonlinearly fuses the firing rate control loss and the pulse count loss through exponential transformation to form a dynamic balance mechanism. When the firing rate approaches the target value, the exponential term grows slowly, and the optimization focus shifts to accuracy; when the firing rate deviates from the target, the exponential term surges to force the firing rate to be constrained, achieving coordinated optimization of the two. The exponential transformation amplifies the firing rate control signal, ensuring that it has a reasonable weight in the total loss, and avoiding the firing rate from being out of control due to the vanishing gradient. Exponential fusion enables the loss function to adaptively adjust its sensitivity to the firing rate, giving priority to constraining high firing rates to stabilize the network in the early stages of training, and focusing on accuracy improvement in the later stages to enhance the robustness of training.

[0071] In some embodiments, the firing rate control loss and the pulse count loss are input into a fusion loss calculation circuit to obtain a fusion loss. Figure 8 As shown, the indexer is implemented as Bottom For exponential operations, the adder is responsible for the summation operation. Specifically, the emission rate controls the loss The input exponentiator uses the characteristic of the exponential function that is sensitive to large values to impose a high-intensity penalty on situations where the firing rate control loss is large, so that the model training focuses more on the control of the pulse firing rate; the exponential output firing rate control loss and the pulse counting loss after the penalty are added together through an adder to obtain the fusion loss, realizing the weighted fusion of multiple loss terms.

[0072] The aforementioned fusion loss calculation circuit achieves loss fusion through a dedicated exponentiator and adder. The exponentiator exploits the exponential function's sensitivity to large values. When the firing rate control loss is large, it generates a high-intensity penalty signal, forcing the model to prioritize reducing the firing rate, thus avoiding firing rate runaway caused by computational delays or gradient imbalances at the software level. The adder efficiently integrates the penalized firing rate control loss and the pulse counting loss, ensuring delay-free coordination at the hardware level. This ensures continuous optimization of task accuracy without weakening the constraints on the firing rate. The modular design of this circuit can be directly integrated into existing SNN hardware architectures without the need for additional software adaptation, resulting in low deployment costs.

[0073] S205. Update the synaptic weights of the spiking neural network according to the fusion loss.

[0074] The updated synaptic weights are used for the next forward calculation.

[0075] It's important to note that traditional deep learning backpropagation relies on continuously differentiable activation functions like the sigmoid function. However, two key operations in spiking neural networks are discrete and non-differentiable: inter-layer spike statistics, such as the statistical processing of spike frequency; and the spike firing process of spiking neurons, such as threshold-based 0 / 1 pulse output. The non-differentiability of these two operations can interrupt gradient backpropagation, making it impossible to directly train the network using conventional backpropagation algorithms. Therefore, this application designs a proxy gradient to approximate the gradient at non-differentiable points, ensuring continuous training of spiking neural networks.

[0076] In some embodiments, a proxy gradient is used to perform back propagation calculation on the fusion loss to obtain a weight gradient; and the weight gradient is used to update the synaptic weight.

[0077] Optionally, for the pulse statistics process between layers, the weight gradient is calculated using the following formula (5): (5) In formula (5), X is the pulse tensor, which contains the spatiotemporal information of neuronal pulse emission; σ is the sigmoid function; For the neuron pulse emission process, the gradient of the sigmoid function is used, that is, As an approximation, although the actual firing of a neuron is a discrete behavior similar to a step function, and the step point is not differentiable, the gradient of the sigmoid function can simulate the gradient change trend before and after the pulse is fired in the continuous domain, thereby providing an effective direction for parameter update during backpropagation.

[0078] The above steps S201 to S205 are repeated to update the synaptic weights of the spiking neural network until the number of training cycles of the spiking neural network reaches a preset value, or the accuracy of the synaptic weights on the validation set converges.

[0079] Experimental data show that the pulse neural network trained in this application has an inter-layer firing rate reduced from 0.30 to 0.15 without affecting the task accuracy, thereby reducing the energy consumption of the pulse neural network.

[0080] In summary, the present invention provides a training method for a spiking neural network. By pre-processing and compressing the data into event frames and mapping them into three-dimensional tensors, the method uses multi-threaded parallel generation of a fixed number of event frames. This method reduces the amount of data transmission from the source, preserves accurate spatiotemporal correlations, avoids excessive response of neurons to redundant information, directly suppresses the unreasonable increase of the pulse firing rate, lays the foundation for subsequent energy consumption optimization, and improves the adaptability of data to SNN. During the forward calculation of the SNN, the pulse firing rate of each layer is counted, and then the firing rate is minimized and the constraint is integrated into the weight update process of the spiking neural network. In response to the problem that mainstream methods ignore the correlation between synaptic weights and firing rates and only focus on accuracy, an innovative dual loss fusion mechanism is introduced: the pulse counting loss ensures task accuracy by quantifying the difference between actual and expected pulse sequences; the firing rate control loss strengthens the penalty for high firing rates through exponential transformation, forcing the total firing rate between layers to be constrained. The two are fused through the loss calculation circuit to achieve efficient hardware-level collaboration. This design reduces the pulse firing rate without sacrificing the accuracy of downstream tasks, which in turn reduces energy consumption, and improves training stability by dynamically balancing gradient weights.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0082] An embodiment of the present application further provides a spiking neural network training device, the device comprising: A data preparation module 901 is used to input event stream data into a spiking neural network to obtain a pulse sequence and a pulse firing rate; a pulse count loss calculation module 902 for calculating the pulse count loss based on the expected pulse sequence and the pulse sequence corresponding to the event stream data; The firing rate control loss calculation module 903 is used to calculate the firing rate control loss according to the pulse firing rate; A fusion loss calculation module 904 is used to calculate the fusion loss based on the pulse counting loss and the firing rate control loss; The weight updating module 905 is used to update the synaptic weights of the spiking neural network according to the fusion loss.

[0083] As an optional implementation provided in an embodiment of the present application, the data preparation module 901 is specifically used to: preprocess the event stream data and compress it to obtain event frames; input the event frames into the pulse neural network to obtain pulse sequences and pulse emission rates.

[0084] As an optional implementation provided in an embodiment of the present application, event stream data includes a multidimensional vector that records spatial light change information; the data preparation module 901 preprocesses the event stream data and compresses it to obtain an event frame, which is specifically used to: map the multidimensional vector to the corresponding three-dimensional tensor according to the spatiotemporal distribution law to obtain the event frame.

[0085] As an optional implementation provided in an embodiment of the present application, the data preparation module 901 preprocesses the event stream data and compresses it to obtain event frames, specifically for: using a multi-threaded method to compress the multi-dimensional vector into a preset number of event frames.

[0086] As an optional implementation provided in an embodiment of the present application, the data preparation module 901 is specifically used to: input event stream data into the pulse neural network for forward propagation, and count the pulse emission rates of each layer of the pulse neural network.

[0087] As an optional implementation provided in an embodiment of the present application, the pulse count loss calculation module 902 is specifically used to: calculate the actual cumulative number of pulses of the pulse sequence within a preset time for each output layer neuron of the pulse neural network; perform a difference operation on the number of expected pulse sequences and the actual cumulative number of pulses to obtain a pulse difference; divide the pulse difference by the preset time, and then perform a square operation to obtain a pulse difference square term; add the pulse difference square terms corresponding to each output layer neuron to obtain the sum of the pulse difference square terms; divide the sum of the pulse difference square terms by the total number of output layer neurons to obtain a pulse count loss.

[0088] As an optional implementation provided in an embodiment of the present application, the pulse count loss calculation module 902 is specifically used to: input the expected pulse sequence and pulse sequence corresponding to the event stream data into the pulse count loss calculation circuit to obtain the pulse count loss; wherein the pulse count loss calculation circuit includes: an asymmetric multi-branch adder tree, a subtractor, a demultiplexer and a multiplier.

[0089] As an optional implementation provided in an embodiment of the present application, the firing rate control loss calculation module 903 is specifically used to: accumulate the pulse firing rates of each layer of the pulse neural network to obtain the sum of the total firing rates; perform a difference operation on the sum of the total firing rates and a preset parameter to obtain the firing rate control loss.

[0090] As an optional implementation provided in an embodiment of the present application, the firing rate control loss calculation module 903 is specifically used to: input the pulse firing rate into the firing rate control loss calculation circuit to obtain the firing rate control loss; wherein the firing rate control loss calculation circuit includes: a symmetrical binary tree adder tree, a subtractor, a demultiplexer and a multiplier.

[0091] As an optional implementation provided in an embodiment of the present application, the fusion loss calculation module 904 is specifically used to: perform an exponential transformation on the firing rate control loss to obtain an exponential transformation result; and perform a sum operation on the exponential transformation result and the pulse counting loss to obtain a fusion loss.

[0092] As an optional implementation provided in an embodiment of the present application, the weight update module 905 is specifically used to: use a proxy gradient method to perform backpropagation calculation on the fusion loss to obtain a weight gradient; and use the weight gradient to update the synaptic weights of the pulse neural network.

[0093] For the description of the features in the embodiment corresponding to the training device of the pulse neural network, please refer to the relevant description of the embodiment corresponding to the training method of the pulse neural network, and no further details will be given here.

[0094] The embodiment of the present application also provides an electronic device, such as Figure 10 As shown, it includes a memory 1001 and a processor 1002, wherein the memory 1001 stores a computer program, and the processor 1002 is configured to run the computer program to execute the steps in any of the above-mentioned pulse neural network training method embodiments.

[0095] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned pulse neural network training method embodiments when running.

[0096] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0097] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned pulse neural network training method embodiments are implemented.

[0098] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned pulse neural network training method embodiments are implemented.

[0099] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0100] The above is a detailed introduction to the training method, device, equipment, medium and product of a pulse neural network provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for training a spiking neural network, characterized in that: include: Input event stream data into the spiking neural network to obtain spike trains and spike firing rates; Calculating a pulse count loss based on an expected pulse sequence corresponding to the event stream data and the pulse sequence; calculating a firing rate control loss based on the pulse firing rate; calculating a fusion loss according to the pulse count loss and the firing rate control loss; The synaptic weights of the spiking neural network are updated according to the fusion loss.

2. The method according to claim 1, characterized in that The method of inputting event stream data into a pulse neural network to obtain a pulse sequence and a pulse firing rate includes: Preprocessing the event stream data and compressing it to obtain event frames; The event frame is input into the spiking neural network to obtain the spike train and the spike firing rate.

3. The method according to claim 2, characterized in that The event stream data includes a multi-dimensional vector recording spatial light change information; The preprocessing of the event stream data and compressing to obtain an event frame includes: According to the spatiotemporal distribution law, the multidimensional vector is mapped into a corresponding three-dimensional tensor to obtain the event frame.

4. The method according to claim 3, characterized in that The preprocessing of the event stream data and compressing to obtain an event frame includes: The multi-dimensional vector is compressed into a preset number of event frames in a multi-threaded manner.

5. The method according to claim 1, characterized in that Input event stream data into the spiking neural network to obtain the spike rate, including: The event stream data is input into the spiking neural network for forward propagation, and the pulse emission rate of each layer of the spiking neural network is counted.

6. The method according to claim 1, characterized in that The calculating the pulse count loss according to the expected pulse sequence corresponding to the event stream data and the pulse sequence includes: For each output layer neuron of the spiking neural network, the actual cumulative number of pulses of the pulse sequence within a preset time is calculated; a difference operation is performed on the number of the expected pulse sequence and the actual cumulative number of pulses to obtain a pulse difference; the pulse difference is divided by the preset time, and then a square operation is performed to obtain a squared term of the pulse difference; Performing a sum operation on the squared pulse difference terms corresponding to the neurons in the output layer to obtain a sum of the squared pulse difference terms; The spike count loss is obtained by dividing the sum of the squared spike differences by the total number of neurons in the output layer.

7. The method according to claim 1, characterized in that The calculating the pulse count loss according to the expected pulse sequence corresponding to the event stream data and the pulse sequence includes: Inputting the expected pulse sequence corresponding to the event stream data and the pulse sequence into a pulse count loss calculation circuit to obtain the pulse count loss; The pulse count loss calculation circuit includes an asymmetric multi-branch adder tree, a subtractor, a demultiplexer and a multiplier.

8. The method according to claim 1, characterized in that Calculating the pulse emission rate control loss according to the pulse emission rate includes: Accumulating the pulse firing rates of each layer of the spiking neural network to obtain a sum of the total firing rates; A difference operation is performed on the sum of the total issuance rates and a preset parameter to obtain the issuance rate control loss.

9. The method according to claim 1, characterized in that Calculating the pulse emission rate control loss according to the pulse emission rate includes: Inputting the pulse firing rate into a firing rate control loss calculation circuit to obtain the firing rate control loss; The firing rate control loss calculation circuit includes: a symmetrical binary tree addition tree, a subtractor, a multiplexer and a multiplier.

10. The method according to claim 1, characterized in that Calculate the fusion loss based on the pulse count loss and the firing rate control loss, including: Performing an exponential transformation on the emission rate control loss to obtain an exponential transformation result; The exponential transformation result and the pulse count loss are summed to obtain the fusion loss.

11. The method according to claim 1, wherein The updating of the spiking neural network according to the fusion loss comprises: Backpropagation calculation is performed on the fusion loss using a proxy gradient method to obtain a weight gradient; The synaptic weights of the spiking neural network are updated using the weight gradient.

12. A training device for a pulse neural network, characterized in that: include: A data preparation module is used to input event stream data into the spiking neural network to obtain spike trains and spike firing rates; a pulse count loss calculation module, configured to calculate the pulse count loss based on the expected pulse sequence corresponding to the event stream data and the pulse sequence; a firing rate control loss calculation module, configured to calculate the firing rate control loss based on the pulse firing rate; a fusion loss calculation module, configured to calculate a fusion loss based on the pulse count loss and the firing rate control loss; A weight updating module is used to update the synaptic weights of the spiking neural network according to the fusion loss.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the pulse neural network training method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the pulse neural network training method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the pulse neural network training method according to any one of claims 1 to 11 are implemented.

Images