Shared balanced potential gated spiking neuron and neural network construction method thereof

Abstract

The invention provides a shared balanced potential gated spiking neuron and a neural network construction method thereof. The method comprises the following steps: S1, constructing a shared balanced potential gated neuron; s2, constructing an initial dynamic pulse neural network by using the shared balance potential gating neurons; and S3, training the initial dynamic spiking neural network through a time-based back propagation method to obtain an optimized dynamic spiking neural network. According to the method, the learning ability, robustness and anti-interference ability of the spiking neural network can be remarkably improved, and the parameter quantity and calculation overhead of the spiking neural network are reduced.

Description

Technical Field

[0001] This invention relates to the fields of neuromorphic computing and spiking neural networks, specifically to a shared equilibrium potential-gated spiking neuron and a method for constructing its network. Background Technology

[0002] With the rapid evolution of artificial intelligence, brain-inspired computing has become a key research direction driving its progress. Its core lies in constructing artificial neural networks that can simulate the basic characteristics of biological neurons and their interconnections. Currently, deep learning-based artificial neural networks have achieved breakthroughs in many tasks such as image recognition and natural language processing, with performance in some tasks even approaching or surpassing human levels. However, such networks are typically accompanied by high energy consumption, thus limiting their deployment in energy-sensitive scenarios such as edge computing and mobile devices. In contrast, the human brain consumes only about 20 watts when performing complex cognitive tasks, demonstrating superior energy efficiency. Therefore, the advantages of the human brain in cognitive function and energy consumption control have become an important reference for brain-inspired artificial intelligence research.

[0003] Because brain information processing relies on the transmission of neural signals in the form of spikes, researchers have proposed spiking neural networks (SNNs). These networks not only constitute key components of neuromorphic computing systems but are also considered powerful tools for understanding the brain's information processing and learning mechanisms. However, the nonlinear dynamics and time-dependent characteristics of spiking neurons make their behavior complex, posing a significant challenge to the training of SNNs and limiting further improvements in their learning performance. Therefore, integrating existing mature artificial neural network training methods and drawing on relevant findings in neuroscience has become an important research path for enhancing the learning capabilities of SNNs.

[0004] Currently, the most common spiking neuron in engineering applications is the Leaky Integrate-and-Fire (LIF) model. This neuron typically uses a fixed membrane potential leakage parameter or a fixed time constant to describe the neuron's potential decay process. Its computational structure is simple, but the membrane potential decay mechanism is independent of input activity, making it difficult to adaptively adjust information to maintain the time scale according to the input state. Therefore, it exhibits limited memory capacity and sensitivity to noise perturbations in complex temporal tasks.

[0005] To overcome the aforementioned shortcomings, some studies have introduced dynamic conductance or gating mechanisms, enabling the evolution of neuronal membrane potentials to be regulated by synaptic input activity, thereby enhancing the ability to model temporal information. Among these, Dynamic Gated Neurons (DGNs) introduce dynamic gating parameters and equilibrium potentials into synapses, making membrane potential changes dependent on the dynamic state of each synapse, thus improving the computational power and robustness of the network to some extent.

[0006] However, existing DGNs typically assign independent balancing potential parameters to each synapse, resulting in a linear increase in the number of trainable parameters with the number of synapses. When the network size is large, this design significantly increases the network's parameter size, computational complexity, and storage overhead, making it unsuitable for deployment in resource-constrained neuromorphic chips, embedded systems, and low-power edge devices. Furthermore, there is some redundancy among the multiple balancing potential parameters, limiting their contribution to improving core computing power.

[0007] Therefore, how to reduce the number of equilibrium potential-related parameters, reduce network complexity and energy consumption, while retaining the core computational mechanism of dynamic gating neurons regulating information flow based on input activity has become an urgent technical problem to be solved. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention provides a method for constructing a lightweight dynamic gated spiking neural network based on shared equilibrium-gated neurons (SEGN). All synapses of a neuron share a single equilibrium potential parameter. While retaining the ability to dynamically adjust the amount of information stored based on input information, this method improves parameter efficiency and engineering feasibility, opening up new directions for research in related fields.

[0009] To address the problems of the existing technology, the present invention adopts the following technical solution: A shared equilibrium potential-gated spiking neuron, wherein the gated spiking neuron updates the membrane potential for each input synapse. Reuse the gating weights corresponding to the input synapse Among them, the original synaptic weights Replace with equilibrium potential With gate weights The product of, and, will balance the potentials of all input synapses connected to this gated spiking neuron. Set them to the same shared parameter ;include: --Input processing module, used to generate input pulse information ; --Gated modulation module, used to generate and update gated variables ; in: This represents the static decay coefficient of the neuron. It is a truncation function; --Membrane potential update module, used to update neuronal membrane potentials under shared equilibrium potential parameter constraints. ; in For neurons in Membrane potential at time t, The threshold of the neuron; To simulate the time step; --Pulse generation module, used to generate pulse output based on membrane potential. ; in: This represents the substitution derivative, which functionally approximates the Herveside unit step function.

[0010] This invention also includes the following technical solutions: A method for constructing a dynamic spiking neural network using neurons, comprising: Select a regular speech dataset or event dataset and divide it into a training set and a test set; By setting the equilibrium potential as a shared parameter and reusing the gating weights of each input synapse, the gating weights of each input synapse of the neuron are optimized. The weights are reshaped to achieve gated spiking neurons with shared equilibrium potentials; An initial dynamic spiking neural network is constructed based on the shared equilibrium potential-gated spiking neurons; An optimized dynamic spiking neural network is obtained by training the initial dynamic spiking neural network using the backpropagation method over time.

[0011] Furthermore, by setting the equilibrium potential as a shared parameter and reusing the gating weights of each input synapse, the gating weights of each input synapse of the neuron are applied. The weights are reshaped to achieve gated spiking neurons sharing a common equilibrium potential; including: The shared equilibrium potential will be determined according to the following formula. and gating weights Multiplication remodels each input synapse of the neuron Trainable weights , to obtain the neuron in membrane potential at time 1 : in: The threshold of the neuron; To simulate the time step; This refers to the static attenuation parameter; For the first The gating weights corresponding to the synapses; The first one connected to this neuron Input information from a synapse; This refers to the shared equilibrium potential parameter corresponding to this neuron; Here is the dynamic membrane potential decay parameter, where: The dynamic membrane potential decay parameter The calculation formula is as follows; in: This is a cutoff function that restricts the calculation result to the interval between 0 and 1; According to neurons Membrane potential pulse at time t and the previous time t The pulse emission status at this moment is obtained. : in: This represents the substitution derivative, which functionally approximates the Herveside unit step function.

[0012] Furthermore, the initial dynamic spiking neural network is trained using a backpropagation method over time to obtain an optimized dynamic spiking neural network, including: Based on the initial dynamic spiking neural network, the following formula is used to calculate... Pulse output at time ; in: For output layer The weight, For the first The output pulse of the layer, It is the internet Output data at any given time.

[0013] Based on the initial dynamic spiking neural network Calculate the category label from the output data at each time step: The loss function is calculated based on the classification labels and the true labels using the following formula: in: For real labels, : is the cross-entropy loss function; According to the loss function The gradient is propagated along the computation graph in both time and space dimensions before the weights of each neuron are updated. and equilibrium potential .

[0014] Beneficial effects 1. This invention fully preserves the gating mechanism of the membrane potential evolution process, which is dynamically regulated by the synaptic input state. The neuronal membrane potential can adaptively adjust its decay rate according to the intensity of input activity and historical state, thereby effectively suppressing membrane potential fluctuations caused by noise, impulse perturbations, or input uncertainties. Compared with integral-leakage-fire neurons that use fixed membrane potential decay parameters, the spiking neural network constructed in this invention exhibits more stable output performance under complex input conditions, significantly improving the overall robustness and anti-interference ability of the network.

[0015] 2. This invention significantly reduces the number of trainable parameters in the network and improves parameter utilization efficiency by simplifying the equilibrium potential parameters related to the number of synapses in the original dynamically gated neurons to a single shared equilibrium potential parameter at the neuron level. The reduced parameter size correspondingly reduces the computational and storage overhead required during training and inference, which helps reduce system complexity and improve engineering feasibility. Simultaneously, the reduced parameter redundancy makes the gradient propagation process more stable, helping to alleviate instability during training, thereby improving the network's training stability and generalization performance on unknown data. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an embodiment of the training of a shared equilibrium potential gated neuron and its dynamic spiking neural network in this invention. Figure 2 It is a dynamic variable and The computational graph of the update process; Figure 3 This is a diagram illustrating the noise effects used in the verification phase; Figure 4 This shows that the network constructed by the neurons proposed in this invention can still maintain the highest accuracy under different noise types and intensities. Among them, the diamond shape represents the network constructed by the neurons proposed in this invention, and the others are representative networks constructed by neurons from several spiking neural networks. Figure 5 This demonstrates that the neural network constructed using the proposed method can maintain the highest accuracy under different types and intensities of interference. The diamond-shaped network represents the neural network constructed using the proposed method, while the others are representative neural networks constructed using neurons from several spiking neural networks. Detailed Implementation

[0017] The following is in conjunction with the appendix Figure 1 ~Appendix Figure 5 The present invention will be described in detail below: A method for constructing a dynamic spiking neural network based on shared equilibrium potential-gated neurons includes: S1: Each input synapse during neuron membrane potential update Trainable weights Change to equilibrium potential and gating weights Multiply, we get Then all synapses connected to that neuron will be... The equilibrium potential of ) Unify into the same shared parameters The membrane potential φ of a neuron satisfies the following continuous-time dynamic equation: (1) in: To fix the leakage parameters; For the first The gating weights corresponding to the synapses; The first one connected to this neuron Input information from a synapse; This represents the shared equilibrium potential parameter corresponding to this neuron.

[0018] The dynamic equations are analyzed and simplified to obtain a discrete mathematical model. This model utilizes... This represents the dynamic membrane potential decay coefficient. Neuron membrane potential. The current state is represented as: (2) in For neurons in Membrane potential at time t, This represents the threshold of the neuron. To simulate the time step. In the shared equilibrium potential-gated neuron, all synapses connected to the same neuron ( Shared single equilibrium potential parameter and through With dynamic gating items They jointly regulate the evolution of neuronal membrane potential.

[0019] The dynamic membrane potential decay coefficient of neurons It is obtained from the following formula: (3) in: This represents the static decay coefficient of the neuron. This is a cutoff function that restricts the calculation result to the interval between 0 and 1; Based on the neuron's previous moment The pulse firing status is obtained at this moment. Pulse delivery status: (4) in: This represents the substitution derivative, which functionally approximates the Heaviside unit step function; S2: Construct an initial dynamic spiking neural network based on the shared equilibrium potential-gated spiking neurons; S3: A dynamic spiking neural network constructed by training shared equilibrium potential-gated neurons. It primarily employs backpropagation through times (BPTT) and can be applied to single-layer and multi-layer networks. The formula for updating the weights is as follows.

[0020] The Internet The output at time t is The classification result is based on the average of these outputs across all time steps, i.e. And the loss function Defined as average output, the usual formula is: , among them For the true classification results The cross-entropy loss function is typically used.

[0021] like Figure 1 As shown, this invention constructs a method for a shared equilibrium potential-gated spiking neuron network: 101: Prepare the dataset. You can choose a regular speech dataset or an event dataset and divide it into a training set and a test set.

[0022] 102: Constructing shared equilibrium potential-gated neurons, wherein the gated spiking neurons are constructed by controlling the neuronal membrane potential at each input synapse. weight Change to constant potential and gating weights Multiplication, and simultaneously, reuse of the synapse. Gating weights Then the balanced potential of all synapses connected to this neuron will be... Unify into the same shared parameters ; in: To fix the leakage parameters; For the first The gating weights corresponding to the synapses; The first one connected to this neuron Input information from a synapse; This represents the shared equilibrium potential parameter corresponding to this neuron.

[0023] The dynamic equations are analyzed and simplified to obtain a discrete mathematical model.

[0024] like Figure 2 As shown, by analyzing and simplifying the differential equations, the discrete mathematical model is obtained: (5) (6) (7) (8) This invention uses dynamic parameters Updating the neuron's membrane potential yields the neuron's state at the current moment. : in For neurons in Membrane potential at time t, This represents the threshold of the neuron. To simulate the time step. In the shared equilibrium potential-gated neuron, all synapses connected to the same neuron ( Shared single equilibrium potential parameter and through With dynamic gating items They jointly regulate the evolution of neuronal membrane potential.

[0025] The dynamic membrane potential decay coefficient of neurons It is obtained from the following formula: in: This represents the static decay coefficient of the neuron. This is a cutoff function that restricts the calculation result to the interval between 0 and 1; Based on the neuron's previous moment The pulse firing status is obtained at this moment. Pulse delivery status: in: This represents the substitution derivative, which functionally approximates the Heaviside unit step function; 103: Construct a spiking neural network based on shared equilibrium potential gated neurons, and use the mathematical formula of the above neurons to construct a specific network structure for a specific task.

[0026] 104: Training a spiking neural network; the trainable parameters in the network mainly include... and E The formula for updating the weights, i.e. the derivation process, is obtained from formula (9-13).

[0027] This invention describes a dynamic spiking neural network constructed by training shared equilibrium potential-gated neurons. It primarily employs backpropagation through times (BPTT) and can be applied to both single-layer and multi-layer networks. The formula for updating the weights is as follows.

[0028] The Internet The output at time t is The classification result is based on the average of these outputs across all time steps, i.e. And the loss function Defined as average output, the usual formula is: , among them For the true classification results The cross-entropy loss function is typically used.

[0029] The learning algorithm used in this invention unfolds the iterative process described in equation (7) and propagates gradients along the computation graph in the time and space dimensions, subsequently at all time steps. The weight update for each layer is determined in the middle: (9) (10) (11) (12) (13) in: Representative substitution derivative The truncation function in formula (7) The derivative; for At point The value at that location. Ultimately, the weight can be obtained. and equilibrium potential Update.

[0030] 105: Test the performance of the spiking neural network on an interference-free test set. After training, test directly using the original partitioned test set.

[0031] 106: Test the performance of the spiking neural network with added noise interference on the test set. After training, add various noises and interferences to the original test set and test the network's performance under different interference intensities.

[0032] We added three types of noise and three types of interference. The noise component is as follows: Additive noise: Randomly adding some pulses to the input pulse sequence.

[0033] Subtractive noise: Randomly reduces some pulses in the input pulse sequence.

[0034] Mixed noise (Mixd): Randomly adds and removes some pulses.

[0035] The three types of interference are: The Fast Gradient Sign Method (FGSM) generates adversarial examples quickly in one step by using the sign of the gradient of the input with respect to the loss function. The formula is as follows: (14) The Basic Iterative Method (BIM) divides the small perturbations of FGSM into multiple steps, with each perturbation representing a small step, accumulating to a larger counter-effect. The formula is as follows: (15) Projected Gradient Descent (PGD): A generalized form of BIM, incorporating random starting points and projection operations, it is one of the most effective white-box attacks. The formula is as follows: (16) The effects of all noise and interference are visible. Figure 3 The original sampling points and the effects under various perturbations are shown respectively. Black represents the pulse point and white represents the background sound.

[0036] 107: Calculate the number of parameters used by the spiking neural network and its energy consumption during operation. After training, calculate the number of parameters used and the energy consumption of the network during operation on the original partitioned test set. See Tables 1 and 2 for details.

[0037] We calculated the number of neurons in the network ( m , n ), and measured the layer-by-layer firing rate of neurons ( , To obtain the total energy consumption, we substitute these trigger rates into the formula in Table 2 and use the estimated energy consumption of a 45nm CMOS transistor, where the energy consumption of AC and MAC operations are respectively... and This leads to the empirical energy values ​​reported in the last column of Table 2.

[0038] Table 1

[0039] Table 2

[0040] See the overall process Figure 1 .

[0041] The final effect of this embodiment is as follows: Figure 4 and Figure 5 As shown.

[0042] Device Examples The present invention also provides a gated spiking neuron device based on a shared equilibrium potential, the device comprising: The input processing module is used to generate input pulse information. ; The gated modulation module is used to generate and update the gated variables. ; The membrane potential update module is used to update the neuronal membrane potential under the constraint of shared equilibrium potential parameters. ; The pulse generation module is used to generate pulse output based on the membrane potential. ; Each module can be implemented by software, firmware, or a combination thereof, and is used to execute the corresponding steps in the foregoing method embodiments. The specific working process can be referred to the method embodiments, and will not be repeated here.

[0043] Electronic device examples The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor, when executing the computer program, implements any of the aforementioned method embodiments.

[0044] Storage Media Examples The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the aforementioned method embodiments.

Claims

1. A shared equilibrium potential-gated spiking neuron, characterized in that: When the gated spiking neuron updates its membrane potential, it checks each input synapse... Reuse the gating weights corresponding to the input synapse Among them, the original synaptic weights Replace with equilibrium potential With gate weights The product of, and, will balance the potentials of all input synapses connected to this gated spiking neuron. Set them to the same shared parameter ;include: --Input processing module, used to generate input pulse information ; --Gated modulation module, used to generate and update gated variables ; in: This represents the static decay coefficient of the neuron. It is a truncation function; --Membrane potential update module, used to update neuronal membrane potentials under shared equilibrium potential parameter constraints. ; in: For neurons in Membrane potential at time t, The threshold of the neuron; To simulate the time step; --Pulse generation module, used to generate pulse output based on membrane potential. ; in: It represents the substitution derivative, which functionally corresponds to the Heaviside unit step function.

2. A method for constructing a dynamic spiking neural network using neurons as described in claim 1, characterized in that, include: Select a regular speech dataset or event dataset and divide it into a training set and a test set; By setting the equilibrium potential as a shared parameter and reusing the gating weights of each input synapse, the gating weights of each input synapse of the neuron are optimized. The weights are reshaped to achieve gated spiking neurons with shared equilibrium potentials; An initial dynamic spiking neural network is constructed based on the shared equilibrium potential-gated spiking neurons; An optimized dynamic spiking neural network is obtained by training the initial dynamic spiking neural network using the backpropagation method over time.

3. The method for constructing a dynamic spiking neural network according to claim 2, characterized in that, By setting the equilibrium potential as a shared parameter and reusing the gating weights of each input synapse, the gating weights of each input synapse of the neuron are optimized. The weights are reshaped to achieve gated spiking neurons sharing a common equilibrium potential; including: The shared equilibrium potential will be determined according to the following formula. and gating weights Multiplication remodels each input synapse of the neuron Trainable weights , to obtain the neuron in membrane potential at time 1 : in: The threshold of the neuron; To simulate the time step; This refers to the static attenuation parameter; For the first The gating weights corresponding to the synapses; The first one connected to this neuron Input information from a synapse; This refers to the shared equilibrium potential parameter corresponding to this neuron; Here is the dynamic membrane potential decay parameter, where: The dynamic membrane potential decay parameter The calculation formula is as follows; in: This is a cutoff function that restricts the calculation result to the interval between 0 and 1; According to neurons Membrane potential pulse at time t and the previous time t The pulse emission status at this moment is obtained. : in: It represents the substitution derivative, which functionally corresponds to the Heaviside unit step function.

4. The method for constructing a dynamic spiking neural network according to claim 2, characterized in that, The initial dynamic spiking neural network is trained using a backpropagation method over time to obtain an optimized dynamic spiking neural network, including: Based on the initial dynamic spiking neural network, the following formula is used to calculate... Pulse output at time: ； in: For output layer The weight, For the first The output pulse of the layer, It is the internet Output data at any given time; Based on the initial dynamic spiking neural network Calculate the category label from the output data at each time step: The loss function is calculated based on the classification labels and the true labels using the following formula: in: For real labels, : is the cross-entropy loss function; According to the loss function The gradient is propagated along the computation graph in both time and space dimensions before the weights of each neuron are updated. and equilibrium potential .

Images