Pulse neural network method based on dynamic synapses and application thereof
By integrating a simplified dynamic synaptic neuron model with a spiking neural network, the learning ability and energy efficiency of the spiking neural network are improved, making it suitable for low-power real-time processing, brain-computer interfaces, and neuroscience research.
Patent Information
- Application Number
- CN202510439215.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-09-05
AI Technical Summary
The complex dynamic characteristics of synapses in existing spiking neural networks limit their learning capabilities, and high energy consumption becomes a bottleneck in their applications, making it difficult to effectively integrate dynamic synaptic characteristics to enhance learning capabilities.
A simplified dynamic synaptic neuron model is adopted. By constructing a dynamic synaptic neuron model based on the short-term synaptic plasticity mechanism and integrating it with the pulse neuron network, the membrane potential expression is updated, and the model parameters are trained in combination with the time backpropagation algorithm to improve the learning ability.
It realizes a low-energy, high-performance spiking neural network with improved noise robustness and stability, making it suitable for applications in edge devices and neuromorphic chips and enhancing adaptability to input changes.
Smart Images
Figure CN120597953A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of brain-like computing and spiking neural networks, and in particular to a spiking neuron network method based on dynamic synapses and its application. Background Art
[0002] Brain-inspired computing, a key cornerstone of modern artificial intelligence, lies at its core in the design of artificial neural networks that highly abstract the properties of biological neurons and their connectivity. While current artificial neural networks powered by deep learning technologies have reached or surpassed human performance in many tasks, their high energy consumption has become a significant bottleneck for widespread application. In comparison, the human brain requires only approximately 20 watts of energy to perform complex cognitive tasks. The human brain's remarkable cognitive abilities and efficient energy consumption provide inspiration and guidance for the development of brain-inspired artificial intelligence.
[0003] The brain's ability to transmit and process information based on impulses has given rise to spiking neural networks (SNNs). As a new type of neural network, SNNs not only form the core of neuromorphic computing systems but also hold promise for revealing the underlying mechanisms of information processing and learning in the brain. However, due to the complexity of neuronal dynamics, training SNNs is challenging, often limiting their learning capabilities. Therefore, exploring ways to enhance the learning capabilities of SNNs is crucial, whether leveraging existing artificial neural network technology or drawing inspiration from neuroscience.
[0004] Most current research on SNNs assumes that synaptic connections are static and unchanging, with their effectiveness changing only on slow timescales during training and remaining constant at all other times. However, short-term plasticity (STP) is ubiquitous across nearly all cortical regions. In the central nervous system, neurons transmit information via chemical synapses in the form of electrical impulses. The effectiveness of these synapses is modulated according to the demands of development, learning, and memory. Chemical synapses adjust their strength in an activity-dependent manner over a wide range of timescales, from milliseconds to seconds. Therefore, exploring the computational potential of short-term plasticity in SNN learning is of great significance and research value. To address this issue, we propose a simplified neuronal model with dynamic synapses, demonstrating the important role of dynamic synapses in enhancing the learning ability of SNNs.
[0005] The complex dynamic properties of synapses in existing technologies pose a significant challenge to their integration into spiking neural networks, thus limiting research progress in this field. Inspired by the short-term plasticity exhibited by dynamic synapses in the human brain. Summary of the Invention
[0006] To address the challenges of the existing technology, the present invention provides a concise and efficient dynamic synaptic neuron model that successfully simplifies this complexity. This model is capable of simulating static, inhibitory, and potentiating synaptic behavior, significantly improving the feasibility of integrating dynamic synapses into spiking neural networks and opening up new research directions in related fields. The simplicity of the dynamic synaptic neuron model makes it particularly suitable for software simulation and hardware implementation of neuromorphic systems, providing broad prospects for its further application.
[0007] In order to solve the problems of the prior art, the present invention adopts the following technical solutions:
[0008] A pulse neuron network method based on dynamic synapses, the system is based on a pulse neural network,
[0009] include:
[0010] S1: Construction of the first dynamic synaptic neuron model based on the short-term synaptic plasticity mechanism;
[0011] S2: Optimizing the parameters of the first dynamic synaptic neuron model to construct a second dynamic synaptic neuron model;
[0012] S3: Fusing the second dynamic synaptic neuron model with the spiking neuron network to construct a dynamic synaptic spiking neuron network; including:
[0013] Add a corresponding dynamic variable u to each static synapse i in the spiking neuron network i ;
[0014] By spiking the synaptic weights w in the neuronal network i With the dynamic variable u i The product w i ·u i Instead of the synaptic weight w in the spiking neuron network i , the membrane potential V(t) of the neuron in the spiking neuron network at time t in the updated dynamic synapse is expressed as:
[0015]
[0016] Where: w i represents the weight of synapse i, z t-1 represents the output of the neuron at time t-1, θ represents the threshold, τ m represents the time constant of the membrane potential, Δt represents the time step, represents the input of the i-th afferent neuron at time t;
[0017] S4: Training spiking neuron networks with dynamic synapses.
[0018] Furthermore, the process of constructing the first dynamic synaptic neuron model based on the synaptic plasticity method in step S1 includes:
[0019] The dynamic synaptic neuron mathematical model is established according to the following formula:
[0020]
[0021] Where: τ u represents the synaptic dynamic time constant, δ represents the Dirac function, t sp represents the arrival time of the presynaptic pulse, u represents the dynamic variable, and U represents the time at t sp The number of points added at time u, u - Indicates the value of u before the input pulse arrives, B indicates the dynamic load coefficient, only in {B fac ,B dep} determines whether the synaptic behavior is inhibitory or potentiating;
[0022] The first dynamic synaptic neuron model is obtained by performing discretization processing on the dynamic variable u in the dynamic synaptic neuron mathematical model according to the following formula:
[0023] u t =ρ u [u t-1 (1-Ux t-1 )+UBx t-1 -1]+1
[0024] Among them, u t Represents the state value of the dynamic variable u at time t, represents the attenuation factor, x t Represents the input at time t, and the discrete time step Δt = 1ms is selected.
[0025] Furthermore, the process of training the dynamic synaptic spiking neuron network in step S4 includes:
[0026] S41: The spiking neural network receives a training data set;
[0027] S42: updating the dynamic variable u in the second dynamic synaptic neuron model at each moment according to a specified learning algorithm; wherein:
[0028] Furthermore, the S42 specifically includes:
[0029] S421: Dynamic variable u for the i-th synapse i The state at time t Use the state of the previous moment Its attenuation factor renew:
[0030]
[0031] S422: Synaptic weight w for the i-th synapse i Use dynamic variables at time t Updated to
[0032] S423: after the update in S421 Use the number of points U i Its dynamic load factor B is updated again:
[0033]
[0034] in represents the input of the i-th incoming neuron at time t, B i Value by U i The sign of the value determines:
[0035]
[0036] S424: Repeat the process of S721-S723 until the sample input and output process is completed;
[0037] S43: Use the time backpropagation algorithm to train the parameters θ of the model.
[0038] Furthermore, the step S2 optimizes the parameters of the first dynamic synaptic neuron model to construct a second dynamic synaptic neuron model; the parameter optimization includes: initializing u 0 =1; Initialize τ u ~N(0.1,0.01); initialize U~N(0.0,0.01); initialize B fac =4 and B dep =0.25.
[0039] Beneficial effects
[0040] 1. This invention achieves high performance and low energy consumption in application systems by integrating a dynamic synaptic neuron model into a spiking neuron network. The sparse activation of spiking neurons combined with the on-demand computational nature of dynamic synapses reduces the constant floating-point operations required in traditional ANNs, making it suitable for deployment on edge devices or neuromorphic chips. Furthermore, the discrete pulse communication and event-driven nature of dynamic synapses naturally complement the asynchronous circuit design of neuromorphic hardware, significantly reducing energy consumption.
[0041] 2. The present invention improves the noise robustness and stability of the application system by integrating a dynamic synaptic neuron model into the pulse neuron network: the short-term plasticity of the dynamic synapse can filter high-frequency noise (such as the synaptic inhibition mechanism), while the pulse coding itself is insensitive to noise; the dynamic adjustment of the synaptic weights can prevent the network from falling into a saturation state (such as over-activation or inactivation), thereby improving adaptability to input changes. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 This is a flow chart of a pulse neuron model based on dynamic synapses in the present invention;
[0043] Figure 2 It is a schematic diagram of the update process of the dynamic variable u;
[0044] Figure 3 2 is an experimental result diagram of an embodiment of the present invention. DETAILED DESCRIPTION
[0045] The following is combined with Figure 1 The present invention is described as follows:
[0046] The present invention provides a pulse neuron model based on dynamic synapses and its application, such as Figure 1 As shown: The system is based on a pulse neural network and includes:
[0047] S1: Construction of the first dynamic synaptic neuron model based on the short-term synaptic plasticity mechanism;
[0048] S2: Optimizing the parameters of the first dynamic synaptic neuron model to construct a second dynamic synaptic neuron model;
[0049] S3: Fusing the second dynamic synaptic neuron model with the spiking neuron network to construct a dynamic synaptic spiking neuron network; including:
[0050] Add a corresponding dynamic variable u to each static synapse i in the spiking neuron network i ;
[0051] By spiking the synaptic weights w in the neuronal network i With the dynamic variable u i The product w i ·u i Instead of the synaptic weight w in the spiking neuron network i , the membrane potential V(t) of the neuron in the spiking neuron network at time t in the updated dynamic synapse is expressed as:
[0052]
[0053] Where: w i represents the weight of synapse i, z t-1represents the output of the neuron at time t-1, θ represents the threshold, τ m It represents the time constant of membrane potential, Δt represents the time step, and Δt=1ms is set. represents the input of the i-th incoming neuron at time t.
[0054] S4: Training spiking neuron networks with dynamic synapses.
[0055] The application scenarios of this invention can be found in neuromorphic engineering: low-power real-time processing (such as dynamic visual sensor data); brain-computer interface: efficient decoding of the temporal characteristics of biological neural signals; robot control: online learning and control that adapts to dynamic environments; neuroscience modeling: studying the impact of synaptic plasticity on memory and learning.
[0056] The embodiment uses the present invention to implement a speech classification task, which mainly includes the following steps:
[0057] 101: Prepare the dataset and divide it into training set and test set;
[0058] 102: Based on the theory of short-term synaptic plasticity in neuroscience, a mathematical model of the dynamic variable u is constructed; 102 specifically includes:
[0059] The dynamic equation of the dynamic variable u is expressed by the following differential equation:
[0060]
[0061] τ u represents the synaptic dynamic time constant, δ represents the Dirac function, t sp represents the arrival time of the presynaptic pulse, and U represents the time at t sp The number of points added at time u, u - Indicates the value of u before the input pulse arrives, B indicates the dynamic load coefficient, only in {B fac ,B dep} determines whether the synaptic behavior is inhibitory or enhancing.
[0062] The dynamic equation of the discrete form of the dynamic variable u is expressed as follows: that is, the first dynamic synaptic neuron model:
[0063] u t =ρ u [u t-1 (1-Ux t-1 )+UBx t-1 -1]+1
[0064] represents the attenuation factor, x trepresents the input at time t; select a discrete time step Δt = 1ms; 103: initialize the parameters of the first dynamic synaptic neuron model in step 102; that is, initialize u 0 =1; Initialize τ u ~N(0.1,0.01); initialize U~N(0.0,0.01); initialize B fac =4 and B dep =0.25; construct the second dynamic synaptic neuron model by setting the above parameters;
[0065] 104: Installing the dynamic synaptic neuron model into a spiking neural network to obtain a dynamic synaptic spiking neural network; wherein:
[0066] The present invention adopts the Leaky Integrate-and-Fire (LIF) neuron model. In the LIF neuron model, the membrane potential of the neuron is reset every time an output pulse is generated. The membrane potential V(t) of the neuron at time t is obtained by integrating the input from all incoming neurons. get:
[0067]
[0068] Among them, w i represents the weight of synapse i, z t-1 represents the output of the neuron at time t-1, θ represents the threshold, τ m Represents the time constant of the membrane potential. Neurons continuously integrate input pulses in an event-driven manner. When there are no input pulses, the neuron's membrane potential gradually decays to a resting potential of 0. When the membrane potential exceeds the threshold, a pulse is emitted, and the membrane potential is reset by subtracting the threshold. The present invention uses 40 input neurons, 100 hidden neurons, and 11 output neurons to construct a single-layer feedforward spike neural network.
[0069] 105: training the dynamic synaptic spiking neural network;
[0070] The spiking neural network receives a training dataset;
[0071] like Figure 2 As shown, the dynamic variable u is updated at each moment according to the specified learning algorithm; where:
[0072] The dynamic variable u for the i-th synapse i The state at time t Use the state of the previous moment Its attenuation factor renew:
[0073]
[0074] The synaptic weight w for the i-th synapse i Use dynamic variables at time t Updated to
[0075] After the update in S721 Use the number of points U i Its dynamic load factor B is updated again:
[0076]
[0077] in: represents the input of the i-th incoming neuron at time t, B i Value by U i The sign of the value determines:
[0078]
[0079] Repeat the process of S721-S723 until the input and output process of the sample is completed.
[0080] The parameters θ of the model are trained using the Backpropagation Through Time (BPTT) algorithm.
[0081] 106: Use the test set to test the performance of the dynamic synaptic spiking neural network, such as Figure 3 As shown in (a), the algorithm of the present invention and other methods have different pulse neuron numbers (N hidden ) under the classification accuracy of the proposed algorithm; R-STP and F-STP represent the dynamic synaptic versions of the recurrent spiking neural network based on LIF neurons and the feedforward spiking neural network, respectively; R-ALIF and F-ALIF represent the recurrent spiking neural network based on ALIF neurons and the feedforward spiking neural network, respectively; R-LIF and F-LIF represent the recurrent spiking neural network based on LIF neurons and the feedforward spiking neural network, respectively. The results show that the proposed algorithm achieves the best performance among all networks.
[0082] (b) The algorithm of the present invention and other methods have different noise frequencies (p noise ) under the classification accuracy; the results show that the algorithm of the present invention achieves the best performance among all networks.
[0083] The above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications may be made based on the above description. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the claims of the present invention.
Claims
1. A pulse neuron network method based on dynamic synapses, characterized in that: The system is based on a spiking neural network and includes: S1: Construction of the first dynamic synaptic neuron model based on the short-term synaptic plasticity mechanism; S2: Optimizing the parameters of the first dynamic synaptic neuron model to construct a second dynamic synaptic neuron model; S3: Fusing the second dynamic synaptic neuron model with the spiking neuron network to construct a dynamic synaptic spiking neuron network; including: Add a corresponding dynamic variable u to each static synapse i in the spiking neuron network i ; By spiking the synaptic weights w in the neuronal network i With the dynamic variable u i The product w i ·u i Instead of the synaptic weight w in the spiking neuron network i , the membrane potential V(t) of the neuron in the spiking neuron network at time t in the updated dynamic synapse is expressed as: Where: w i represents the weight of synapse i, z t-1 represents the output of the neuron at time t-1, θ represents the threshold, τ m represents the time constant of the membrane potential, Δt represents the time step, represents the input of the i-th afferent neuron at time t; S4: Training spiking neuron networks with dynamic synapses.
2. A pulse neuron network method based on dynamic synapses according to claim 1, characterized in that: The process of constructing the first dynamic synaptic neuron model based on the synaptic plasticity method in step S1 includes: The dynamic synaptic neuron mathematical model is established according to the following formula: Where: τ u represents the synaptic dynamic time constant, δ represents the Dirac function, t sp represents the arrival time of the presynaptic pulse, u represents the dynamic variable, and U represents the time at t sp The number of points added at time u, u - Indicates the value of u before the input pulse arrives, B indicates the dynamic load coefficient, only in {B fac ,B dep } determines whether the synaptic behavior is inhibitory or potentiating; The first dynamic synaptic neuron model is obtained by performing discretization processing on the dynamic variable u in the dynamic synaptic neuron mathematical model according to the following formula: he t =ρ u [he t-1 (1-Ux t-1 )+UBx t-1 −1]+1 Among them, u t Represents the state value of the dynamic variable u at time t, represents the attenuation factor, x t Represents the input at time t, and the discrete time step Δt = 1ms is selected.
3. The method of a pulse neuron network based on dynamic synapses according to claim 1, characterized in that: The process of training the dynamic synaptic pulse neuron network in step S4 includes: S41: The spiking neural network receives a training data set; S42: updating the dynamic variable u in the second dynamic synaptic neuron model at each moment according to a specified learning algorithm; wherein: Furthermore, the S42 specifically includes: S421: Dynamic variable u for the i-th synapse i The state at time t Use the state of the previous moment Its attenuation factor renew: S422: Synaptic weight w for the i-th synapse i Use dynamic variables at time t Update to w i · S423: after the update in S421 Use the number of points U i Its dynamic load factor B is updated again: in represents the input of the i-th incoming neuron at time t, B i Value by U i The sign of the value determines: S424: Repeat the process of S721-S723 until the sample input and output process is completed; S43: Use the time backpropagation algorithm to train the parameters θ of the model.
4. The pulse neural network method based on a dynamic synaptic neuron model according to claim 1, characterized in that: The step S2 optimizes the parameters of the first dynamic synaptic neuron model to construct a second dynamic synaptic neuron model; the parameter optimization includes: initializing u 0 =1; Initialize τ u ~N(0.1,0.01); initialize U~N(0.0,0.01); initialize B fac =4 and B dep =0.
25.
5. An application system of a pulse neuron network based on dynamic synapses, characterized in that: The application system includes a software simulation part and a hardware implementation part to implement the method of any one of claims 1 to 4, wherein the software simulation part can perform corresponding simulation operations based on the pulse neuron model of the dynamic synapse, and the hardware implementation part can deploy the pulse neuron model of the dynamic synapse in a hardware circuit to realize the corresponding function.