A linear memory complexity pulse neural network online gradient learning method

Abstract

The application discloses a linear memory complexity pulse neural network online gradient learning method and belongs to the field of artificial intelligence and neural network technologies.The application decomposes a traditional two-dimensional eligibility trace matrix into an outer product form of two one-dimensional eligibility trace vectors of a pre-synapse and a post-synapse through three core mathematical derivations of diagonal approximation, Jacobian decomposition and rank-one approximation, respectively adopts an exponential smoothing mechanism to update the two one-dimensional vectors online, and completes parameter gradient calculation and weight updating when a learning signal arrives; while keeping the universality to multiple neuron and synapse dynamic models, the application reduces the eligibility trace memory complexity from O(H2) to O(H), greatly improves the memory and calculation efficiency, and can be widely applied to the fields of neuromorphic computing, brain simulation, intelligent robot control, edge computing device real-time learning, large-scale pulse neural network training and the like.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and neural network technology, specifically relating to an online gradient learning method for spiking neural networks with linear memory complexity. Background Technology

[0002] Spiking Neural Networks (SNNs) are neural network models that simulate the information processing mechanisms of biological nervous systems. Unlike traditional Artificial Neural Networks (ANNs), which use continuous activation values, SNNs transmit information through discrete spike events, offering advantages such as high biological rationality, good energy efficiency, and strong temporal processing capabilities. SNNs have broad application prospects in fields such as neuromorphic chips, brain simulation, and intelligent sensors.

[0003] Currently, the training methods for spiking neural networks mainly include the following categories: (1) Backpropagation in Time (BPTT): This method borrows the training ideas of traditional recurrent neural networks and combines surrogate gradient technology to handle the non-differentiability of impulse functions. BPTT can calculate accurate sequence-level gradients and is currently the most widely used method. However, BPTT requires storing the intermediate states of the entire time series, and its memory consumption is proportional to the sequence length T, i.e., the memory complexity is O(T). For training spiking neural networks at biological timescales, a one-second simulation may correspond to thousands of time steps, which leads to extremely large memory overhead for BPTT, severely limiting the length of trainable sequences and the network size.

[0004] (2) Real-time Recursive Learning (RTRL) and its variants: Real-time recursive learning computes gradients in a forward manner, avoiding the need to store complete time series. However, standard RTRL requires maintaining and updating the eligibility trace matrix, which has a dimension of H cubed, resulting in a memory complexity of up to O(H³) and a corresponding increase in computational complexity, making it difficult to scale RTRL in practical applications.

[0005] (3) Online learning methods such as e-prop and OTL: In recent years, researchers have proposed a variety of improved online learning methods, such as e-prop and OTL. These methods reduce the computational complexity of RTRL through various approximation strategies. However, these methods still require maintaining a qualification trace variable of size O(H²), where H is the number of neurons. When the network size increases, the quadratic memory growth makes these methods unsuitable for training large-scale spiking neural networks.

[0006] (4) Linear complexity methods for specific models: Some researchers have proposed linear memory complexity learning methods for simplified leaky integral firing (LIF) neuron models. However, these methods usually rely on specific neuron abstractions and cannot be generalized to more complex neuron dynamics (such as adaptive neurons, conduction synapses, short-term plasticity, etc.), which limits their use in brain simulation applications that require biofidelity.

[0007] In summary, existing online learning techniques for spiking neural networks have the following main problems: 1. High memory complexity: Existing online learning methods that can achieve near-BPTT accuracy (such as e-prop and OSTTL) require maintaining an qualification trace matrix of size O(H²), where H is the number of neurons. For large-scale networks containing tens of thousands or even hundreds of thousands of neurons, the memory requirement increases quadratically, far exceeding the memory capacity of a single GPU.

[0008] 2. Poor model generalization: Existing methods for achieving linear memory complexity are typically limited to simplified LIF neuron models and cannot support complex neuronal and synaptic dynamics. This contradicts the need for rich dynamic models in applications such as brain simulation.

[0009] 3. Lack of a unified theoretical framework: There is currently no method that can simultaneously achieve model independence (supporting multiple neuron and synapse models) and linear memory complexity, two goals that are considered contradictory in existing technologies.

[0010] 4. Limited scalability: Due to the above limitations, existing methods cannot effectively train whole-brain-scale spiking neural network models (such as the Drosophila whole-brain connectome constraint model containing more than 100,000 neurons), which restricts the development of computational neuroscience.

[0011] Therefore, there is an urgent need for an online gradient learning method for spiking neural networks that can achieve linear memory complexity while maintaining model generality. Summary of the Invention

[0012] The purpose of this invention is to provide an online gradient learning method for spiking neural networks with linear memory complexity, so as to solve the problems mentioned in the background art.

[0013] To achieve the above objectives, the present invention provides the following technical solution: an online gradient learning method for spiking neural networks with linear memory complexity, applied to spiking neural networks comprising an input layer, a recursive layer, and an output layer, characterized by comprising the following steps: S1 Initialization: Initialize the anterior synaptic qualification trace vector, the posterior synaptic qualification trace vector, and the initial hidden state of the network to zero vectors. At the same time, initialize the synaptic connection weight parameters, qualification trace exponential smoothing factor, and gradient descent learning rate of the network. S2 Time-series Forward Propagation: For each time step of the input sequence, perform the following operations sequentially: S21 calculates the synaptic current at the current moment, which is obtained by the initialized synaptic connection weight parameters and the synaptic input at the current moment through a preset neuron interaction operator; S22 Update Network Hidden State: Based on the hidden state of the previous moment and the synaptic current calculated at the current moment, the hidden state at the current moment is calculated through a preset neuron dynamics function; S23 Calculate the diagonal Jacobian matrix: Calculate the diagonal Jacobian matrix of the current hidden state versus the previous hidden state, and the diagonal Jacobian matrix of the current hidden state versus the synaptic current, respectively. S24 Online update of qualification traces: Using an exponential smoothing mechanism, based on the diagonal Jacobian matrix obtained in step S23, the presynaptic qualification trace vector and the postsynaptic qualification trace vector are updated online respectively. S3 Gradient Calculation and Parameter Update: When the learning target reaches the current time step, the instantaneous loss and learning signal at the current moment are calculated. The complete qualification trace is approximately obtained based on the outer product of the anterior synaptic qualification trace vector and the posterior synaptic qualification trace vector. Then, the gradient of the network parameters is calculated and the synaptic connection weights are updated. S4 repeats steps S2 to S3 until all time steps of the input sequence have been traversed, and outputs the final updated network synaptic connection weight parameters.

[0014] In a preferred embodiment, the neuron dynamics of the spiking neural network satisfy a discretized expression form:

[0015] in: Let represent the hidden states of H neurons, where each neuron has d state variables. I t ∈R H Indicates synaptic current. x t ∈R I Representing synaptic input (such as presynaptic pulse or synaptic conductance), θ∈R P Indicates synaptic connection weights. f For element-wise activation functions, describe the intrinsic neuron dynamics. It is a neuronal interaction operator that describes the conversion from synaptic input to synaptic current.

[0016] In a preferred embodiment, in step S23, the diagonal Jacobian matrix of the hidden state relative to the hidden state at the previous time step is calculated based on the sparse activation characteristics of the spiking neural network. The complete hidden state Jacobian matrix of a spiking neural network is decomposed into the sum of a diagonal matrix generated by intrinsic neuronal dynamics and a sparse matrix generated by recursive pulse connections. Under biologically reasonable sparse firing conditions, the cosine similarity between the diagonal matrix and the complete hidden state Jacobian matrix exceeds 99%. Therefore, the complete hidden state Jacobian matrix is ​​approximated as the diagonal Jacobian matrix.

[0017] In a preferred embodiment, in step S23, the diagonal Jacobian matrix of the hidden state with respect to the synaptic current satisfies the Kronecker decomposition form of the hidden state-weight Jacobian matrix:

[0018] Among them, D t f The diagonal Jacobian matrix for the hidden state with respect to synaptic currents has the following diagonal elements: , This represents the Kronecker product operator.

[0019] In a preferred embodiment, in step S3, the approximation of the complete qualification trace is achieved based on the sign consistency mathematical property of the input signal of the spiking neural network; the sign consistency mathematical property is: The sum of multiple rank-one matrices with identical element signs can be approximated as the outer product of a single rank-one matrix of weighted vector sums; Based on this characteristic, the complete eligibility trace is approximated as the outer product of the presynaptic eligibility trace vector and the postsynaptic eligibility trace vector, reducing the memory complexity of the eligibility trace from O(H0) to O(H0). 2 The value is reduced to O(H), where H is the number of neurons in the network.

[0020] In a preferred embodiment, in step S24, the update formula for the presynaptic eligibility trace vector is:

[0021] Where α is a preset smoothing factor, with a value range of 0 < α < 1, and ε t-1 x The presynaptic eligibility trace vector from the previous time step. x t This is the synaptic input vector at the current moment. ε t x is the presynaptic eligibility trace vector at the current moment.

[0022] In a preferred embodiment, in step S24, the update formula for the postsynaptic eligibility trace vector is:

[0023] Where α is a preset smoothing factor, and Dt Let diag(D) be the diagonal Jacobian matrix of the hidden state with respect to the hidden state at the previous time step. t ) is D t The diagonal element vector, ε t-1 f Let D be the postsynaptic eligibility trace vector from the previous time step. t f For the hidden state with respect to the synaptic current, diag(D) is the diagonal Jacobian matrix. t f ) is D t f The diagonal element vector, ε t f is the postsynaptic eligibility trace vector at the current moment.

[0024] In a preferred embodiment, the neuron dynamics function supports a neuron model that includes any one of the following: leaky integral firing (LIF) neuron, adaptive LIF neuron, and generalized integral firing (GIF) neuron; the neuron interaction operator supports a synapse model that includes any one of the following: Delta synapse, exponential synapse, conductance synapse, short-term inhibition (STD) synapse, and short-term plasticity (STP) synapse.

[0025] In a preferred embodiment, in step S3, the formula for calculating the network parameter gradient is:

[0026] Among them, ∂L t / ∂ h t Let be the learning signal at time t. For element-wise multiplication, ε t f This is the postsynaptic qualification trace vector. For the Kronecker product operator, ε t x This is the presynaptic qualification trace vector.

[0027] As a preferred embodiment, the method is applied to any one of the following tasks: neuromorphic dataset classification, whole-brain-scale spiking neural network training, real-time online learning on edge computing devices, intelligent robot control, and large-scale brain simulation.

[0028] Compared with the prior art, the beneficial effects of the present invention are: 1. Significantly improved memory efficiency: This invention reduces the memory complexity of qualification traces from O(H²) to O(H), where H is the number of neurons. This significantly reduces the memory usage of qualification traces and breaks through the bottleneck of memory growth with the quadratic increase of the number of neurons in existing technologies. This enables the training of ultra-large-scale spiking neural networks to be completed on a single conventional hardware device, solving the pain point that existing methods cannot adapt to large-scale network training. 2. Significantly improved computational efficiency: The qualification trace update of this invention requires only linear complexity operations, instead of quadratic complexity operations of existing methods, which significantly reduces the computational overhead of qualification trace update and greatly improves the network training speed; at the same time, this invention can update parameters without waiting for the complete sequence to end, and can maintain a stable computational advantage on various mainstream hardware platforms such as CPU, GPU, and TPU. 3. Strong model versatility: This invention supports a variety of neuronal and synaptic dynamics models, including but not limited to: Lack of integral firing (LIF) neurons, adaptive LIF (ALIF) neurons, generalized integral firing (GIF) neurons; Delta synapses, exponential synapses, conductance synapses; and complex synaptic dynamics such as short-term inhibition (STD) and short-term plasticity (STP). No customized modifications are required for specific neuronal models, resulting in strong adaptability. 4. Excellent learning performance: On standard neuromorphic benchmark tests and various temporal cognitive tasks, this invention can achieve learning accuracy comparable to the classic backpropagation method, while also possessing excellent long temporal dependency learning ability, which can effectively capture temporal correlation features in long sequences. 5. Supports online real-time learning: This invention adopts a forward gradient calculation method, which can update parameters in real time without waiting for the complete sequence to end, perfectly adapting to continuous learning application scenarios on edge computing devices; at the same time, the locality and temporality of the qualification trace are highly consistent with biological learning mechanisms, and have good biological rationality. Attached Figure Description

[0029] Figure 1 This is a schematic diagram illustrating the principle of the pp-prop algorithm. Figure 2 Here is the flowchart for the pp-prop algorithm; Figure 3 This is a comparison chart of memory consumption between the present invention and existing methods; Figure 4 This is a comparison chart of the calculation speed of the present invention and existing methods; Figure 5 This is a gradient approximation accuracy analysis diagram comparing the present invention with existing methods; Figure 6 This is a diagram of the architecture of a spiking neural network training system based on the pp-prop algorithm. Figure 7This is a comparison chart of the training results for the whole-brain model. Detailed Implementation

[0030] The present invention will be further described below with reference to embodiments.

[0031] The following embodiments are used to illustrate the present invention, but should not be used to limit the scope of protection of the present invention. The conditions in the embodiments can be further adjusted according to specific conditions, and simple improvements to the method of the present invention under the premise of the concept of the present invention are all within the scope of protection claimed by the present invention.

[0032] Please see Figure 1-7 This invention provides an online gradient learning method for spiking neural networks with linear memory complexity. The core of this method is the pp-prop online gradient learning algorithm, and the applicable spiking neural networks satisfy the discretized dynamic form defined in the disclosure document.

[0033] in: h t ∈R Hd This represents the hidden states of H neurons, with each neuron having d state variables; I t ∈R H Indicates synaptic current; x t ∈R I Indicates synaptic input (such as presynaptic pulse or synaptic conductance); θ∈R P Indicates synaptic connection weights; f For element-wise activation functions, describe the intrinsic neuron dynamics; It is a neuronal interaction operator that describes the conversion from synaptic input to synaptic current.

[0034] In the standard Real-Time Recursive Learning (RTRL) framework, when the learning objective is provided at time point t∈T, the weight gradient is calculated as follows:

[0035] Where L t The instantaneous loss at time t, For the learning signal, ε t The eligibility trace is calculated using the following recursive formula:

[0036] In Formula 3, the qualification trace ε tThe dimension is H×H×I, resulting in a memory complexity of O(H³), which is unsuitable for large-scale network training. This invention achieves online learning with linear memory complexity through the following steps: Step S1 Initialization Before inputting the input sequence, all variables and parameters are initialized, specifically including: 1. State variable initialization: Initialize the presynaptic eligibility trace vector ε x Postsynaptic qualification trace vector ε f All are initialized to zero vectors matching the input dimension and the number of neurons; the initial hidden state h0 of the network is initialized to a zero vector to provide an initial baseline for temporal forward propagation.

[0037] 2. Training Parameter Initialization: Initialize the network synaptic connection weight parameters θ, which can be achieved using conventional neural network weight initialization methods such as normal distribution, uniform distribution, or Xavier initialization; set the eligibility trace exponential smoothing factor α, with a value range of 0 < α < 1, corresponding to the time constant. Set the gradient descent learning rate η, and adapt the value according to the task scenario.

[0038] Step S2: Timing Forward Propagation For each time step t (t=1,2,...,T, where T is the total length of the input sequence), the forward computation, Jacobian matrix solution, and eligibility trace update operations are performed sequentially to realize the forward propagation of time series information and the online update of eligibility traces. The specific sub-steps are as follows: S21 calculates the sap current It at the current moment. Based on the initialized synaptic connection weight parameters θ and the current synaptic input. x t Through preset neuron interaction operators Complete the calculation to obtain the synaptic current at the current moment. This is completely consistent with the definition in Formula 1.

[0039] S22 Update Network Hidden Status h t Based on the hidden state of the previous moment h t-1 The synaptic current calculated in step S21 at the current moment I t The hidden state at the current time is calculated using a preset neuron dynamics function f. This is completely consistent with the definition in Formula 1, and completes the temporal iterative update of the neuron state.

[0040] S23 Calculate the diagonal Jacobian matrix This step corresponds to the core innovative principle of diagonal approximation and Jacobian decomposition of this invention, calculating the two core diagonal Jacobian matrices respectively, as follows: 1. Diagonal approximation for solving the diagonal Jacobian matrix D of the hidden state with respect to the previous time step. t The complete hidden state Jacobian matrix of a spiking neural network can be decomposed into two parts:

[0041] Where D t J is a diagonal matrix generated by the intrinsic neuronal dynamics f. t It is a square matrix generated by recursive pulse connections.

[0042] Based on the inherent sparse activation characteristic of spiking neural networks—in biologically rational cortical networks, the activation rate of neurons at any given time is typically less than 10%, J t The non-zero elements of J are generated only during pulse firing, therefore J t It is highly sparse in both time and space. Experimental verification shows that, under this condition, the diagonal matrix J... t Since the cosine similarity to the Jacobian matrix of the complete hidden state exceeds 99%, this invention adopts the following diagonal approximation:

[0043] Based on the above approximation, the diagonal Jacobian matrix D at the current time can be obtained. t .

[0044] 2. Kronecker decomposition to solve for the diagonal Jacobian matrix D of hidden states with respect to synaptic currents. t f Based on the physical constraints of synaptic connections in spiking neural networks, this invention proves that the hidden state-weight Jacobian matrix can be decomposed into the following Kronecker product form:

[0045] Where D t f It is a diagonal matrix with diagonal elements as follows: , Representing the Kronecker product, based on the above decomposition, the diagonal Jacobian matrix D of the hidden state with respect to the synaptic current at the current time is obtained. t f .

[0046] S24 qualification record online update This step corresponds to the core innovative principle of the rank-one approximation and eligibility trace decomposition of this invention. It employs an exponential smoothing mechanism and, based on the two diagonal Jacobian matrices obtained in step S23, updates the anterior synaptic eligibility trace vector and the posterior synaptic eligibility trace vector online, as detailed below: 1. The theoretical basis of the rank-one approximation is based on formulas 5 and 6. The RTRL (D-RTRL) qualification trace update formula for the diagonal approximation is:

[0047] At this point, the memory complexity is O(H). 2 This invention further utilizes the sign consistency mathematical property of the input signals of spiking neural networks to achieve dimensionality reduction optimization of linear memory complexity. The core theorem is as follows: Let a i ∈R m and b i ∈R n For vectors with uniform element signs (i.e., all elements are either positive or all are negative), the cosine similarity between the outer product of the sum of multiple rank-1 matrices and the weighted sum of the vectors approaches 1 as r→∞.

[0048] In spiking neural networks: for modeling methods based on postsynaptic alignment, x t Let x represent a binary pulse vector, where all elements are non-negative (0 or 1); for modeling based on presynaptic alignment, x t This represents synaptic conductance, which is always non-negative. Furthermore, for a monotonically active function f, its derivative D... t f The sign remains consistent at all times, fully satisfying the applicable conditions of the above theorem.

[0049] 2. Qualification trace decomposition and online update Based on the aforementioned mathematical properties, the pp-prop algorithm proposed in this invention approximates the qualification trace as the outer product of two vectors:

[0050] Among them, the presynaptic qualification trace ε t x and postsynaptic qualification trace ε t f Updated online using the following recursive formulas respectively:

[0051] Where α (0 < α < 1) is the smoothing factor, corresponding to the time constant τ = -Δt / ln(α). ε x ∈R I Tracking presynaptic activity, ε f ∈R H The memory complexity of both postsynaptic activities is O(H), achieving a linear dimensionality reduction in memory complexity.

[0052] Step S3: Gradient Calculation and Parameter Update When the learning goal y t Upon reaching the current time step, perform gradient calculation and weight update operations as follows: 1. Calculate the instantaneous loss: based on the hidden state h at the current moment. t Learning objectives y to be achieved t The instantaneous loss I is calculated using a preset loss function. t For classification tasks, the cross-entropy loss function can be used, while for regression tasks, the mean squared error loss function can be used. The appropriate loss function can be adapted according to the actual task type.

[0053] 2. Calculate the learning signal: Solve for the partial derivative of the instantaneous loss with respect to the hidden state at the current time step to obtain the learning signal. .

[0054] 3. Calculate the parameter gradient: based on the presynaptic eligibility trace vector ε updated in step S24. x Postsynaptic qualification trace vector ε f The parameter gradient is calculated by approximating the complete qualification trace using an outer product form:

[0055] Where ∘ represents element-wise multiplication, and t is the set of time steps to reach the learning target.

[0056] 4. Update network parameters: Using the gradient descent algorithm, based on the calculated parameter gradients and the preset learning rate η, update the synaptic connection weight parameters of the network. This completes a single learning iteration.

[0057] Step S4 is executed in a time-sequence loop. After completing all operations at the current time step, proceed to the next time step and repeat steps S2 to S3 until all time steps of the input sequence have been traversed. Finally, output the synaptic connection weight parameters θ of the trained network.

[0058] The complete pseudocode for this implementation is as follows: The complete pseudocode for the pp-prop algorithm is as follows: Algorithm: pp-prop online gradient learning Input: Input sequence {x} t Learning objectives {y} t}, smoothing factor α, learning rate η Output: Updated network parameters θ

[0059] 13. end if 14. end for Specific application example 1: Neuromorphological classification task based on LIF neurons

[0060] This embodiment applies the pp-prop algorithm described above to a neuromorphic dataset classification task to verify the classification accuracy and efficiency advantages of the present invention. The experimental data and formulas in the disclosure document are fully preserved. The specific implementation is as follows: 1. System Architecture This embodiment employs a two-layer recurrent spiking neural network architecture for classification tasks on neuromorphic datasets. The network structure includes: an input layer that receives pulse-coded sensor data; a recurrent layer containing leaky integral discharge (LIF) neurons with Delta synapses; and an output layer that uses a leak rate readout mechanism for classification.

[0061] 2. Neuron dynamics model The membrane potential dynamics of the LIF neurons used in this embodiment are as follows:

[0062] Where v t Z is the membrane potential, β is the membrane potential decay factor, and z is the membrane potential. t-1 For the pulse output of the previous moment, I t This is the input for synaptic current. When the membrane potential exceeds the threshold v... th Pulses are generated at that time:

[0063] Where H(·) is the Heaviside step function.

[0064] 3. Implementation of the pp-prop algorithm For the LIF neuron described above, the specific implementation of the pp-prop algorithm is as follows: Step 1 Initialization: Set the presynaptic eligibility trace ε x and postsynaptic qualification trace ε f Initialize to a zero vector.

[0065] Step 2 Forward Propagation: At each time step t, perform neuron dynamics update and calculate membrane potential v. t and pulse output z t .

[0066] Step 3: Calculate the Jacobian matrix: Calculate the diagonal matrix D t and D t f :

[0067] Step 4: Update eligibility traces: Update the presynaptic and postsynaptic eligibility traces according to Formulas 10 and 11.

[0068] Step 5: Calculate the gradient: When the learning target is reached, calculate the parameter gradient according to Formula 12 and update the weights.

[0069] 4. Key parameter settings

[0070] 5. Experimental verification results This embodiment fully retains the experimental data from the disclosure documents, and the results are as follows: Classification accuracy: On the SHD dataset, pp-prop achieves a test accuracy of 93.93±0.28%, comparable to BPTT's 94.01±0.12%; on the DVS Gesture dataset, it achieves an accuracy of 94.26±0.32%.

[0071] Memory efficiency: Reduces memory consumption by approximately 35 times compared to D-RTRL (0.5GB vs 17.5GB), and has a significant advantage over BPTT for long sequences.

[0072] Calculation speed: Approximately 10 times faster than D-RTRL, comparable to BPTT speed.

[0073] Specific Application Example 2: Whole-Brain Scale Spiking Neural Network Training This embodiment applies the pp-prop algorithm described above to the training of a whole-brain-scale spiking neural network constrained by anatomical connectomes, verifying the large-scale network scalability of the present invention, while fully preserving the formulas and experimental data from the disclosure documents. The specific implementation is as follows: 1. Application Background This embodiment demonstrates the application of the pp-prop algorithm in training a whole-brain-scale spiking neural network based on anatomical connectome constraints, with the goal of enabling the network to reproduce experimentally observed resting-state neural activity patterns.

[0074] 2. Network Architecture The whole-brain spiking neural network contains over 125,000 neurons and approximately 50 million synaptic connections, with the synaptic weight matrix constrained by anatomical connectome data. Neurons are modeled using the LIF model, and the dynamic equations are as follows:

[0075] 3. Training Strategies The training is divided into the following stages: (1) Data preprocessing: The calcium imaging signals recorded in the experiment were converted into discharge rate estimates by deconvolution.

[0076] (2) Input encoding: The brain region firing rate of the previous time step is converted into the external input of the current time step using a gated recurrent unit (GRU).

[0077] (3) Online training: The network receives the experimental discharge rate at time t as input and predicts the discharge rate at time t+1. The loss function is defined as the mean square error between the simulated and experimental discharge rates. The pp-prop algorithm is used to calculate the gradient online and update the weights.

[0078] (4) Readout conversion: The neuron-level firing rate is converted into the brain region-level firing rate by using a weighted average method based on synaptic projection weights.

[0079] 4. Experimental Results Memory consumption: pp-prop consumes only about 8.9GB of GPU memory, while D-RTRL and BPTT both exceed the single GPU (32GB) capacity limit.

[0080] Training results: After training, the model can accurately reproduce the resting-state activity patterns of 68 brain regions, and the brain region-level functional connectivity matrix is ​​highly consistent with the experimental data.

[0081] Generalization ability: The model can generate activity patterns consistent with the experiment on unseen test data.

[0082] The following points should be noted regarding the above plan: Figure 1 The core idea of ​​qualification trace decomposition is shown, including: the complete qualification trace matrix (left), the decomposed presynaptic qualification trace vector and postsynaptic qualification trace vector (middle), and the outer product reconstruction (right), with the dimensions and memory complexity of each part marked.

[0083] Figure 2 This demonstrates the complete execution flow of the pp-prop algorithm, including the sequence and data flow of steps such as initialization, forward propagation, Jacobian calculation, qualification trace update, gradient calculation, and parameter update.

[0084] Figure 3 The GPU memory consumption of the three methods, BPTT, D-RTRL and pp-prop, under different sequence lengths is compared in the form of a bar chart, which intuitively demonstrates the memory efficiency advantage of the present invention.

[0085] Figure 4 The training time per batch for the three methods under different sequence lengths is compared using a line graph to demonstrate the differences in computational efficiency.

[0086] Figure 5 This paper presents the statistical distribution of cosine similarity between gradients computed by D-RTRL and pp-prop and the exact gradients of BPTT across different neuron models and datasets.

[0087] Figure 6This paper demonstrates the overall architecture of a spiking neural network training system based on the pp-prop algorithm, including the relationship between the input encoding module, network simulation module, qualification trace calculation module, gradient calculation module, and parameter update module.

[0088] Figure 7 This study presents a comparison of brain region-level functional connectivity matrices before and after training of the whole-brain spiking neural network, along with a consistency analysis with experimental data.

[0089] Explanation of main symbols

[0090] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A linear memory complexity online gradient learning method for spiking neural networks, applied to spiking neural networks comprising an input layer, a recursive layer, and an output layer, characterized in that, Includes the following steps: S1 Initialization: Initialize the anterior synaptic qualification trace vector, the posterior synaptic qualification trace vector, and the initial hidden state of the network to zero vectors. At the same time, initialize the synaptic connection weight parameters, qualification trace exponential smoothing factor, and gradient descent learning rate of the network. S2 Time-series Forward Propagation: For each time step of the input sequence, perform the following operations sequentially: S21 calculates the synaptic current at the current moment, which is obtained by the initialized synaptic connection weight parameters and the synaptic input at the current moment through a preset neuron interaction operator; S22 Update Network Hidden State: Based on the hidden state of the previous moment and the synaptic current calculated at the current moment, the hidden state at the current moment is calculated through a preset neuron dynamics function; S23 Calculate the diagonal Jacobian matrix: Calculate the diagonal Jacobian matrix of the current hidden state versus the previous hidden state, and the diagonal Jacobian matrix of the current hidden state versus the synaptic current, respectively. S24 Online update of qualification traces: Using an exponential smoothing mechanism, based on the diagonal Jacobian matrix obtained in step S23, the presynaptic qualification trace vector and the postsynaptic qualification trace vector are updated online respectively. S3 Gradient Calculation and Parameter Update: When the learning target reaches the current time step, the instantaneous loss and learning signal at the current moment are calculated. The complete qualification trace is approximately obtained based on the outer product of the anterior synaptic qualification trace vector and the posterior synaptic qualification trace vector. Then, the gradient of the network parameters is calculated and the synaptic connection weights are updated. S4 repeats steps S2 to S3 until all time steps of the input sequence have been traversed, and outputs the final updated network synaptic connection weight parameters.

2. The linear memory complexity online gradient learning method for spiking neural networks according to claim 1, characterized in that: The neuron dynamics of the spiking neural network satisfy a discretized expression form: Let represent the hidden states of H neurons, where each neuron has d state variables. I t ∈R H Indicates synaptic current. x t ∈R I Representing synaptic input (such as presynaptic pulse or synaptic conductance), θ∈R P Indicates synaptic connection weights. f For element-wise activation functions, describe the intrinsic neuron dynamics. It is a neuronal interaction operator that describes the conversion from synaptic input to synaptic current.

3. The linear memory complexity online gradient learning method for spiking neural networks according to claim 1, characterized in that, In step S23, the diagonal Jacobian matrix of the hidden state relative to the hidden state at the previous time step is calculated based on the sparse activation characteristics of the spiking neural network. The complete hidden state Jacobian matrix of a spiking neural network is decomposed into the sum of a diagonal matrix generated by intrinsic neuronal dynamics and a sparse matrix generated by recursive pulse connections. Under biologically reasonable sparse firing conditions, the cosine similarity between the diagonal matrix and the complete hidden state Jacobian matrix exceeds 99%. Therefore, the complete hidden state Jacobian matrix is ​​approximated as the diagonal Jacobian matrix.

4. The linear memory complexity online gradient learning method for spiking neural networks according to claim 2, characterized in that, In step S23, the diagonal Jacobian matrix of the hidden state with respect to the synaptic current satisfies the Kronecker decomposition form of the hidden state-weight Jacobian matrix: Among them, D t f The diagonal Jacobian matrix for the hidden state with respect to synaptic currents has the following diagonal elements: , This represents the Kronecker product operator.

5. The linear memory complexity online gradient learning method for spiking neural networks according to claim 1, characterized in that, In step S3, the approximation of the complete qualification trace is achieved based on the mathematical property of the sign consistency of the input signal of the spiking neural network; the mathematical property of sign consistency is: The sum of multiple rank-one matrices with identical element signs can be approximated as the outer product of a single rank-one matrix of weighted vector sums; Based on this characteristic, the complete eligibility trace is approximated as the outer product of the presynaptic eligibility trace vector and the postsynaptic eligibility trace vector, reducing the memory complexity of the eligibility trace from O(H0) to O(H0). 2 The value is reduced to O(H), where H is the number of neurons in the network.

6. The online gradient learning method for spiking neural networks with linear memory complexity according to claim 1, characterized in that, In step S24, the update formula for the presynaptic eligibility trace vector is: Where α is a preset smoothing factor, with a value range of 0 < α < 1, and ε t-1 x The presynaptic eligibility trace vector from the previous time step. x t This is the synaptic input vector at the current moment. ε t x is the presynaptic eligibility trace vector at the current moment.

7. The linear memory complexity online gradient learning method for spiking neural networks according to claim 1, characterized in that, In step S24, the update formula for the postsynaptic eligibility trace vector is: Where α is a preset smoothing factor, and D t Let diag(D) be the diagonal Jacobian matrix of the hidden state with respect to the hidden state at the previous time step. t ) is D t The diagonal element vector, ε t-1 f Let D be the postsynaptic eligibility trace vector from the previous time step. t f For the hidden state with respect to the synaptic current, diag(D) is the diagonal Jacobian matrix. t f ) is D t f The diagonal element vector, ε t f is the postsynaptic eligibility trace vector at the current moment.

8. The linear memory complexity online gradient learning method for spiking neural networks according to claim 1, characterized in that, The neuron dynamics function supports neuron models including any one of leaky integral firing (LIF) neurons, adaptive LIF neurons, and generalized integral firing (GIF) neurons; the neuron interaction operator supports synaptic models including any one of Delta synapses, exponential synapses, conductance synapses, short-term inhibition (STD) synapses, and short-term plasticity (STP) synapses.

9. The linear memory complexity online gradient learning method for spiking neural networks according to claim 1, characterized in that, In step S3, the formula for calculating the gradient of the network parameters is: in, Let be the learning signal at time t. For element-wise multiplication, ε t f This is the postsynaptic qualification trace vector. For the Kronecker product operator, ε t x This is the presynaptic qualification trace vector.

10. The linear memory complexity online gradient learning method for spiking neural networks according to claim 1, characterized in that, The method can be applied to any one of the following tasks: neuromorphic dataset classification, whole-brain-scale spiking neural network training, real-time online learning on edge computing devices, intelligent robot control, and large-scale brain simulation.

Images