Connection group constraint training method for whole brain scale spiking neural network

Abstract

The application discloses a connection group constraint training method for a whole brain scale pulse neural network, belongs to the technical field of artificial intelligence and computational neuroscience, and comprises the following steps: S1, connection group constraint network construction; S2, adaptive pulse input coding; S3, forward propagation and eligibility trace synchronous updating; and S4, loss calculation and parameter closed-loop updating.The method is not dependent on connection group data of a specific species, can adapt to whole brain or local brain area connection group data of various model organisms such as nematodes, fruit flies and zebra fish, and supports the extension of various neuron and synapse dynamics models.Meanwhile, the application provides a complete end-to-end process from biological experimental data to pulse input, from neuron pulse output to brain area activity reading, and function connection verification, and no additional preprocessing framework adaptation is needed.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computational neuroscience, specifically relating to a connection group constraint training method for whole-brain-scale spiking neural networks. Background Technology

[0002] The connectome is a complete map of synaptic connections between neurons in the nervous system. In recent years, with the development of electron microscopy and automated image analysis techniques, researchers have successfully reconstructed complete or partial connectomes in several species, including nematodes, fruit fly larvae and adults, and zebrafish larvae. These connectome data provide an unprecedented foundation for understanding the structure-function relationships in the nervous system. Among them, the reconstruction of the whole-brain connectome of fruit flies is a significant milestone in the field of connectomics.

[0003] However, spiking neural network models built directly based on anatomical connectomes have significant functional limitations: anatomical connectomes only provide information on the structural connections between neurons, but the functional emergence of the nervous system also depends on a variety of factors such as synaptic weights and neuronal dynamic parameters.

[0004] Existing connectome-constrained networks suffer from a critical gap between anatomical structure and brain function, hindering the effective translation from structural maps to functional brain models. Anatomical connectomes only provide the topological structure of connections between neurons, while the emergence of brain function highly depends on refined parameters such as synaptic weights and neuronal dynamics. Networks directly constructed based on connectome topology cannot generate neural activity patterns consistent with biological experimental observations, nor can they reproduce the signal transmission and collaborative mechanisms between brain regions. Furthermore, the biological brain exhibits stable functional connectivity patterns between brain regions in a resting state. These patterns are the core manifestation of the brain's intrinsic dynamic characteristics and a crucial foundation for brain function research. Existing unoptimized connectome-constrained networks cannot spontaneously generate resting-state functional connections that match experimental observations, thus failing to meet the core requirements of brain function simulation. Summary of the Invention

[0005] The purpose of this invention is to provide a connection group constraint training method for whole-brain-scale spiking neural networks to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a connection group constraint training method for whole-brain-scale spiking neural networks, comprising the following steps: S1. Connection group constraint network construction: Obtain biological anatomical connection group data, fix the total number of neurons, synaptic connection topology and the relationship between neurons and brain regions of the spiking neural network based on the data, construct an anatomically constrained spiking neural network, and initialize the learnable parameters of the network. S2, Adaptive Pulse Input Encoding: For the spiking neural network constructed in S1, a gated recurrent unit is used to extract the temporal-dependent features of the experimentally observed brain region neural activity data, and the features are mapped to the instantaneous firing rate of each neuron in the network. An input pulse sequence adapted to the network is generated based on the Poisson process. S3. Forward Propagation and Eligibility Track Synchronization Update: The input pulse sequence generated in S2 is input into the spiking neural network constructed in S1, and the forward propagation calculation of the network is performed to obtain the pulse output of each neuron. During the synchronization process of forward propagation, the eligibility track used for gradient calculation is decomposed into presynaptic eligibility track and postsynaptic eligibility track, and the iterative update of the two eligibility tracks is completed respectively to obtain the presynaptic eligibility track and postsynaptic eligibility track at the corresponding time. S4. Loss Calculation and Parameter Closed-Loop Update: Based on the neuronal impulse output obtained in S3 and the neuron-brain region attribution relationship determined in S1, calculate the simulated firing rate of each brain region; calculate the loss function value based on the simulated firing rate and the experimentally observed brain region firing rate used in S2; calculate the gradient of the learnable parameters based on the presynaptic qualification traces and postsynaptic qualification traces obtained in S3; iteratively update the learnable parameters initialized in S1 according to the gradient to complete network training.

[0007] In a preferred embodiment, in step S1, the topology of the spiking neural network follows the following mandatory constraints: the total number of neurons in the network is completely consistent with the total number of neurons reconstructed in the connection group; the non-zero pattern of the synaptic connection matrix is ​​completely matched with the synaptic connection distribution of the connection group data; and the attribution relationship between neurons and brain regions is completely consistent with the anatomical partition information of the connection group. The learnable parameters include synaptic weights θ, where θ∈R S S is the total number of synapses in the connection group; optionally, the learnable parameters also include neuronal dynamic parameters, including membrane time constant, synaptic time constant, and impulse firing threshold.

[0008] As a preferred implementation, in step S3, the network forward propagation adopts a conductance-type leaky integral firing neuron model, and its state update rule is as follows:

[0009] in Let be the membrane potential of neuron i. For synaptic conductance, This is the resting potential. This is a synaptic reversal potential. The membrane time constant is The synaptic time constant, Let be the synaptic weights from neuron j to neuron i. For the pulse sequence of neuron j, when the membrane potential... Exceeding threshold V th At that time, the neuron fires a pulse and resets.

[0010] In a preferred embodiment, step S2 specifically includes the following sub-steps: S21. Temporal Feature Extraction: Gated recurrent units are used to process the brain region neural activity data observed in the experiment. The update rule is as follows: ,in Let be the observed discharge rate vector at time t. for The hidden state, for The hidden state of the time-gated loop unit; S22. Instantaneous firing rate mapping: The hidden state is mapped to the instantaneous firing rate of each neuron in the network through a linear transformation. The mapping formula is as follows:

[0011] in, Let be the instantaneous firing rate vector of each neuron at time t. These are the learnable encoder linear transform parameters; S23. Poisson Pulse Generation: The input pulse for the adaptive network is generated based on the instantaneous discharge rate and the simulation time step. The generation rule is as follows:

[0012] in, Let be the input pulse of neuron i at time t. Let be the instantaneous firing rate of neuron i at time t. This is the simulation time step.

[0013] In a preferred implementation, in step S3, the qualification trace decomposition and iterative update follow the following rules: the complete qualification trace used for gradient calculation is decomposed into two independent components, the presynaptic qualification trace and the postsynaptic qualification trace, and the complete qualification trace is approximated by the outer product of the two components; The presynaptic qualification trace is used to track historical information of synaptic input, while the postsynaptic qualification trace is used to track historical information of neuronal state. The presynaptic qualification trace is iteratively updated based on a preset smoothing factor, combining the presynaptic qualification trace from the previous moment with the network input pulse at the current moment; The postsynaptic eligibility trace is iteratively updated based on the same smoothing factor, combining the postsynaptic eligibility trace from the previous time step, the diagonal elements of the current neuron hidden state Jacobian matrix, and the diagonal elements of the current neuron hidden state-current Jacobian matrix.

[0014] As a preferred implementation, in step S4, the gradient of the learnable parameter is obtained by multiplying the partial derivative of the loss function with respect to the hidden state of the neuron at each time step with the outer product of the presynaptic qualification trace and the postsynaptic qualification trace at the corresponding time step, and then summing them over the entire simulation time step.

[0015] In a preferred embodiment, in step S4, the simulated firing rate of each brain region is obtained by weighting the pulse outputs of all neurons belonging to that brain region by a preset projection weight. The projection weight can be any one of uniform weight, neuron-based weight, or learnable weight. The uniform weight is set with the same weight value for all neurons, while the neuron-based weight is set according to the total strength of the neuron output synapse.

[0016] In a preferred implementation, in step S4, the loss function adopts online mean squared error loss, and the total loss value is the sum of the squares of the differences between the simulated discharge rate of each brain region at each time point and the corresponding experimentally observed brain region discharge rate; when a task-related training objective is introduced, a multi-task joint loss function is adopted, and the total loss value is the result of the weighted sum of the functional connectivity loss and the task-related loss according to the preset weight coefficients.

[0017] As a preferred implementation, after training, the functional connectivity matrix of brain regions is calculated based on the neuronal pulse sequence output by the spiking neural network. The correlation between this matrix and the experimentally observed functional connectivity matrix of brain regions is compared to verify the model's fitting effect on brain function.

[0018] In a preferred embodiment, in step S1, neurons and synaptic connections corresponding to the target functional brain region are extracted based on biological anatomical connectome data, and a sub-network topology of the target functional brain region is constructed; the target functional brain region is the Drosophila central complex region, which includes an ellipsoid, a sector, a prototubercula, and a tuberous brain region.

[0019] Compared with the prior art, the beneficial effects of the present invention are: This invention is the first to realize the training of a whole-brain connectome constrained spiking neural network containing 125,000 neurons and 50 million synapses based on a linear memory complexity algorithm, breaking through the memory bottleneck of existing methods.

[0020] This invention provides a complete conversion process from experimental calcium imaging data to pulse input and from pulse output to brain region activity, enabling end-to-end training optimization.

[0021] In this invention, the trained model can spontaneously emerge brain region functional connectivity patterns consistent with experimental observations, bridging the gap between anatomical structure and function.

[0022] In this invention, the pp-prop algorithm is biologically plausible, and the locality and temporality of the qualification trace are consistent with biological learning mechanisms.

[0023] The method of this invention does not rely on connectome data from specific species and can be adapted to connectome data from the whole brain or local brain regions of various model organisms such as nematodes, fruit flies, and zebrafish. It supports the expansion of various neuronal and synaptic dynamics models. At the same time, this invention provides a complete end-to-end process from biological experimental data to impulse input, from neuronal impulse output to brain region activity readout, and functional connectivity verification. No additional preprocessing framework adaptation is required, and it can be directly applied to multiple scenarios such as resting-state brain function simulation, specific cognitive function modeling, neurological disease mechanism research, and neuromorphic hardware design, with a wide range of applications. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the overall system architecture of the present invention; Figure 2 This is a schematic diagram of the connection group constraint network construction of the present invention; Figure 3 This is a structural diagram of the input encoder of the present invention; Figure 4 This is a flowchart of the pp-prop qualification trace update process of the present invention; Figure 5 This is a schematic diagram of brain region activity readout in this invention; Figure 6 This is a comparison chart of memory consumption for this invention; Figure 7 This is a comparison diagram of the functional connections of the present invention.

[0025] Figure 8 This is a schematic diagram illustrating the steps of the connection group constraint training method for whole-brain scale spiking neural networks according to the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to embodiments.

[0027] The following embodiments are used to illustrate the present invention, but should not be used to limit the scope of protection of the present invention. The conditions in the embodiments can be further adjusted according to specific conditions, and simple improvements to the method of the present invention under the premise of the concept of the present invention are all within the scope of protection claimed by the present invention.

[0028] Please see Figures 1-8 This invention provides a connection group constraint training method for whole-brain-scale spiking neural networks, comprising the following core components: Connectome-constrained network construction module: Constructs the topology of a spiking neural network based on anatomical connectome data, including the number of neurons, synaptic connection matrix, and brain region information.

[0029] Input encoding module: Converts experimentally observed neural activity data (such as calcium imaging signals) into Poisson pulse inputs acceptable to spiking neural networks.

[0030] Linear Memory Online Learning Module: Employs the pre-postsynaptic qualification trace propagation algorithm (pp-prop) to achieve gradient calculation with O(H) memory complexity.

[0031] Brain region activity readout module: Converts neuronal-level impulse activity into brain region-level firing rates and compares them with experimental data.

[0032] Functional connectivity assessment module: Calculates the brain region functional connectivity matrix generated by simulation and compares it with experimental observations for verification.

[0033] This invention constructs a spiking neural network based on anatomical connectome data, specifically including: Structural constraints: The network topology strictly follows the connection group data, including: The number of neurons H is equal to the total number of neurons reconstructed in the connectome; The non-zero sparsity pattern of the synaptic connection matrix is ​​determined by the connectoset; The attribution of neurons to brain regions is determined by anatomical partitioning information.

[0034] Learnable parameters: While keeping the connection topology unchanged, the following parameters can be optimized through training: Synaptic weight θ∈R S , where S is the total number of synapses in the connection group; Optional neuron dynamics parameters (such as time constant, threshold, etc.).

[0035] This invention employs a leaky integral firing (LIF) neuron model, combined with conductance-based synaptic dynamics: (Formula 1) (Formula 2) in Let be the membrane potential of neuron i. For synaptic conductance, This is the resting potential. This is a synaptic reversal potential. The membrane time constant is The synaptic time constant, Let be the synaptic weights from neuron j to neuron i. For the pulse sequence of neuron j, when the membrane potential... Exceeding threshold V th At that time, the neuron fires a pulse and resets.

[0036] When the membrane potential Vi exceeds the threshold Vth, the neuron fires a pulse and resets: This invention designs an encoder that converts experimental neural activity data into pulse inputs. The encoder comprises two sub-modules: Discharge rate estimation module: This module uses a gated recurrent unit (GRU) network to process experimental observation data (such as discharge rate estimation after deconvolution of calcium imaging signals) and capture temporal dependencies. in Let be the observed discharge rate vector at time t. for The hidden state.

[0037] Pulse generation module: Transforms the pulse into a linear form. The output is mapped to the instantaneous firing rate of each neuron. And generate pulse input based on the Poisson process:

[0038]

[0039] in and For learnable encoder parameters, This is the simulation time step.

[0040] Linear Memory Online Learning Algorithm This invention employs the pre- and post-synaptic eligibility trace propagation algorithm (pp-prop) to achieve training with linear memory complexity. The core idea of ​​this algorithm is to decompose the eligibility trace into the outer product of pre-synaptic and post-synaptic components:

[0041] Where: ε t x ∈ R I For presynaptic qualification traces, the history of synaptic input is tracked, ε t f ∈ R H Postsynaptic traces are used to track the history of neuronal states.

[0042] The update rules for the two eligibility criteria are as follows:

[0043]

[0044] in Let Dt be the smoothing factor, and Dt be the diagonal element of the hidden state Jacobian. For the hidden state-current Jacobian, the diagonal element is ∘, which represents element-wise multiplication.

[0045] The parameter gradient is calculated as follows:

[0046] This method reduces the memory complexity of the qualification trace from O(H) 2 The value is reduced to O(H), making it possible to train whole-brain-scale models.

[0047] This invention designs a brain region activity readout method based on synaptic projection weights, converting neuronal-level impulse activity into brain region-level firing rates. For brain region k, its estimated firing rate is:

[0048] in Let be the set of neurons belonging to brain region k, and pi be the projected weight of neuron i. Let be the pulse output of neuron i at time t.

[0049] This invention employs an online mean square error loss function to minimize the difference between the simulated discharge rate and the experimentally observed discharge rate.

[0050] in To simulate the generated brain region k-discharge rate, The brain region k discharge rate was observed in the experiment.

[0051] Example 1: Training of Drosophila Whole-Brain Connectome Constraint Networks

[0052] Connectome data: This embodiment uses FlyWire Drosophila whole-brain connectome data, with specific parameters as follows:

[0053] Training data: The training data comes from a resting-state calcium imaging experiment in fruit flies. Acquisition method: Two-photon calcium imaging; Brain regions covered: 68 neural thalamus; Time resolution: 10 Hz (100 ms / frame); Preprocessing: The OASIS algorithm is used to deconvolve the data to obtain the discharge rate estimate.

[0054] Network architecture and parameters

[0055] Training process pseudocode: Algorithm: Connection group-constrained whole-brain spiking neural network training Input: Connectome data C, calcium imaging sequence {rt,obs} Output: Optimized synaptic weights θ 1. Initialization: Construct the network topology from the connection group C and initialize the weights θ. 2. Initialization: ε x ←0,ε f ←0,h GRU ←0

[0056]

[0057] 20. End for Experimental verification results

[0058] Example 2: Functional simulation of specific brain regions This embodiment focuses on the functional simulation of the central complex region of the fruit fly, which involves cognitive functions such as navigation and spatial memory.

[0059] Subnetwork extraction: Extracting neurons and their connections related to the central complex from the whole-brain connectome: Number of neurons: approximately 5000; Brain regions included: ellipsoid (EB), sector (FB), prototuberous tract (PB), tubercle (NO); Input sources: visual input neurons, proprioceptive input neurons.

[0060] Task-related training: In addition to resting-state functional connectivity reconstruction, this embodiment introduces an additional task-related training objective: head-facing encoding task: train the network to generate the correct head-facing signal encoding given visual input; Path integration task: Train the network to integrate motion signals to update spatial location representations.

[0061] Multi-task joint loss function:

[0062] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A connection group constraint training method for whole-brain-scale spiking neural networks, characterized in that, Includes the following steps: S1. Connection group constraint network construction: Obtain biological anatomical connection group data, fix the total number of neurons, synaptic connection topology and the relationship between neurons and brain regions of the spiking neural network based on the data, construct an anatomically constrained spiking neural network, and initialize the learnable parameters of the network. S2, Adaptive Pulse Input Encoding: For the spiking neural network constructed in S1, a gated recurrent unit is used to extract the temporal-dependent features of the experimentally observed brain region neural activity data, and the features are mapped to the instantaneous firing rate of each neuron in the network. An input pulse sequence adapted to the network is generated based on the Poisson process. S3. Forward Propagation and Eligibility Trace Synchronous Update: Input the input pulse sequence generated in S2 into the spiking neural network constructed in S1, and perform forward propagation calculation to obtain the pulse output of each neuron; During the forward propagation synchronization process, the qualification trace used for gradient calculation is decomposed into presynaptic qualification trace and postsynaptic qualification trace, and the two qualification traces are iteratively updated to obtain the presynaptic qualification trace and postsynaptic qualification trace at the corresponding time. S4. Loss Calculation and Parameter Closed-Loop Update: Based on the neuronal impulse output obtained in S3 and the neuron-brain region attribution relationship determined in S1, calculate the simulated firing rate of each brain region; calculate the loss function value based on the simulated firing rate and the experimentally observed brain region firing rate used in S2; calculate the gradient of the learnable parameters based on the presynaptic qualification traces and postsynaptic qualification traces obtained in S3; iteratively update the learnable parameters initialized in S1 according to the gradient to complete network training.

2. The connection group constraint training method for whole-brain-scale spiking neural networks according to claim 1, characterized in that: In step S1, the topology of the spiking neural network follows the following mandatory constraints: the total number of neurons in the network is completely consistent with the total number of neurons reconstructed in the connection group; the non-zero pattern of the synaptic connection matrix is ​​completely matched with the synaptic connection distribution of the connection group data; and the attribution relationship between neurons and brain regions is completely consistent with the anatomical partition information of the connection group. The learnable parameters include synaptic weights θ, where θ ∈ R S S is the total number of synapses in the connection group; optionally, the learnable parameters further include neuron dynamics parameters, including membrane time constant, synaptic time constant, spike firing threshold.

3. The connection group constraint training method for whole-brain-scale spiking neural networks according to claim 1, characterized in that: In step S3, the network forward propagation adopts a conductance-based leaky integral firing neuron model, and its state update rule is as follows: ； in Let be the membrane potential of neuron i. For synaptic conductance, This is the resting potential. This is a synaptic reversal potential. The membrane time constant is The synaptic time constant, Let be the synaptic weights from neuron j to neuron i. For the pulse sequence of neuron j, when the membrane potential... Exceeding threshold V th At that time, the neuron fires a pulse and resets.

4. The connection group constraint training method for whole-brain-scale spiking neural networks according to claim 1, characterized in that: Step S2 specifically includes the following sub-steps: S21. Temporal Feature Extraction: Gated recurrent units are used to process the brain region neural activity data observed in the experiment. The update rule is as follows: ,in Let be the observed discharge rate vector at time t. for The hidden state, for The hidden state of the time-gated loop unit; S22. Instantaneous firing rate mapping: The hidden state is mapped to the instantaneous firing rate of each neuron in the network through a linear transformation. The mapping formula is as follows: ； in, Let be the instantaneous firing rate vector of each neuron at time t. These are the learnable encoder linear transform parameters; S23. Poisson Pulse Generation: The input pulse for the adaptive network is generated based on the instantaneous discharge rate and the simulation time step. The generation rule is as follows: ； in, Let be the input pulse of neuron i at time t. Let be the instantaneous firing rate of neuron i at time t. This is the simulation time step.

5. The connection group constraint training method for whole-brain-scale spiking neural networks according to claim 1, characterized in that: In step S3, the qualification trace decomposition and iterative update follow the following rules: the complete qualification trace used for gradient calculation is decomposed into two independent components, the presynaptic qualification trace and the postsynaptic qualification trace, and the complete qualification trace is approximated by the outer product of the two components. The presynaptic qualification trace is used to track historical information of synaptic input, while the postsynaptic qualification trace is used to track historical information of neuronal state. The presynaptic qualification trace is iteratively updated based on a preset smoothing factor, combining the presynaptic qualification trace from the previous moment with the network input pulse at the current moment; The postsynaptic eligibility trace is iteratively updated based on the same smoothing factor, combining the postsynaptic eligibility trace from the previous time step, the diagonal elements of the current neuron hidden state Jacobian matrix, and the diagonal elements of the current neuron hidden state-current Jacobian matrix.

6. The connection group constraint training method for whole-brain-scale spiking neural networks according to claim 1, characterized in that: In step S4, the gradient of the learnable parameter is obtained by multiplying the partial derivatives of the loss function with respect to the hidden state of the neuron at each time step with the outer product of the presynaptic qualification trace and the postsynaptic qualification trace at the corresponding time step, and then summing them over the entire simulation time step.

7. The connection group constraint training method for whole-brain-scale spiking neural networks according to claim 1, characterized in that: In step S4, the simulated firing rate of each brain region is obtained by weighting the pulse output of all neurons belonging to that brain region by a preset projection weight. The projection weight can be any one of uniform weight, neuron-based weight, or learnable weight. The uniform weight is set with the same weight value for all neurons, while the neuron-based weight is set according to the total strength of the neuron output synapse.

8. The connection group constraint training method for whole-brain scale spiking neural networks according to claim 1, characterized in that: In step S4, the loss function adopts online mean squared error loss, and the total loss value is the sum of the squares of the differences between the simulated firing rate of each brain region at each time and the corresponding experimentally observed firing rate of the brain region. When a task-related training objective is introduced, a multi-task joint loss function is adopted, and the total loss value is the result of the weighted sum of the functional connectivity loss and the task-related loss according to the preset weight coefficients.

9. The connection group constraint training method for whole-brain-scale spiking neural networks according to claim 1, characterized in that: After training, the functional connectivity matrix of brain regions is calculated based on the neuronal spiking sequences output by the spiking neural network. The correlation between this matrix and the experimentally observed functional connectivity matrix of brain regions is compared to verify the model's fitting effect on brain function.

10. The connection group constraint training method for whole-brain-scale spiking neural networks according to claim 1, characterized in that: In step S1, neurons and synaptic connections corresponding to the target functional brain region are extracted based on biological anatomical connectome data, and the sub-network topology of the target functional brain region is constructed; the target functional brain region is the Drosophila central complex region, which includes an ellipsoid, a sector, a prototubercula, and a tuberous brain region.

Images