Network model training method and system based on multi-synaptic connected spiking neurons

Abstract

The present application discloses a network model training method and system based on multi-synaptic connected spiking neurons. The method obtains a training data set, the training data includes any one of image data, audio data and EEG signal data; constructs a multi-synaptic connected spiking neuron for simultaneously encoding the intensity information and time information of the input signal, the input signal is an image, audio or EEG signal input into the multi-synaptic connected spiking neuron; constructs a network model based on multi-synaptic connected spiking neurons; constructs a target gradient substitution function for multi-synaptic connected spiking neurons; based on the target gradient substitution function and the training data set, trains the network model based on multi-synaptic connected spiking neurons by back propagation. The present application can simultaneously encode the intensity information and time information of the stimulus, and improve the classification accuracy of the network model.

Description

Technical Field

[0001] The present application relates to the field of computational neuroscience technology, and in particular to a network model training method and system based on multi-synaptic connected spiking neurons. Background Art

[0002] Spiking neural networks, inspired by brain function and structure, are considered the third generation of artificial neural networks, and are known for their low power consumption and spatiotemporal dynamics of biological neural networks. In recent years, many studies have applied spiking neural networks to tasks such as image classification, speech recognition, and event stream processing, and have continuously tried to improve the scale and performance of SNNs on these tasks, but they still cannot reach or surpass artificial neural networks.

[0003] Current spiking neural networks are often built based on leaky integrate-and-fire (LIF) neurons, which use rate coding and time coding to encode information. Rate coding can encode the intensity information of the stimulus (i.e., input signal), but loses the time information, and is often used to process static data, such as images. Time coding can encode the time information of the stimulus, but loses the intensity information of the stimulus, and is often used to process time series data. Therefore, current spiking neurons cannot encode both the intensity information and the time information of the stimulus at the same time, resulting in low classification accuracy of current spiking neurons. Summary of the invention

[0004] The present application aims to propose a network model training method and system based on multi-synaptic connected spiking neurons, which can simultaneously encode the intensity information and time information of the stimulus, thereby improving the classification accuracy of the network model.

[0005] In a first aspect, an embodiment of the present application provides a network model training method based on multi-synaptic connected spiking neurons, the method comprising:

[0006] Acquire a training data set, wherein the training data includes any one of image data, audio data, and electroencephalogram signal data;

[0007] Constructing a multi-synaptic connected spiking neuron for simultaneously encoding intensity information and time information of an input signal, wherein the input signal is an image, audio, or electroencephalogram signal input into the multi-synaptic connected spiking neuron;

[0008] Constructing a network model of spiking neurons based on the multi-synaptic connections;

[0009] constructing a target gradient substitution function for the multi-synaptic connected spiking neuron;

[0010] Based on the target gradient substitute function and the training data set, the network model of the spiking neurons based on the multi-synaptic connections is trained by back propagation.

[0011] Compared with the prior art, the first aspect of the present application has the following beneficial effects:

[0012] This method constructs a multi-synaptic connected spiking neuron for simultaneously encoding the intensity information and time information of the input signal, and the input signal is an image, audio or EEG signal input into the multi-synaptic connected spiking neuron; constructs a network model based on the multi-synaptic connected spiking neuron; constructs a target gradient substitution function for the multi-synaptic connected spiking neuron; and trains the network model based on the multi-synaptic connected spiking neuron by back propagation based on the target gradient substitution function and the training data set. In this way, a network model is constructed based on the constructed multi-synaptic connected spiking neuron, so that the network model can simultaneously encode the intensity information and time information of the input signal, and by constructing a good target gradient substitution function, the gradient vanishing and explosion problems that often occur during the network model training process are solved. Therefore, the multi-synaptic connected spiking neurons can greatly improve the performance of the network model while ensuring low power consumption, and improve the classification accuracy of the network model.

[0013] In some embodiments, constructing a multi-synaptic connected spiking neuron for simultaneously encoding intensity information and time information of an input signal comprises:

[0014] ;

[0015] in, Indicates Moment The input pulse or input signal of the layer, Indicates The weight of the layer, Indicates Moment The output pulse of the layer, represents the maximum number of synapses, Indicates Moment The membrane potential of the layer, Indicates Moment The hidden state of the layer, Indicates Moment The output pulse of the layer, represents a step function, Indicates The threshold of membrane potential corresponding to each synapse is Indicates Moment The hidden state of the layer, represents the time constant, represents the Hadamard product, represents the symbolic function, Indicates Moment Layer 1 of spiking neurons The output pulse of a synapse.

[0016] In some embodiments, the constructing a network model of spiking neurons based on the multi-synaptic connections comprises:

[0017] Based on the multi-synaptic connected spiking neurons, convolutional layers with different convolutional kernels, and batch normalization, constructing a spiking convolutional layer of a first convolutional kernel and a spiking convolutional layer of a second convolutional kernel;

[0018] Based on the pulse convolution layer of the first convolution kernel, construct a coding module;

[0019] Based on the impulse convolution layer of the second convolution kernel, construct a residual block;

[0020] Construct a feature extraction module containing multiple residual blocks;

[0021] Construct an output module consisting of an average pooling layer and a fully connected layer;

[0022] The encoding module, the feature extraction module and the output module are connected to obtain a network model of spiking neurons based on the multi-synaptic connection.

[0023] In some embodiments, constructing a feature extraction module including a plurality of residual blocks includes:

[0024] Connecting multiple pulse convolution layers of the first convolution kernels to form a multi-scale vital sign extraction submodule;

[0025] Constructing a pulse convolution layer including the second convolution kernel, the multi-scale vital sign extraction submodule and a residual block of the attention submodule;

[0026] A plurality of the residual blocks are connected to form a feature extraction module.

[0027] In some embodiments, when the network model based on the multi-synaptic connected spiking neuron is trained by back propagation based on the target gradient substitute function and the training data set, the method further includes a process in which the multi-synaptic connected spiking neuron processes an input signal:

[0028] In the receiving phase, the input signal is received through at least one synapse in the multi-synaptic connected spiking neuron, or a pulse train output from an upper layer neuron is received;

[0029] In the accumulation phase, the neuron is caused to sum the voltage according to the membrane potential at the previous moment and the received input signal or pulse sequence;

[0030] During the activation phase, when the membrane potential of the neuron exceeds a preset threshold, the neuron triggers pulse firing.

[0031] In some embodiments, the target gradient substitute function of constructing the multi-synaptic connected spiking neuron model comprises:

[0032] ;

[0033] in, The target gradient surrogate function representing the spiking neuron model with multiple synaptic connections, Indicates Moment The membrane potential of the layer, represents the threshold interval, represents the maximum number of synapses, Indicates synapses, represents the initial threshold of membrane potential, represents the gradient surrogate function.

[0034] In some embodiments, after constructing the target gradient surrogate function of the multi-synaptic connected spiking neuron, the method further comprises:

[0035] Get a preset gradient replacement function that includes a scale factor;

[0036] According to the target gradient substitution function, converting the preset gradient substitution function into a first gradient substitution function;

[0037] Calculating the first gradient substitution function to obtain a first result value;

[0038] When the first result value is equal to 1, determining an optimal threshold interval in the first gradient substitution function;

[0039] After determining the maximum number of synapses and the optimal threshold interval, the oscillation amplitudes of the membrane potential at different scale factors are calculated, and the scale factor corresponding to the minimum oscillation amplitude is selected as the scale factor in the first gradient substitution function.

[0040] In a second aspect, the embodiment of the present application further provides a network model training system based on multi-synaptic connected spiking neurons, the system comprising:

[0041] A data acquisition unit, used to acquire a training data set, wherein the training data includes any one of image data, audio data and EEG signal data;

[0042] A first construction unit is used to construct a multi-synaptic connected spiking neuron for simultaneously encoding intensity information and time information of an input signal, wherein the input signal is an image, audio or EEG signal input into the multi-synaptic connected spiking neuron;

[0043] A second construction unit is used to construct a network model of spiking neurons based on the multi-synaptic connections;

[0044] A third construction unit is used to construct a target gradient substitution function of the multi-synaptic connected spiking neuron;

[0045] A model training unit is used to train the network model of the spiking neurons based on the multi-synaptic connections by back propagation based on the target gradient substitution function and the training data set.

[0046] In a third aspect, an embodiment of the present application also provides an electronic device, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a network model training method based on multi-synaptic connected pulse neurons as described above.

[0047] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a network model training method based on multi-synaptic connected pulse neurons as described above.

[0048] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0050] Figure 1 It is a flow chart of an embodiment of a network model training method based on multi-synaptic connected spiking neurons provided by the present application;

[0051] Figure 2It is a schematic diagram of the structure of a spiking neuron with multi-synaptic connections in the best embodiment of the network model training method based on spiking neurons with multi-synaptic connections provided by the present application;

[0052] Figure 3 It is a schematic diagram of experimental results of gradient substitution functions of different scale factors in the best embodiment of the network model training method based on multi-synaptic connected spiking neurons provided by the present application;

[0053] Figure 4 is the maximum number of synapses in the best embodiment of the network model training method based on multi-synaptic connected spiking neurons provided by the present application , scale parameter Schematic diagram of the experimental results of the Sigmoid type gradient substitution function when ;

[0054] Figure 5 is the maximum number of synapses in the best embodiment of the network model training method based on multi-synaptic connected spiking neurons provided by the present application , scale parameter Schematic diagram of the experimental results of the Sigmoid type gradient substitution function when ;

[0055] Figure 6 It is a schematic diagram of experimental results of multiple gradient substitution functions in the best embodiment of the network model training method based on multi-synaptic connected spiking neurons provided by the present application;

[0056] Figure 7 It is a schematic diagram of the structure of a network model of spiking neurons based on multi-synaptic connections in the best embodiment of the network model training method of spiking neurons based on multi-synaptic connections provided by the present application;

[0057] Figure 8 It is a structural schematic diagram of an embodiment of a network model training system based on multi-synaptic connected spiking neurons provided by the present application;

[0058] Fig. 9 It is a schematic diagram of the structure of an embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0060] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0061] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0062] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.

[0063] Spiking neural networks, inspired by brain function and structure, are considered the third generation of artificial neural networks, and are known for their low power consumption and spatiotemporal dynamics of biological neural networks. In recent years, many studies have applied spiking neural networks to tasks such as image classification, speech recognition, and event stream processing, and have continuously tried to improve the scale and performance of SNNs on these tasks, but they still cannot reach or surpass artificial neural networks.

[0064] Current spiking neural networks are often built based on leaky integrate-and-fire (LIF) neurons, which use rate coding and time coding to encode information. Rate coding can encode the intensity information of the stimulus (i.e., input signal), but loses the time information, and is often used to process static data, such as images. Time coding can encode the time information of the stimulus, but loses the intensity information of the stimulus, and is often used to process time series data. Therefore, current spiking neurons cannot encode both the intensity information and the time information of the stimulus at the same time, resulting in low classification accuracy of current spiking neurons.

[0065] In order to solve the above-mentioned problem that current spiking neurons cannot simultaneously encode stimulus intensity information and time information, resulting in low classification accuracy of current spiking neurons, the present application proposes a network model training method and system based on spiking neurons with multi-synaptic connections.

[0066] Reference Figure 1 The present application embodiment provides a network model training method based on multi-synaptic connected spiking neurons, the method comprising the following steps:

[0067] Step S100, obtaining a training data set, where the training data includes any one of image data, audio data and EEG signal data;

[0068] Step S200, constructing a multi-synaptic connected spiking neuron for simultaneously encoding intensity information and time information of an input signal, wherein the input signal is an image, audio, or EEG signal input into the multi-synaptic connected spiking neuron;

[0069] Step S300, constructing a network model of spiking neurons based on multi-synaptic connections;

[0070] Step S400, constructing a target gradient substitution function for a spiking neuron with multiple synapses;

[0071] Step S500: Based on the target gradient substitution function and the training data set, a network model of spiking neurons based on multi-synaptic connections is trained by back propagation.

[0072] In this embodiment, a multi-synaptic connected spiking neuron is constructed for simultaneously encoding the intensity information and time information of the input signal, and the input signal is an image, audio or EEG signal input into the multi-synaptic connected spiking neuron; a network model based on the multi-synaptic connected spiking neuron is constructed; a target gradient substitution function of the multi-synaptic connected spiking neuron is constructed; based on the target gradient substitution function and the training data set, the network model based on the multi-synaptic connected spiking neuron is trained by back propagation. In this way, a network model is constructed based on the constructed multi-synaptic connected spiking neuron, so that the network model can simultaneously encode the intensity information and time information of the input signal, and by constructing a good target gradient substitution function, the gradient vanishing and explosion problems that often occur during the network model training process are solved. Therefore, the multi-synaptic connected spiking neurons can greatly improve the performance of the network model while ensuring low power consumption, and improve the classification accuracy of the network model.

[0073] The above-mentioned network model of spiking neurons based on multi-synaptic connections can be achieved by applying spiking neurons with multi-synaptic connections constructed to simultaneously encode the intensity information and time information of the input signal to a large-scale deep spiking neural network model. The large-scale deep spiking neural network model can be a spiking neural network model existing in the prior art, except that the neurons use spiking neurons with multi-synaptic connections.

[0074] The target gradient substitution function for constructing the spiking neuron with multi-synaptic connection can be a target gradient substitution function constructed based on the spiking neuron with multi-synaptic connection. It can also be a gradient substitution function in the prior art, such as a Rectangular gradient substitution function, a Sigmoid gradient substitution function, or an Arctangent gradient substitution function.

[0075] In some embodiments, constructing a multi-synaptic connected spiking neuron for simultaneously encoding intensity information and time information of an input signal comprises:

[0076] ;

[0077] in, Indicates Moment The input pulse or input signal of the layer, Indicates The weight of the layer, Indicates Moment The output pulse of the layer, represents the maximum number of synapses, Indicates Moment The membrane potential of the layer, Indicates Moment The hidden state of the layer, Indicates Moment The output pulse of the layer, represents a step function, Indicates The threshold of membrane potential corresponding to each synapse is Indicates Moment The hidden state of the layer, represents the time constant, represents the Hadamard product, represents the symbolic function, Indicates Moment Layer 1 of spiking neurons The output pulse of a synapse.

[0078] In this embodiment, when the input signal at a moment is large, the membrane potential will rise to a higher potential value, trigger multiple pulses through multiple threshold decisions, and then be transmitted to the postsynaptic neuron simultaneously through the multi-synaptic structure. The number of pulses emitted at each time step can be used to measure the size of the instantaneous input signal at the current moment, that is, it encodes the strength of the input signal. The input signals at different moments are encoded in the pulse outputs at different moments, and the output at the current moment is also affected by the historical moments (i.e. moment), thereby encoding the time information of the input signal.

[0079] In some embodiments, constructing a network model of spiking neurons based on multi-synaptic connections comprises:

[0080] Based on multi-synaptic connected spiking neurons, convolutional layers with different convolutional kernels, and batch normalization, a spiking convolutional layer of the first convolutional kernel and a spiking convolutional layer of the second convolutional kernel are constructed;

[0081] Based on the impulse convolution layer of the first convolution kernel, a coding module is constructed;

[0082] Based on the impulse convolution layer of the second convolution kernel, a residual block is constructed;

[0083] Construct a feature extraction module containing multiple residual blocks;

[0084] Construct an output module consisting of an average pooling layer and a fully connected layer;

[0085] The encoding module, the feature extraction module and the output module are connected to obtain a network model of spiking neurons based on multi-synaptic connections.

[0086] In this embodiment, based on spiking neurons with multi-synaptic connections, convolution layers with different convolution kernels and batch normalization, a spiking convolution layer of a first convolution kernel and a spiking convolution layer of a second convolution kernel are constructed, and both the encoding module and the residual block are constructed based on the spiking convolution layer. Therefore, the constructed network model based on spiking neurons with multi-synaptic connections can simultaneously encode the intensity information and time information of the input signal, so that the network model can better extract the spatiotemporal features in the input signal, thereby improving the classification accuracy of the network model.

[0087] The impulse convolution layer of the first convolution kernel and the impulse convolution layer of the second convolution kernel are two impulse convolution layers with different convolution kernel sizes. This embodiment does not impose any specific limitation on the size of the convolution kernel.

[0088] In some implementations, constructing a feature extraction module including a plurality of residual blocks includes:

[0089] Connecting multiple pulse convolution layers of the first convolution kernels to form a multi-scale vital sign extraction submodule;

[0090] Construct a residual block containing a spike convolution layer with a second convolution kernel, a multi-scale vital sign extraction submodule, and an attention submodule;

[0091] Connect multiple residual blocks to form a feature extraction module.

[0092] In this embodiment, the residual block includes both the pulse convolution layer of the first convolution kernel and the pulse convolution layer of the second convolution kernel, and then multiple residual blocks are connected to form a feature extraction module, which can better extract the spatiotemporal features in the input signal, thereby improving the classification accuracy of the network model.

[0093] In some embodiments, when a network model based on multi-synaptic spiking neurons is trained by back propagation based on a target gradient substitute function and a training data set, the method further includes a process in which the multi-synaptic spiking neurons process input signals:

[0094] In the receiving phase, at least one synapse in the multi-synaptic connected spiking neurons receives an input signal, or receives a spike train output from an upper layer neuron;

[0095] In the accumulation phase, the neuron sums the voltage according to the membrane potential at the previous moment and the input signal or pulse sequence received;

[0096] During the activation phase, when the neuron's membrane potential exceeds a preset threshold, the neuron triggers a pulse firing.

[0097] In this embodiment, during the receiving phase, at least one synapse in the multi-synaptic connected pulse neuron receives an input signal, or receives a pulse train output from an upper-layer neuron; during the accumulation phase, the neuron sums the voltage according to the membrane potential at the previous moment and the received input signal or pulse train; during the activation phase, when the membrane potential of the neuron exceeds a preset threshold, the neuron triggers the pulse emission. In this way, the multi-synaptic structure allows the neuron to excite multiple pulses at the same time. This mechanism not only effectively encodes the intensity of the input stimulus, but also takes into account the timing of the input signal. By adjusting the number and interval of thresholds, it is possible to flexibly adapt to data sets of different complexities and improve the computational efficiency of the network model constructed later.

[0098] In some embodiments, constructing a target gradient surrogate function for a multi-synaptic spiking neuron model includes:

[0099] ;

[0100] in, The target gradient surrogate function representing the spiking neuron model with multiple synaptic connections, Indicates Moment The membrane potential of the layer, represents the threshold interval, represents the maximum number of synapses, Indicates synapses, represents the initial threshold of membrane potential, represents the gradient surrogate function.

[0101] In some embodiments, after constructing the target gradient surrogate function for the multi-synaptically connected spiking neuron, the method further comprises:

[0102] Get a preset gradient replacement function that includes a scale factor;

[0103] According to the target gradient substitution function, converting the preset gradient substitution function into a first gradient substitution function;

[0104] Calculating a first gradient substitution function to obtain a first result value;

[0105] When the first result value is equal to 1, determining an optimal threshold interval in the first gradient substitution function;

[0106] After determining the maximum number of synapses and the optimal threshold interval, the oscillation amplitude of the membrane potential at different scale factors is calculated, and the scale factor corresponding to the minimum oscillation amplitude is selected as the scale factor in the first gradient substitution function.

[0107] In this embodiment, the first result value is obtained by calculating the first gradient substitution function; when the first result value is equal to 1, the optimal threshold interval in the first gradient substitution function is determined; after determining the maximum number of synapses and the optimal threshold interval, the oscillation amplitude of the membrane potential of different scale factors is calculated, and the scale factor corresponding to the minimum oscillation amplitude is selected as the scale factor in the first gradient substitution function. In this way, by determining the scale factor corresponding to the optimal threshold interval and the minimum oscillation amplitude, the classification accuracy can be improved while solving the gradient vanishing and exploding problems that often occur in the network model training process.

[0108] To facilitate understanding by those skilled in the art, a set of best embodiments is provided below:

[0109] Spiking neural networks, inspired by brain function and structure, are considered the third generation of artificial neural networks, and are known for their low power consumption and spatiotemporal dynamics of biological neural networks. In recent years, many studies have applied spiking neural networks to tasks such as image classification, speech recognition, and event stream processing, and have continuously tried to improve the scale and performance of SNNs on these tasks, but they still cannot reach or surpass artificial neural networks.

[0110] Current spiking neural networks are often built based on leaky integrate-and-fire (LIF) neurons, which use rate coding and time coding to encode information. Rate coding can encode the intensity information of the stimulus, but loses the time information, and is often used to process static data such as images. Time coding can encode the time information of the stimulus, but loses the intensity information of the stimulus, and is often used to process time series data. Therefore, the current spiking neuron model is caught in a dilemma and cannot encode both the intensity and time information of the stimulus at the same time.

[0111] Based on the above problems, this embodiment designs a pulse neuron that can efficiently encode the intensity information and time information of the stimulus. The pulse neuron is a method that can effectively solve the above problems. And the method of this embodiment also needs to consider how to apply and generalize to large-scale deep pulse neural networks to solve the gradient vanishing and explosion problems that often occur during the training process of deep pulse neural networks. And further consider how to deploy it on brain-like computing edge devices to solve the problems of low computational efficiency and low accuracy of the current pulse neural network model on edge devices. This embodiment provides the following Figure 2 The multi-synaptic connected spiking neuron model (i.e., multi-synaptic connected spiking neuron) shown in the figure designs the gradient replacement function and parameter selection of the multi-synaptic connected spiking neuron, the network model structure and training method, the experimental effect and the related products of the neural network model. The technical solution of this embodiment specifically includes the following contents:

[0112] 1. Neuron model.

[0113] The neuron model in this embodiment adopts a multi-synaptic spiking neuron model, which is an extension of the traditional single-synaptic LIF (Leaky Integrate-and-Fire) neuron model. Each neuron can have multiple synapses, and each synapse corresponds to a firing threshold in the cell body. The multi-synaptic spiking neuron model can effectively encode the intensity and time information of the input signal.

[0114] The mathematical representation of the spiking neuron model with multi-synaptic connections is:

[0115] ;

[0116] in, Indicates Moment The input pulse or input signal of the layer, Indicates The weight of the layer, Indicates Moment The output pulse of the layer, Indicates the maximum number of synapses, which can be set according to specific tasks. Indicates Moment The membrane potential of the layer, Indicates Moment The hidden state of the layer, Indicates Moment The output pulse of the layer, represents a step function, Indicates The threshold of membrane potential corresponding to each synapse is Indicates Moment The hidden state of the layer, represents the time constant, represents the Hadamard product, Represents a symbolic function.

[0117] The signal processing process of the multi-synaptic spike neuron model is as follows:

[0118] ① Reception phase: During this phase, neurons transmit One or more of the synapses receive preprocessed input pulses or input signals, or pulse trains output from upper-layer neurons. The membrane potential of the neuron will be updated based on the received pulses. Input pulses can come from external stimuli (such as images, audio signals) or the outputs of other neurons in the network. When the input external stimulus is directly an image, audio signal or EEG signal, these input signals need to be encoded into pulses first. If the input is a pulse signal, no encoding is required.

[0119] ② Accumulation period: During the accumulation period, the neuron sums the voltage based on the membrane potential at the previous moment and the pulse sequence received. Specifically, the neuron membrane potential .

[0120] ③Activation phase: When the neuron membrane potential When the accumulation exceeds a certain excitation threshold, the neuron will trigger a pulse. Neurons set multiple thresholds (in represents the number of synapses), each threshold corresponds to a different excitation state. The excitation process of neurons follows the following rules:

[0121] When the membrane potential Reaching a certain threshold within a single simulation time step and does not exceed a higher threshold, the neuron will fire pulses. Specifically, assuming that the thresholds are , the neuron will determine whether the membrane potential exceeds these thresholds in turn. For example, when the membrane potential Greater than and less than When the neuron fires a pulse, if the membrane potential Greater than and less than When , the neuron will fire 2 pulses, and so on. If the pulse excitation requirements are met, the pulse is allowed to be fired, and no response is given in other cases.

[0122] In the time interval between adjacent pulse inputs, if there is no excitation, the membrane voltage will change according to the time constant Attenuation.

[0123] The multi-synaptic structure allows neurons to fire multiple pulses at the same time. This mechanism not only effectively encodes the intensity of the input stimulus (i.e., the input signal), but also takes into account the timing of the input signal. By adjusting the number and interval of thresholds, it can flexibly adapt to data sets of different complexities, improve the computational efficiency of the multi-synaptic spike neuron model, and thus improve the computational efficiency of the network model built based on the multi-synaptic spike neurons.

[0124] The number of thresholds (ie, the number of synapses) and the distribution of thresholds (ie, the intervals between thresholds) both fall within the protection scope of this embodiment.

[0125] The method for encoding the stimulus intensity and stimulus time of the multi-synaptic spike neuron model is as follows:

[0126] ① When When the input stimulus is large, the membrane potential The potential rises to a higher value, triggers multiple pulses through multiple threshold decisions, and then is transmitted to the postsynaptic neuron simultaneously through the multi-synaptic structure. The number of pulses fired at each time step can be used to measure the size of the instantaneous input stimulus at the current moment, that is, it encodes the intensity of the input stimulus.

[0127] ② Input stimuli at different times are encoded in pulse outputs at different times In the process, the output at the current moment is also affected by the historical moments, thus encoding the stimulus time information.

[0128] 2. Gradient substitution function and parameter selection.

[0129] The Heaviside step function is usually used to describe the neuron firing process. , whose derivative is the impulse function, or Dirac A function is a function that is equal to zero at all points except zero, and its integral over the entire domain is equal to 1:

[0130] ;

[0131] However, directly using the impulse function for gradient descent will obviously make the network training extremely unstable. In order to solve this problem, various surrogate gradient methods have been proposed. The principle is to use the impulse function in the forward propagation. , while in back propagation we use , rather than ,in is the gradient substitute function. Commonly used gradient substitute functions include Sigmoid, Linear, Rectangular and Arctangent functions. Since the integral of the impulse function on the real axis is equal to 1, the gradient substitute function The integral on the real axis must also be equal to 1, that is:

[0132] ;

[0133] In this embodiment, the gradient replacement function of the impulse function is assumed to be ,in is the scale factor, which controls For example, the Rectangular gradient replacement function ; Sigmoid type gradient substitution function ; Arctangent type gradient substitution function It is worth noting that if a single parameter No control The scale transformation can be extended to multi-scale parameters.

[0134] Figure 3 Shows the Rectangular type ( ), Sigmoid type ( ) and Arctangent type ( )Graph of the gradient substitution function.

[0135] Brief Notes for , in this embodiment, the alternative gradient function of the multi-synaptic spiking neuron is:

[0136] ;

[0137] in, is the replacement gradient for a multisynaptic spiking neuron, is the membrane potential, for The threshold of a synapse.

[0138] In order to measure the stimulus intensity in equal proportion, the interval between multiple thresholds is set to a fixed interval. .set up As the initial threshold, the alternative gradient of the multi-synaptic spike neuron can be transformed into:

[0139] ;

[0140] The expression is simplified by translating it so that the sum is symmetric about zero. ,have to:

[0141] ;

[0142] when When , the sum can be approximated by a continuous integral. The integral on the approximation of discrete The sum of , that is:

[0143] ;

[0144] make , substituting it into the integral, we get:

[0145] ;

[0146] To prevent the vanishing or exploding gradient problem in deep spiking neural networks, it is necessary to ensure that the replacement gradients of multi-synaptic spiking neurons Fluctuates around 1, that is Therefore, the threshold interval is the optimal parameter selection. Maximum number of synapses You can choose according to the task and actual situation.

[0147] To more intuitively demonstrate the above theoretical results, taking the Sigmoid gradient substitute function as an example, we first need to convert the Sigmoid gradient substitute function into the form of a substitute gradient function for a multi-synaptic spike neuron, and then conduct experiments. The experimental results show that when the maximum number of synapses , scale parameter When , we can see that the image of the alternative gradient function is Nearby vibrations, such as Figure 4 In fact, adjusting the scale parameter It does not change the substitution gradient function in The fact that the vibrates nearby, such as Figure 5 As shown ( ). The form of converting the Sigmoid type gradient substitute function into the substitute gradient function of the multi-synaptic spike neuron specifically includes:

[0148] First select the Sigmoid type gradient replacement function:

[0149] ;

[0150] Then Replace related functions with , substitute into the following formula:

[0151] ;

[0152] get:

[0153] ;

[0154] Right now:

[0155] ;

[0156] Then, when the maximum number of synapses OK, threshold interval By choosing an appropriate scale parameter , the gradient surrogate function can be further optimized.

[0157] like Figure 6 The first three graphs shown have the following conclusions:

[0158] Reference Figure 6 (a) is a schematic diagram of the experimental results of the Rectangular gradient substitution function. First, the Rectangular gradient substitution function needs to be converted into the form of the substitution gradient function of the multi-synaptic spike neuron before the experiment is carried out. It can be seen from the schematic diagram of the experimental results that when When the Rectangular gradient substitution function is When , its substitution gradient is stable at 1, compared with and is a better parameter choice.

[0159] Reference Figure 6 (b) is a schematic diagram of the experimental results of the Arctangent gradient substitution function. First, the Arctangent gradient substitution function needs to be converted into the form of the substitution gradient function of the multi-synaptic spike neuron before the experiment is carried out. It can be seen from the schematic diagram of the experimental results that when When , the Arctangent type gradient substitution function is When , the maximum value of the replacement gradient is less than 1, which will cause the gradient to disappear during network training. When , its substitution gradient oscillates too violently at 1. When , its substitution gradient oscillates slightly at 1. Therefore, Compared to and Better parameter selection.

[0160] Reference Figure 6 (c) is a schematic diagram of the experimental results of the Sigmoid gradient substitution function. First, the Sigmoid gradient substitution function needs to be converted into the form of the substitution gradient function of the multi-synaptic spike neuron before the experiment is carried out. It can be seen from the schematic diagram of the experimental results that when When , the Sigmoid type gradient substitute function is When , its substitution gradient oscillates greatly at 1. When , the maximum value of the replacement gradient is less than 1, which will cause the gradient to disappear during network training. When , its substitution gradient oscillates slightly at 1. Therefore, Compared to and Better parameter selection.

[0161] The experimental results are consistent with theoretical judgments and are task-oriented. In this embodiment, the classification task on the CIFAR-10 dataset is taken as an example. Figure 6 (d) is a schematic diagram of the classification accuracy of multiple gradient surrogate functions on the CIFAR-10 dataset. It can be seen from the figure that the Rectangular gradient surrogate function The highest classification accuracy is achieved when ; the Arctangent gradient substitution function is The highest classification accuracy is achieved when ; the Sigmoid gradient substitution function is The highest classification accuracy was achieved.

[0162] This embodiment designs a corresponding gradient substitution function based on a multi-synaptic spiking neuron model to obtain the optimal threshold interval , and at the maximum number of synapses If determined, by adjusting The scale parameter in , which can further optimize the gradient surrogate function.

[0163] 3. Network model structure and training method.

[0164] In terms of the neural network model structure, this embodiment adopts a multi-layer pulse neural network design. Figure 7 (a) is a schematic diagram of the principle of the network model of spiking neurons based on multi-synaptic connections. The network model structure of spiking neurons based on multi-synaptic connections consists of a coding module, a feature extraction module and an output module. The coding module can receive the input of a pulse signal or a continuous signal such as an image, audio, etc. For the input of a continuous signal, the coding module is used to convert the continuous input signal into a pulse sequence. The coding module is often composed of a layer of pulse convolution layer or a layer of pulse fully connected layer (i.e., the pulse convolution layer of the first convolution kernel), and the specific composition of the coding module can be set according to the task requirements. The feature extraction module is often composed of multiple layers of pulse convolution layers (including the pulse convolution layer of the first convolution kernel and the pulse convolution layer of the second convolution kernel) or fully connected layers, in which all neurons use a multi-synaptic connected spiking neuron model to extract the spatiotemporal features in the signal. The output module performs the final classification or regression task according to the pulse emission sequence output by the feature extraction module.

[0165] Specifically, as a specific implementation of this embodiment, a network model of spiking neurons based on multi-synaptic connections for image classification tasks in this embodiment has an encoding module that is a spiking convolution layer with a convolution kernel of 3x3 (i.e., a spiking convolution layer of the first convolution kernel). Figure 7 (b) is a schematic diagram of the structure of the feature extraction module, which is a stack of multiple residual blocks. Each residual block consists of two pulse convolution layers with a convolution kernel of 1x1 (i.e., the pulse convolution layer of the second convolution kernel), a multi-scale feature extraction module, and an attention submodule. The multi-scale feature extraction module is composed of a stack of multiple pulse convolution layers with a convolution kernel of 3x3. The attention submodule uses a series of channel attention and spatial attention. The output module is an average pooling layer and a fully connected layer.

[0166] The network model training process of the spiking neuron based on multi-synaptic connection adopts the backpropagation through time (BPTT) algorithm for training, and the connection weights, biases and other parameters of the network model of the spiking neuron based on multi-synaptic connection are fed back according to the size of the loss value to minimize the value of the loss function. This embodiment does not limit the specific form of the backpropagation through time algorithm.

[0167] In this embodiment, by establishing a spiking neuron model with multi-synaptic connections, the network model based on the spiking neurons with multi-synaptic connections can simultaneously and efficiently encode the intensity information and time information of the input stimulus (i.e., the input signal), and by optimizing the alternative gradient function of the spiking neurons with multi-synaptic connections, the gradient vanishing and explosion problems that often occur in the training process of deep spiking neural networks are solved, so that the spiking neural network constructed based on the spiking neurons with multi-synaptic connections can be expanded to a large-scale deep spiking neural network, greatly improving the performance of the spiking neural network while ensuring low power consumption.

[0168] It should be noted that this embodiment can also convert the continuously input signal into audio or EEG signals, etc. The training and classification method used is the same as the above-mentioned image classification method, which is not described in detail in this embodiment.

[0169] For better explanation, the following experiments were performed in this embodiment:

[0170] 1. Experimental results.

[0171] In order to verify the effectiveness of the technical solution of this embodiment, experiments were conducted on the static image classification CIFAR-10 dataset and the dynamic human action recognition HARDVS dataset, and the specific network model structure of the feature extraction module adopted ResNet-29. The experimental results show that after adopting the multi-synaptic connection spiking neuron model, the network model based on multi-synaptic connection spiking neurons can more efficiently encode the spatiotemporal information of the input signal, and greatly improve the classification accuracy of the model while maintaining low power consumption.

[0172] Table 1 shows the performance of different models on the CIFAR-10 dataset. The network model based on multi-synaptic connected spiking neurons proposed in this embodiment has a classification accuracy of 96.41% in 1 time step, which is about 0.71% higher than the LIF model, and the power consumption is only increased by 0.11mJ. In 4 time steps, the accuracy reaches 97%.

[0173] Table 1 Classification performance and power consumption of different models on the CIFAR-10 dataset

[0174]

[0175] Similarly, in addition to showing superior performance on static tasks, in order to further verify that it can also perform better on dynamic tasks, it was trained and tested on the dynamic human action recognition HARDVS dataset. Table 2 shows the performance of different models on the HARDVS dataset. At the same time step, the network model based on multi-synaptic connected spiking neurons achieved a classification accuracy of 51.3%, which is about 3.8% higher than the LIF model, and the power consumption only increased by 0.2mJ.

[0176] Table 2 Classification performance and power consumption of different models on the HARDVS dataset

[0177]

[0178] 2. Products related to neuron models.

[0179] In addition to the aforementioned neural network model (i.e., a network model of spiking neurons based on multi-synaptic connections) and training methods, this embodiment also discloses the following products related to neural networks. For the sake of brevity, the aforementioned neural network model and training methods are not described in detail. The following products all cite any one or more of the aforementioned neural network models and their training methods, and use them as part of the product. Specifically including:

[0180] Training device: includes a memory and at least one processor coupled to the memory, configured to execute any of the above-mentioned neural network training methods. The training device can be a common computer, a server, a high-performance computing device dedicated to machine learning (such as a computing device including a GPU), a high-performance computer, or an FPGA and ASIC device, etc.

[0181] Storage device: A device configured to store source code of a neural network training method written in a programming language, or machine code that can be directly run on a machine. The storage device includes, but is not limited to, memory carriers such as RAM, ROM, magnetic disks, solid-state drives, and optical disks, and may be part of the training device or remotely separated from it.

[0182] Neural network accelerator: A hardware device used to accelerate the computation of a neural network model, deployed with configuration parameters trained using any of the above neural network training methods. The accelerator may be configured as a coprocessor on the CPU side to perform specific tasks, such as image recognition.

[0183] Brain-like chips: chips that deploy configuration parameters trained by any of the above neural network models and training methods, chips developed to simulate the working mode of biological neurons, usually based on event triggering, with low power consumption and low latency response characteristics. Existing brain-like chips include Intel's Loihi, IBM's TrueNorth and Synsense's Dynap-CNN.

[0184] Neural network configuration parameter deployment method: deploy the configuration parameters trained by any of the neural network training methods included in the above items to the neural network accelerator. With the help of dedicated deployment software, the deployment phase transmits the configuration data generated in the training phase (which can be directly stored in the training device or stored in the dedicated deployment device) to the storage unit (such as the storage unit simulating synapses) of the neural network accelerator (such as an artificial intelligence chip or a mixed-signal brain-like chip) through a channel (such as a cable or various types of networks) to complete the configuration parameter deployment process of the neural network accelerator.

[0185] Neural network configuration parameter deployment device: a device that stores the configuration parameters trained by any of the above neural network training methods and transmits the parameters to the neural network accelerator through a channel.

[0186] Those skilled in the art know that, in addition to implementing the neuron model and training method provided by the present invention in a purely computer-readable program code, the neuron model provided by the present invention can be completely implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and memristors through logic programming to achieve the same function. Therefore, the neuron model provided by the present application can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0187] Reference Figure 8 The embodiment of the present application also provides a network model training system based on multi-synaptic connected spiking neurons, the system comprising a data acquisition unit 100, a first construction unit 200, a second construction unit 300, a third construction unit 400 and a model training unit 500, wherein:

[0188] The data acquisition unit 100 is used to acquire a training data set, where the training data includes any one of image data, audio data and EEG signal data;

[0189] A first construction unit 200 is used to construct a multi-synaptic connected spiking neuron for simultaneously encoding intensity information and time information of an input signal, wherein the input signal is an image, audio or EEG signal input into the multi-synaptic connected spiking neuron;

[0190] A second construction unit 300 is used to construct a network model of spiking neurons based on multi-synaptic connections;

[0191] A third construction unit 400 is used to construct a target gradient substitution function of a spiking neuron with multiple synaptic connections;

[0192] The model training unit 500 is used to train a network model of spiking neurons based on multi-synaptic connections by back propagation based on a target gradient substitution function and a training data set.

[0193] It should be noted that since the network model training system based on spiking neurons with multi-synaptic connections in this embodiment and the above-mentioned network model training method based on spiking neurons with multi-synaptic connections are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to the system embodiment and will not be described in detail here.

[0194] Reference Fig. 9 The present application also provides an electronic device, which includes:

[0195] at least one memory;

[0196] at least one processor;

[0197] at least one program;

[0198] The program is stored in the memory, and the processor executes at least one program to implement the network model training method based on multi-synaptic connected pulse neurons implemented in the present disclosure.

[0199] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0200] The electronic device according to the embodiment of the present application is described in detail below.

[0201] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present disclosure;

[0202] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1700, and the processor 1600 calls and executes the network model training method of the spiking neuron based on multi-synaptic connection in the embodiment of the present disclosure.

[0203] Input / output interface 1800, used to implement information input and output;

[0204] Communication interface 1900, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0205] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0206] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0207] An embodiment of the present disclosure also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned network model training method based on multi-synaptic connected pulse neurons.

[0208] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0209] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0210] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present disclosure and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0211] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0212] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0213] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0214] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0215] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0216] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0217] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0218] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs. The above is a detailed description of the embodiments of the present application in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Within the scope of knowledge possessed by ordinary technicians in the relevant technical field, various changes can be made without departing from the purpose of the present application.

[0219] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A network model training method based on multi-synaptic connected spiking neurons, characterized in that: The method comprises: Acquire a training data set, wherein the training data includes any one of image data, audio data, and electroencephalogram signal data; Constructing a multi-synaptic connected spiking neuron for simultaneously encoding intensity information and time information of an input signal, wherein the input signal is an image, audio, or electroencephalogram signal input into the multi-synaptic connected spiking neuron; Constructing a network model of spiking neurons based on the multi-synaptic connections; constructing a target gradient substitution function for the multi-synaptic connected spiking neuron; Get a preset gradient replacement function that includes a scale factor; According to the target gradient substitution function, converting the preset gradient substitution function into a first gradient substitution function; Calculating the first gradient substitution function to obtain a first result value; When the first result value is equal to 1, determining an optimal threshold interval in the first gradient substitution function; After determining the maximum number of synapses and the optimal threshold interval, calculating the oscillation amplitude of the membrane potential at different scale factors, and selecting the scale factor corresponding to the minimum oscillation amplitude as the scale factor in the first gradient substitution function; Based on the scale factor, the first gradient substitute function and the training data set, the network model of the spiking neuron based on the multi-synaptic connection is trained by back propagation.

2. The network model training method based on multi-synaptic connected spiking neurons according to claim 1, characterized in that: The multi-synaptic connected spiking neuron constructed to simultaneously encode the intensity information and time information of the input signal comprises: ； in, Indicates Moment The input pulse or input signal of the layer, Indicates The weight of the layer, Indicates Moment The output pulse of the layer, represents the maximum number of synapses, Indicates Moment The membrane potential of the layer, Indicates Moment The hidden state of the layer, Indicates Moment The output pulse of the layer, represents a step function, Indicates The threshold of membrane potential corresponding to each synapse is Indicates Moment The hidden state of the layer, represents the time constant, represents the Hadamard product, represents the symbolic function, Indicates Moment Layer 1 of spiking neurons The output pulse of a synapse.

3. The network model training method based on multi-synaptic connected spiking neurons according to claim 1, characterized in that: The constructing of a network model of spiking neurons based on the multi-synaptic connections comprises: Based on the multi-synaptic connected spiking neurons, convolutional layers with different convolutional kernels, and batch normalization, constructing a spiking convolutional layer of a first convolutional kernel and a spiking convolutional layer of a second convolutional kernel; Based on the pulse convolution layer of the first convolution kernel, construct a coding module; Based on the impulse convolution layer of the second convolution kernel, construct a residual block; Construct a feature extraction module containing multiple residual blocks; Construct an output module consisting of an average pooling layer and a fully connected layer; The encoding module, the feature extraction module and the output module are connected to obtain a network model of spiking neurons based on the multi-synaptic connection.

4. The network model training method based on multi-synaptic connected spiking neurons according to claim 3 is characterized in that: The construction includes a feature extraction module containing multiple residual blocks, including: Connecting multiple pulse convolution layers of the first convolution kernels to form a multi-scale vital sign extraction submodule; Constructing a pulse convolution layer including the second convolution kernel, the multi-scale vital sign extraction submodule and a residual block of the attention submodule; A plurality of the residual blocks are connected to form a feature extraction module.

5. The network model training method based on multi-synaptic connected spiking neurons according to claim 1, characterized in that: When the network model based on the multi-synaptic connected spiking neurons is trained by back propagation based on the target gradient substitute function and the training data set, the method further includes a process in which the multi-synaptic connected spiking neurons process input signals: In the receiving phase, the input signal is received through at least one synapse in the multi-synaptic connected spiking neuron, or a pulse train output from an upper layer neuron is received; In the accumulation phase, the neuron is caused to sum the voltage according to the membrane potential at the previous moment and the received input signal or pulse sequence; During the activation phase, when the membrane potential of the neuron exceeds a preset threshold, the neuron triggers pulse firing.

6. The network model training method based on multi-synaptic connected spiking neurons according to claim 1, characterized in that: The target gradient substitution function of constructing the multi-synaptic connected spiking neuron model comprises: ； in, The target gradient surrogate function representing the spiking neuron model with multi-synaptic connections, Indicates Moment The membrane potential of the layer, represents the threshold interval, represents the maximum number of synapses, Indicates synapses, represents the initial threshold of membrane potential, represents the gradient surrogate function.

7. A network model training system based on multi-synaptic connected spiking neurons, characterized in that: The system comprises: A data acquisition unit, used to acquire a training data set, wherein the training data includes any one of image data, audio data and EEG signal data; A first construction unit is used to construct a multi-synaptic connected spiking neuron for simultaneously encoding intensity information and time information of an input signal, wherein the input signal is an image, audio or EEG signal input into the multi-synaptic connected spiking neuron; A second construction unit is used to construct a network model of spiking neurons based on the multi-synaptic connections; A third construction unit is used to construct a target gradient substitution function of the multi-synaptic connected spiking neuron; Get a preset gradient replacement function that includes a scale factor; According to the target gradient substitution function, converting the preset gradient substitution function into a first gradient substitution function; Calculating the first gradient substitution function to obtain a first result value; When the first result value is equal to 1, determining an optimal threshold interval in the first gradient substitution function; After determining the maximum number of synapses and the optimal threshold interval, calculating the oscillation amplitude of the membrane potential at different scale factors, and selecting the scale factor corresponding to the minimum oscillation amplitude as the scale factor in the first gradient substitution function; A model training unit is used to train the network model of the spiking neurons based on the multi-synaptic connections by back propagation based on the scale factor, the first gradient substitution function and the training data set.

8. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the network model training method based on multi-synaptic connected pulse neurons as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the network model training method based on multi-synaptic connected pulse neurons as described in any one of claims 1 to 6.

Images