A pulse data classification model construction method and a pulse data classification method

Abstract

This invention discloses a method for constructing a pulse data classification model and a pulse data classification method, belonging to the field of classification and recognition technology using spiking neural networks. The construction method includes: training a spiking neural network; the gated parameter neuron layer includes: a synaptic current input terminal for calculating the synaptic current input at time t, and a membrane voltage input terminal for obtaining the membrane voltage v of the input neuron at time t. t The first to fourth logic gates are used to calculate the hidden membrane voltage calculation unit, which is used to calculate the hidden membrane voltage at time t. The pulse emission unit is used to emit the pulse and calculate the output membrane voltage. The output membrane voltage calculation unit is used to calculate the output membrane voltage at time t+1. By introducing the gating structure, the spiking neural network can better fit biological characteristics in terms of parameter distribution, better adapt to backpropagation training, improve the training effect of the spiking neural network and the classification accuracy of the spiking neural network for classification.

Description

Technical Field

[0001] This invention belongs to the field of classification and recognition technology using spiking neural networks, and more specifically, relates to a method for constructing a spiking data classification model and a spiking data classification method. Background Technology

[0002] Spiking Neural Networks (SNNs), often referred to as "third-generation neural networks," are brain-inspired models that simulate the dynamics of the human brain. Unlike artificial neural networks widely used in deep learning, SNNs employ spiking neurons to output binary spikes instead of continuous values. SNNs utilize both spike firing rate and spike timing information to process spatiotemporal information. Due to their sparse, asynchronous operating mechanism, SNNs can better handle event-based data and reduce network energy consumption. SNNs have been applied to image and audio classification, brain mapping and understanding, robot control, and more.

[0003] The basic unit in a spiking neural network is the neuron, typically as follows: Figure 1 The leaky-integration-firing (LIF) neuron shown simulates the dynamics of a real neuron. In the computer simulation, discrete time steps are considered, and at time t, the membrane voltage v weighted by the time constant τ is calculated. t With synaptic current x t The sum of these values ​​yields the hidden state membrane voltage h. t If h t If the voltage exceeds the threshold voltage, then pulse s is emitted. t =1 parallel membrane voltage v t+1 To reset the potential; otherwise, no pulse s is issued. t =0 and retain v t+1 =h t .

[0004] However, LIF has the following limitations: (1) The neuron parameters (time constant and threshold potential) of LIF neurons are usually set to the same value across different neurons and are not changeable over time. However, it has been widely observed in experiments that these electrophysiological characteristics of real neurons in human and animal nervous systems are dynamically heterogeneous in time and space; (2) The initial values ​​of neuron parameters in traditional LIF models are usually manually specified. However, finding the optimal values ​​of the parameters is difficult and may require time-consuming experimental trial and error; (3) Existing SNNs, similar to classic recurrent neural networks, also encounter the problem of gradient vanishing over time. Gradient vanishing will make the network difficult to train, difficult to capture long-distance semantic information, and difficult to handle long-term dependencies.

[0005] In summary, the biomimetic performance of spiking neural networks still has shortcomings. They cannot accurately model the operating mechanism of the real brain, and the training effect and classification accuracy of spiking neural networks are not ideal. Summary of the Invention

[0006] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a method for constructing a pulse data classification model and a pulse data classification method. The purpose is to fit real biological characteristics from the perspective of parameter distribution, to more realistically model the operating mechanism of the real brain, and to improve the training effect and classification accuracy of the spiking neural network.

[0007] To achieve the above objectives, according to one aspect of the present invention, a method for constructing a pulse data classification model is provided, comprising:

[0008] The spiking neural network is trained by using audio or image pulse data with classification labels as input and the classification results as output to make the output results of the spiking neural network converge.

[0009] The gating parameter neuron layer in the spiking neural network includes:

[0010] The synaptic current input terminal is used to calculate the synaptic current vector input at time t. in, W is the firing pulse vector of the previous gating parameter neuron layer at time t. l-1 This is the synaptic weight matrix;

[0011] The membrane voltage input terminal is used to obtain the membrane voltage vector v of the input neuron at time t. t ;

[0012] The first to fourth logic gates, and the kth logic gate are used to determine the membrane voltage vector v. t and synaptic current vector x t Calculate the signal of the k-th logic gate In the formula, σ(*) is the activation function. All of these are weight matrices for the k-th logic gate;

[0013] The hidden membrane voltage calculation unit is used to calculate the hidden membrane voltage vector at time t. In the formula, ⊙ represents element-wise multiplication;

[0014] Pulse delivery unit, used to deliver pulse vectors ∈(·) is the Heaviside function;

[0015] The output membrane voltage calculation unit is used to calculate the output membrane voltage vector at time t+1. In the formula, v reset To reset the potential vector;

[0016] The membrane voltage output terminal is used to output the membrane voltage v at time t+1. t+1 .

[0017] In one embodiment, the activation function used by the first, second, and fourth logic gates is the Sigmoid function, and the activation function used by the third logic gate is the Tanh function.

[0018] In one embodiment, when each batch of samples is trained, the distribution of each reset potential element in the reset potential vector follows a normal distribution.

[0019] In one embodiment, the normal distribution has a mean of 0 and a standard deviation of 0.05.

[0020] In one embodiment, during training, a non-differentiable Heaviside activation function is used in forward propagation, and a differentiable arctangent activation function is used in backpropagation.

[0021] In one embodiment, the pulse data is obtained by converting audio data or image data into pulses.

[0022] According to another aspect of the present invention, a pulse data classification method is provided, comprising:

[0023] Audio or image pulse data is input into a pulse data classification model for classification, wherein the pulse data classification model is constructed using the pulse data classification model construction method described above.

[0024] In one embodiment, the pulse data is obtained by converting audio data or image data into pulses.

[0025] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0026] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0027] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0028] This invention improves the neuron layer by adding four logic gates: the first is a forget gate, the second is a threshold gate, the third is a bypass gate, and the fourth is an input gate. By introducing this gated structure, the spiking neural network can better fit biological characteristics in terms of parameter distribution, better adapt to backpropagation training, and improve the training effect and classification accuracy of the spiking neural network. A detailed analysis follows:

[0029] (1) It features heterogeneous neuron parameters without requiring manual initialization. Traditional LIF neurons typically have the same neuron parameters (time constant and threshold potential) across different neurons, and these parameters are immutable over time. In this invention, a threshold gate (a second logic gate) is used, and the gate's output is determined by the gate input v at each time step t. t and x t Therefore, the threshold potential output by the threshold gate changes over time, and manual initialization of the threshold potential is not required. The weights of the threshold gate can be trained to different parameters during training based on backpropagation between different neurons in different layers, so the threshold potential output by the threshold gate is also different in the spatial domain, and manual initialization is not required. Thus, the spiking neural network of this invention achieves spatial-temporal heterogeneous threshold potentials without manual initialization. The forget gate (first logic gate) and input gate (fourth logic gate) also have the above characteristics; their outputs change over time, and the gate weight parameters at different spatial locations can be different, also exhibiting spatiotemporal heterogeneity.

[0030] (2) Alleviating long-term dependence. By replacing the traditional membrane leakage factor with a forget gate (first logic gate) and an input gate (fourth logic gate), the gradient vanishing in the membrane potential pathway can be mitigated. Simultaneously, to preserve long-distance semantic information, a bypass gate is added to establish a direct connection between adjacent membrane potentials v. t-1 and v t The bypass bypasses the pulse firing mechanism, allowing the membrane voltage state to be directly and continuously transmitted. Attached Figure Description

[0031] Figure 1 A schematic diagram simulating the structure of a real neuron in the Leak-Integration-Fire (LIF) neuron of traditional technology;

[0032] Figure 2 The pulse propagation mechanism of a spiking neural network is illustrated in one embodiment.

[0033] Figure 3 This is a schematic diagram of the structure of a gated parameter neuron layer according to one embodiment;

[0034] Figure 4(a) is a probability density histogram of the time constant τ1 of the gate parameter neuron layer in one embodiment;

[0035] Figure 4(b) is a probability density histogram of the time constant τ2 of the gate parameter neuron layer in one embodiment;

[0036] Figure 4(c) is a probability density histogram of the membrane time constant of the spinous neurons in the primary visual cortex of a mouse brain in an Allen brain atlas of an embodiment.

[0037] Figure 5(a) is a histogram of the threshold potential probability density of a gating parameter neuron layer in one embodiment;

[0038] Figure 5(b) is a probability density histogram of the threshold potential of a mouse neuron in one embodiment;

[0039] Figure 6 This is a processed pulse-form audio sample from one embodiment;

[0040] Figure 7 This is a comparison chart of model classification performance on the Spiking Heidelberg digits dataset, which is an example of this implementation. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0042] To facilitate understanding of this invention, the pulse propagation mechanism of spiking neural networks will be briefly introduced first.

[0043] like Figure 2 The diagram illustrates the pulse propagation mechanism of a spiking neural network in one embodiment. The spiking neural network has multiple neuron layers, and each neuron layer contains multiple neurons. The pulse signal propagates along both the time and space dimensions.

[0044] In the direction of time propagation, in computer simulations, considering discrete time steps, the membrane voltage v output by each neuron layer at the previous time step... t The membrane voltage input at the current time step of this neuron layer, and the membrane voltage output at the current time step, v. t+1 The membrane voltage is used as the input for the next time step of this neuron layer.

[0045] In the direction of spatial propagation, the current neuron layer calculates the synaptic current based on the firing pulse of the previous neuron layer, and obtains the firing pulse of the current neuron layer through a series of calculations. The firing pulse of the current neuron layer is provided to the next neuron layer for the next neuron layer to calculate the synaptic current.

[0046] This invention proposes a method for constructing a pulse data classification model, comprising: training a spiking neural network with audio or image pulse data bearing classification labels as input and the classification results as output, so that the output results of the spiking neural network converge. The specific training method is a backpropagation algorithm with surrogate gradients, which can be referenced from the training of traditional spiking neural networks. The spiking neural network trained in this invention is an improved gated parameter neural network, wherein the neuron layers are gated parameter neuron layers with added logic gates.

[0047] like Figure 3 As shown, in one embodiment, the gated parameter neuron layer includes a synaptic current input terminal, a membrane voltage input terminal, first to fourth logic gates, a hidden membrane voltage calculation unit, a pulse firing unit, an output membrane voltage calculation unit, and a membrane voltage output terminal.

[0048] The synaptic current input terminal is used to calculate the synaptic current vector input at time t. in, W is the firing pulse vector of the previous gating parameter neuron layer at time t. l-1 This is the synaptic weight matrix. Each neuron layer contains a large number of neurons, and there are multiple parallel synaptic current input nodes at the synaptic current input end, with each neuron corresponding to one synaptic current input node. Each neuron in a neuron layer is connected to every neuron in the previous neuron layer to receive the emitted pulses. The connections between adjacent neuron layers form a fully connected layer, and the synaptic current vector x... t Each element in the matrix represents the input information of the corresponding synaptic current input node, and the synaptic weight matrix is ​​actually the weight matrix of the fully connected layer.

[0049] The membrane voltage input terminal is used to obtain the membrane voltage vector v of the input neuron at time t. t Similarly, the membrane voltage input terminal also has multiple parallel voltage input nodes. Each neuron has one membrane voltage input node, and each neuron obtains its output membrane voltage at the current time step through its respective membrane voltage input node, which is the membrane voltage output after processing at the previous time step. Membrane voltage vector v t Each element in the table represents the input information for the corresponding membrane voltage input node.

[0050] In the first to fourth logic gates, the k-th logic gate is used to determine the membrane voltage vector v. t and synaptic current vector x t Calculate the signal of the k-th logic gate In the formula, σ(*) is the activation function. All of these are weight matrices for the k-th logic gate.

[0051] To better distinguish the logic gates, the first logic gate is defined as the forget gate F, and the signal of the first logic gate is the forget gate signal, i.e. All of these are the weight matrices of the forget gate;

[0052] The second logic gate is defined as the threshold gate T, and the signal of the second logic gate is the threshold gate signal, i.e. in, All are weight matrices of the threshold gates; the third logic gate is defined as bypass gate B, and the signal of the third logic gate is the bypass gate signal, i.e. in, All of these are the weight matrices of the bypass gates; the fourth logic gate is defined as input gate I, and the signal of the fourth logic gate is the input gate signal, i.e. in, All of these are the weight matrices of the input gate.

[0053] Forgot Gate Signal Threshold gate signal Bypass door signal and input gate signal All are vectors, and each element in the vector corresponds to a neuron. The calculation formula for each logic gate is the same, but the weight matrix is ​​different and needs to be determined based on training.

[0054] In one embodiment, the activation function used for the first, second, and fourth logic gates is the Sigmoid function; that is, the forget gate, input gate, and threshold gate use the Sigmoid function as their activation function. Since the leakage factors calculated by the first and fourth logic gates must be in the range of 0-1, and the threshold potential calculated by the second logic gate is typically chosen to be in the range of 0-1, using the Sigmoid function as the activation function ensures the generation of positive values ​​(0-1). The activation function used for the third logic gate is the Tanh function; that is, the bypass gate uses the Tanh function as its activation function to allow for either enhanced (positive) or suppressed (negative) transmission of the membrane voltage.

[0055] The hidden membrane voltage calculation unit is used to calculate the hidden membrane voltage vector at time t. In the formula, ⊙ represents element-wise multiplication. The hidden-state membrane voltage calculation unit essentially utilizes the forget gate signal. and input gate signal Calculate the hidden state membrane voltage Hidden membrane voltage h t Let be a vector, where each element is the hidden membrane voltage of the corresponding neuron.

[0056] For example:

[0057]

[0058] These are the latent membrane voltages of the first, second, and third neurons in the neuron layer, respectively. These are the forgetting gate signals of the first, second, and third neurons in the neuron layer, respectively. These are the membrane voltages acquired from the first, second, and third neurons in the neuron layer, respectively. These are the input gate signals for the first, second, and third neurons in the neuron layer, respectively. These are the synaptic currents acquired by the first, second, and third neurons in the neuron layer, respectively.

[0059] The pulse delivery unit is used to deliver pulse vectors. ∈(·) is the Heaviside function. For each neuron, for example, the i-th neuron, Let be the threshold signal of the i-th neuron, if Then emit pulse otherwise

[0060] The output membrane voltage calculation unit is used to calculate the output membrane voltage vector at time t+1. In the formula, v reset To reset the potential vector, the output membrane voltage calculation unit essentially calculates the output membrane voltage based on the hidden membrane voltage, the emitted pulse, and the bypass gate signal.

[0061] For a single neuron:

[0062] when

[0063] when

[0064] That is, if the i-th neuron fires a pulse, the output membrane voltage is independent of the latent membrane voltage and is the sum of the reset potential and the bypass gate signal; if the i-th neuron does not fire a pulse, the output membrane voltage is independent of the reset potential and is the sum of the latent membrane voltage and the bypass gate signal.

[0065] The membrane voltage output terminal is used to output the membrane voltage v at time t+1. t+1 .

[0066] The membrane voltage output terminal has multiple output nodes. Each neuron has one output node, which outputs the membrane voltage of the corresponding neuron at time t+1. This membrane voltage is then input into the voltage input node of the same neuron to perform processing for the next time step. This process is repeated to extract features and perform autonomous learning.

[0067] The weights in the gating parameter neuron layer mentioned above are the parameters that need to be trained.

[0068] In one embodiment, during model training, reset potentials are randomly initialized at the beginning of each batch of sample training. The distribution of each reset potential element in the reset potential vector follows a normal distribution, meaning that the reset potentials corresponding to all neurons in the neuron layer follow a normal distribution. Specifically, this can be a normal distribution with a mean of 0 and a standard deviation of 0.05. The absolute difference between the reset potential and the threshold potential determines how much the membrane voltage needs to change to generate a pulse. Considering that the threshold potential has been generated by a trainable threshold gate, the reset potentials during training are randomly initialized to maintain parameter heterogeneity. After training is complete, the reset potentials during testing are set to a constant to avoid the influence of randomness. Similar to other methods, the reset potentials during testing are set to 0 in this embodiment.

[0069] In one embodiment, the spiking network is trained using backpropagation with alternative gradients. Specifically, a non-differentiable heaviside activation function is used in the forward propagation, and a differentiable arctangent function is used instead in the backpropagation.

[0070]

[0071] Specifically, the pulse data classification model targets pulse data obtained after pulse conversion from images, audio, etc., that need to be classified and identified. The specific pulse conversion process can be implemented using existing methods.

[0072] Accordingly, the present invention also relates to a pulse data classification method, comprising:

[0073] Audio or image pulse data is input into a pulse data classification model for classification. This pulse data classification model is constructed using the aforementioned pulse data classification model construction method. Specifically, the pulse data is obtained by converting audio or image data into pulses.

[0074] Accordingly, the present invention also relates to an electronic device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the above-described method.

[0075] Accordingly, the present invention also relates to a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the above-described method.

[0076] Through the above approach, a neuronal structure that can generate heterogeneous neuron parameters, eliminates the need for manual initialization, and alleviates long-term dependence can be used to fit biological characteristics in terms of parameter distribution. It can also better adapt to backpropagation training, thereby improving the training effect and classification accuracy of spiking neural networks.

[0077] First, verify the biomimetic performance of the spiking neural network. For example... Figures 4(a) to 4(c) The diagram shows the distribution of the time constant, with the horizontal axis representing the value and the vertical axis representing the probability density. Specific scale details are omitted from the diagram. Figure 4(a) is a histogram showing the distribution of the time constant τ1 of the gating parameter neuron layer. The distribution of the time constant τ1 reflects the forgetting gate signal. The distribution of values ​​of each neuron in the middle, firstly... Take the average over time and then transform it: Figure 4(b) is a histogram showing the distribution of the time constant τ2 of the gating parameter neuron layer. The distribution of the time constant τ2 reflects the input gate signal. The distribution of values ​​of each neuron in the middle, firstly... Take the average over the time dimension, and then transform it: Figure 4(c) is a histogram of the membrane time constants of the spinous neurons in the primary visual cortex of the mouse brain in the Allen brain atlas. The distributions of the time constants τ1 and τ2 of the gating parameter neuron layer are similar to those of the membrane time constants of the spinous neurons in the primary visual cortex of the mouse brain, both approximating a log-normal distribution. Figures 5(a) and 5(b) show schematic diagrams of the threshold potential distribution, with the horizontal axis representing the values ​​and the vertical axis representing the probability density. Specific scales are omitted in the diagrams. Figure 5(a) shows the threshold potential distribution of the gating parameter neuron layer, and Figure 5(b) shows the threshold potential distribution of mouse neurons. Both are also very similar, approximating a log-normal distribution. In summary, this demonstrates that gating parameter neural networks can well fit biological characteristics in terms of parameter distribution.

[0078] The following specific embodiment verifies the training effect of the above-mentioned gate parameter neural network.

[0079] We chose Spiking Heidelberg digits (SHD) as our dataset. Spiking Heidelbergdigits (SHD) is a pulse-based audio dataset consisting of high-quality spoken English and German digit recordings, with 20 classes from 0 to 9. SHD contains 8,156 and 2,264 digit sequences, used for training and testing, respectively. We further retained 25% of the sequences from the training set for validation. The pulses in the dataset have 700 input channels, which is the input dimension of the first fully connected layer of the network.

[0080] Data preprocessing involves padding audio samples shorter than one second with zeros and truncating samples longer than one second. The samples are then divided into multiple time steps, and pulses are calculated for each time step. A processed sample is shown below. Figure 6 As shown.

[0081] For speech recognition tasks on the SHD dataset, we used cross-entropy loss, backpropagation with alternative gradients, Adam optimizer, and cosine annealing algorithm to train the GPN network. Figure 7 Experimental results were presented, showing that GPN (the gated parameter neural network of this invention) achieved superior classification accuracy compared with classic networks such as CNN (convolutional neural network), GRU (gated recurrent unit), LSTM (long short-term memory network), VanillaRNN (recurrent neural network), Cuba-LIF (spiking neural network based on current leakage-integration-firing neurons), and LIF (spiking neural network using leakage-integration-firing neurons).

[0082] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a pulse data classification model, characterized in that, Comprising: training the spiking neural network with audio or image spiking data classified with a classification label as input and a classification result as output to make the spiking neural network output result converge; wherein the gating parameter neuron layer in the spiking neural network comprises: synaptic current input for computing the synaptic current vector at time t wherein, is the firing vector of the previous gated parameter neuron layer at time t, W l-1 is the synaptic weight matrix; a membrane voltage input terminal for obtaining a membrane voltage vector v of the input neuron at time t t ; The first to fourth logic gates, the kth logic gate is used to calculate the kth logic gate signal according to the membrane voltage vector v t And the synaptic current vector x t The kth logic gate signal is calculated Wherein, sigma (*) is an activation function, And w k are weight matrices of the kth logic gate; a hidden state membrane voltage calculation unit configured to calculate a hidden state membrane voltage vector h at time t t wherein denotes element-wise multiplication.​ a pulse emitting unit for emitting a pulse vector ∈(·) is the Heaviside function; an output film voltage calculation unit for calculating an output film voltage vector at time t+1 where v reset is a reset potential vector; a membrane voltage output terminal for outputting the membrane voltage v at time t+1 t+1 . 2.The method of claim 1, wherein, the activation function used by the first, second and fourth logic gates is a Sigmoid function, and the activation function used by the third logic gate is a Tanh function. 3.The method of claim 1, wherein, When training each batch of samples, the distribution of each reset potential element in the reset potential vector is subject to a normal distribution. 4.The method of claim 3, wherein, The mean of the normal distribution is 0 and the standard deviation is 0.

05. 5.The method of claim 1, wherein, During training, a non-differentiable Heaviside activation function is used in forward propagation, and a differentiable inverse tangent activation function is used in back propagation. 6.The method of claim 1, wherein, The spiking data is obtained by converting audio data or image data into spikes.

7. A method of classifying pulse data, characterized by, Comprising: inputting audio or image spiking data into a spiking data classification model for classification, wherein the spiking data classification model is constructed by the spiking data classification model construction method of any one of claims 1 to 6.

8. The method of claim 7, wherein, The spiking data is obtained by converting audio data or image data into spikes. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images