MI-EEG classification and identification method based on spiking neural network

Abstract

The invention discloses an MI-EEG classification and recognition method based on a pulse neural network. The method comprises the steps that electroencephalogram signals are collected and preprocessed; converting the two-dimensional time-frequency image into a pulse sequence in a Poisson coding mode; generating a frequency histogram of excitation neuron pulse distribution by inputting the pulse sequence into a pulse neural network; and classifying the output pulses by adopting a voting method. According to the invention, the pulse neural network is used to identify and classify the image, so that the precision meets the requirement, and the expenditure of power consumption and the like is reduced; meanwhile, an STDP learning mechanism and a lateral inhibition mechanism are introduced, the connection strength is adjusted through the STDP learning mechanism, and excitation neurons of unissued pulses are inhibited through the lateral inhibition mechanism; the STDP learning mechanism and the side suppression mechanism jointly influence the neuron group, so that the neurons of the corresponding instructions can emit pulses more easily, the membrane potential of the inactive neurons is reduced, the cost of emitting the pulses is increased, and the pulses are less likely to be emitted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biological signal recognition, and in particular to a method for MI-EEG classification and recognition based on a spiking neural network. BACKGROUND

[0002] With the rapid development of computer technology and brain science, more and more researchers have joined the field of BCI rehabilitation based on motor imagery. The key and difficulty of the BCI rehabilitation system lies in the decoding of motor intention in MI-EEG, i.e. motor imagery recognition, including signal preprocessing, feature extraction and classification.

[0003] The prior art solves the above technical problems by using a motor imagery recognition method based on machine learning, but there are still some defects compared to deep learning, such as the complexity of the step of manually extracting features, the inability to mine hidden spatio-temporal information in raw data, etc., which is exactly the advantage of deep learning, so more researchers have begun to study motor imagery recognition methods based on deep learning.

[0004] In order to solve this problem, researchers have proposed a scheme using a neural network. The neural network is used to recognize and classify the EEG signals extracted by the feature, so as to effectively control the external device. For example, by analyzing and recognizing the EEG signals of the patient's motor imagery, the results are used to control the movements of the characters in the virtual environment, and then the information is fed back to the patient through the feedback link, and the patient adjusts the way of motor imagery according to the feedback information. However, most of the second generation neural networks such as convolutional neural networks are used for processing. The second generation neural network has done a good job in image processing, but it requires a large amount of calculation and power consumption, especially in complex neural networks. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a method for MI-EEG classification and recognition based on a spiking neural network. The present application uses a spiking neural network to recognize and classify images, which reduces the expenditure in terms of power consumption while meeting the accuracy requirements.

[0006] The technical scheme of the present application is as follows: a method for MI-EEG classification and recognition based on a spiking neural network, comprising the following steps:

[0007] S1), collecting EEG signals and preprocessing to obtain a two-dimensional time-frequency image;

[0008] S2), converting the two-dimensional time-frequency image into a pulse sequence using a Poisson coding method;

[0009] S3), generating a frequency histogram of the excited neuron pulse firing by inputting the pulse sequence into the pulse neural network, and screening the histogram data to obtain corresponding control external device instructions;

[0010] S4), repeating steps S1)-S3) until the set number of images is reached, and then classifying the pulses output by step S3) using a voting method.

[0011] Preferably, in step S1), the C3, Cz, and C4 channel brain electrical signals are subjected to 8-30 Hz band-pass filtering; then the filtered C3, Cz, and C4 channel MI-EEG is subjected to wavelet transform; and the frequency bands of the three electrodes are combined and adjusted to the same size of two-dimensional time-frequency map.

[0012] Preferably, in step S2), the pulse neural network comprises an encoding layer, an excitation layer, and an inhibition layer, one-to-one excitation connections are used between the encoding layer and the excitation layer, and full inhibition connections are used between the excitation layer and the inhibition layer.

[0013] Preferably, in step S2), the pulse neurons in the pulse neural network use LIF pulse neurons.

[0014] Preferably, in step S2), the connection strength between the encoding layer and the excitation layer is dynamically updated using the STDP rule, and the connection strength between the excitation layer and the inhibition layer is fixed.

[0015] Preferably, in step S2), a lateral inhibition mechanism is introduced through the inhibition layer, which sends inhibition signals to the excited neurons that do not fire pulses, so that the membrane potential of these neurons is reduced and it is more difficult to fire pulses. The lateral inhibition mechanism suppresses the inactive neuron group and reduces the influence of the interference signal.

[0016] Preferably, in step S2), the two-dimensional time-frequency map is subjected to Poisson coding through the encoding layer of the pulse neural network, and the image data is converted into a corresponding pulse sequence.

[0017] Preferably, in step S2), the expression of the STDP rule is:

[0018]

[0019] In the formula, Δy represents the increment of the connection strength between synapses, which is updated continuously with the firing of synaptic neurons; y max is the maximum upper limit value of the synaptic connection strength; y is the connection strength between synapses; b and a represent the weight update rate and the dependence on the weight of the previous time, respectively; u p represents the presynaptic trace; u lThe presynaptic trace of a neuron firing a spike; is the time constant.

[0020] The beneficial effects of the present invention are:

[0021] 1. The present invention utilizes a pulse neural network to perform image recognition and classification processing, while achieving the required accuracy while minimizing power consumption and other expenses;

[0022] 2. The present invention simultaneously introduces the STDP learning mechanism and the lateral inhibition mechanism, which adjusts the connection strength through the STDP learning mechanism and inhibits the excited neurons that have not fired pulses through the lateral inhibition mechanism; the STDP learning mechanism and the lateral inhibition mechanism jointly affect the neuronal population, making it easier for neurons corresponding to instructions to fire pulses, and reducing the membrane potential of inactive neurons, making it more costly for them to fire pulses and making it less likely to fire pulses. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0024] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0025] like Figure 1 As shown, this embodiment provides a method for MI-EEG classification and recognition based on a spiking neural network, comprising the following steps:

[0026] S1), collecting EEG signals and preprocessing them to obtain two-dimensional time-frequency images;

[0027] The EEG signals collected in this embodiment are from BCI Competition II datasets III, and the preprocessing is as follows:

[0028] The EEG signals of the C3, Cz, and C4 channels were band-pass filtered at 8-30 Hz. The MI-EEG signals of the filtered C3, Cz, and C4 channels were then subjected to wavelet transform. The frequency bands of the three electrodes were combined and adjusted to a 64×64 two-dimensional time-frequency diagram.

[0029] In this embodiment, the wavelet transform (CWT) can flexibly change the window size according to the state of the EEG frequency by using a series of basis functions with specific time resolution and frequency resolution; the basis functions are square-integrable functions whose amplitude increases from zero and then decreases until it reaches zero. The wavelets in this embodiment include Morlet wavelets and Haar wavelets.

[0030] Among them, the Morlet wavelet is defined as:

[0031]

[0032] Where ω is the average of the lower cutoff frequency and the upper cutoff frequency of the Morlet wavelet; σ is the control Gaussian function The scale parameter; ψ(t) is the mother wavelet; j represents the imaginary number; t represents time;

[0033] Given a mother wavelet ψ(t), wavelet bases of different sizes and displacements are obtained by scaling and shifting the mother wavelet ψ(t) in the time domain. Among them, α is the scaling factor, corresponding to the frequency dimension, which mainly controls the scale of the wavelet in the time domain; τ is the translation factor, corresponding to the time dimension, which mainly controls the factor of the wavelet in displacement. The scale parameter can be set arbitrarily in the continuous wavelet transform (CWT). In this way, the continuous wavelet transform can simultaneously achieve resolution analysis of EEG signals in the time domain and frequency domain by using the translation factor and scaling factor. Different wavelet bases can be selected as needed to better adapt to the characteristics and requirements of the signal. The continuous wavelet transform (CWT) of an EEG signal is the convolution of the EEG signal and the wavelet basis function, that is:

[0034]

[0035] Where X(t,α) represents the wavelet coefficient of the signal at time t and scaling factor α, reflecting the time-frequency energy distribution; x(τ) represents the original EEG signal, which serves as the input of CWT and contains the time domain information to be analyzed;

[0036] The square of the amplitude of X(t,α) is called the wavelet scalogram.

[0037] In this embodiment, EEG is a complex biological signal, and its ERD / ERS pattern is mainly reflected in the 8-30 Hz frequency band. In this embodiment, frequency components irrelevant to the motor imagery task are filtered out through preprocessing. For example, most of the electromyographic signals are distributed in the frequency range greater than 30 Hz; and the eye movement signals are concentrated in the frequency range below 5 Hz.

[0038] S2) using Poisson coding to convert the two-dimensional time-frequency image into a pulse sequence; specifically comprising the following steps:

[0039] S21) Normalize the input intensity and normalize the intensity value of the time-frequency graph to the interval [0,1], that is:

[0040]

[0041] Where P(t,f) represents the normalized intensity of the time-frequency graph; f is the frequency of the time-frequency graph; I min , I max are the minimum and maximum intensity values ​​of the time-frequency graph; I(t,f) is the input time-frequency graph;

[0042] S22), define time discretization

[0043] Divide the continuous time into discrete time windows, time step is Δt, each time window corresponds to a time point t of the time-frequency diagram;

[0044] S23), generate Poisson pulse sequence

[0045] For each time-frequency unit (t, f), the probability of generating a pulse in the time window (t·Δt, (t+1)·Δt) is:

[0046] λ(f,t)=λ max ·P(t,f);

[0047] Wherein, λ max is the maximum pulse rate; the dimension is: channel number C×time T×frequency F, then independently encode Poisson for each channel:

[0048] S c (t,f)~Bernoulli(λ max ·P c (t,f)·Δt);

[0049] Wherein, P c (t,f) is the normalized intensity value of channel c; S c (t,f) represents whether channel c generates a pulse in the time window (t·Δt, (t+1)·Δt) and the frequency unit f;

[0050] S3), generate the frequency histogram of excitatory neuron pulse firing by inputting the pulse sequence into the pulse neural network, and obtain the corresponding control external device instruction by screening the histogram data.

[0051] S4), repeat steps S1)-S3) until the set number of images is reached, then classify the pulses output by step S3) using voting method, which includes the following steps:

[0052] S41), pulse counting

[0053] For each output neuron i, count the total number of pulses W i , that is:

[0054]

[0055] In the formula, S i ∈{0,1} represents whether neuron i fires a pulse at time step t;

[0056] S42), voting decision

[0057] The class corresponding to the neuron with the most pulse counts is selected as the prediction result, i.e.:

[0058] PredictedClass=arg max(W i );

[0059] In the formula, Predicted Class represents the final class label predicted by the voting decision mechanism.

[0060] If there are multiple classes with the same number of pulses, a random selection or classification combined with pulse timing information can be performed, such as preferentially selecting the class that reaches the threshold earliest.

[0061] As preferred in the embodiment, in step S2), the pulse neural network includes an encoding layer, an excitation layer, and an inhibition layer, one-to-one excitation connections are used between the encoding layer and the excitation layer; and full inhibition connections are used between the excitation layer and the inhibition layer.

[0062] The connection strength between the encoding layer and the excitation layer is dynamically updated using the STDP rule; and the connection strength between the excitation layer and the inhibition layer is fixed.

[0063] A lateral inhibition mechanism is introduced through the inhibition layer, the lateral inhibition mechanism sends an inhibition signal to the excitation neurons that do not fire pulses, so that the membrane potential of these neurons is reduced and it is more difficult to fire pulses. The lateral inhibition mechanism suppresses the inactive neuron group and reduces the influence of the interference signal.

[0064] The expression of the STDP rule is:

[0065]

[0066] In the formula, Ay represents the increment of the connection strength between synapses, which is continuously updated with the firing of synaptic neurons; y max is the maximum upper limit value of the synaptic connection strength; y is the connection strength between synapses; b and a represent the weight update rate and the weight dependence on the previous time, respectively; u p represents the presynaptic trace; u l is the presynaptic trace when the neuron fires a pulse; is a time constant.

[0067] As preferred in the embodiment, in step S2), the pulse neurons in the pulse neural network use LIF pulse neurons.

[0068] As preferred in the embodiment, in step S2), the two-dimensional time-frequency graph is Poisson encoded through the encoding layer of the pulse neural network, and the image data is converted into a corresponding pulse sequence.

[0069] The foregoing embodiments and description of the application only illustrate the principles of the application and the best mode presently contemplated by the inventors. Nothing in this detailed description should be taken to imply that any aspect or feature of the application is essential. The scope of the application should be determined by the appended claims and their legal equivalents, and the full scope of replacement claims to be submitted to the Patent and Trademark Office (PTO) if appropriate.

Claims

1. A method for MI-EEG classification and recognition based on a spiking neural network, characterized in that: The steps include: S1), collecting EEG signals and preprocessing them to obtain two-dimensional time-frequency images; S2), using Poisson coding to convert the two-dimensional time-frequency image into a pulse sequence; S3), generating a frequency histogram of the pulse emission of the excitatory neurons by inputting the pulse sequence into the pulse neural network, and obtaining the corresponding control instructions for the external device by filtering the histogram data; S4), repeating steps S1)-S3) until the set number of images is reached, and then classifying the pulses output in step S3) using a voting method.

2. The method for MI-EEG classification and recognition based on a spiking neural network according to claim 1, characterized in that: In step S1), the EEG signals of the C3, Cz, and C4 channels are band-pass filtered at 8-30 Hz; then, the MI-EEG signals of the filtered C3, Cz, and C4 channels are subjected to wavelet transform respectively; and the frequency bands of the three electrodes are combined and adjusted to a two-dimensional time-frequency diagram of the same size.

3. The method for MI-EEG classification and recognition based on a spiking neural network according to claim 1, characterized in that: In step S2), the spiking neural network includes a coding layer, an excitation layer and an inhibition layer, and a one-to-one excitation connection is used between the coding layer and the excitation layer; and a full inhibition connection is used between the excitation layer and the inhibition layer.

4. The method for MI-EEG classification and recognition based on a spiking neural network according to claim 3, characterized in that: In step S2), the spiking neurons in the spiking neural network are LIF spiking neurons.

5. The method for MI-EEG classification and recognition based on spiking neural network according to claim 3, characterized in that: In step S2), the connection strength between the coding layer and the excitation layer is dynamically updated using the STDP rule; the connection strength between the excitation layer and the inhibition layer is fixed.

6. The method for MI-EEG classification and recognition based on spiking neural network according to claim 3, characterized in that: In step S2), a lateral inhibition mechanism is introduced through the inhibitory layer. The lateral inhibition mechanism sends inhibitory signals to the excitatory neurons that have not yet fired pulses, thereby lowering the membrane potential of these neurons and making it more difficult for them to fire pulses. The lateral inhibition mechanism inhibits the inactive neuronal population and reduces the impact of interference signals.

7. The method for MI-EEG classification and recognition based on spiking neural network according to claim 5, characterized in that: In step S2), the expression of the STDP rule is: Where Δy represents the increment of the connection strength between synapses, which is continuously updated as the synaptic neuron pulses are released; y max is the maximum upper limit of synaptic connection strength; y is the connection strength between synapses; b and a represent the weight update rate and the weight dependence on the previous moment respectively; u p represents the presynaptic trace; u l The presynaptic trace of a neuron firing a spike; is the time constant.

8. The method for MI-EEG classification and recognition based on a spiking neural network according to claim 3, characterized in that: In step S2), the two-dimensional time-frequency image is Poisson encoded through the encoding layer of the pulse neural network to convert the image data into a corresponding pulse sequence.

9. The method for MI-EEG classification and recognition based on spiking neural network according to claim 8, characterized in that: In step S2), the two-dimensional time-frequency image is converted into a pulse sequence using Poisson coding, which specifically includes the following steps: S21) Normalize the input intensity and normalize the intensity value of the time-frequency graph to the interval [0,1], that is: Where P(t,f) represents the normalized intensity of the time-frequency graph; f is the frequency of the time-frequency graph; I min , I max are the minimum and maximum intensity values ​​of the time-frequency graph; I(t,f) is the input time-frequency graph; S22) Define time discretization Divide the continuous time into discrete time windows with a time step of Δt. Each time window corresponds to a time point t in the time-frequency graph. S23), generate Poisson pulse sequence For each time-frequency unit (t,f), the probability of generating a pulse within the time window (t·Δt, (t+1)·Δt) is: λ(t,f)=λ max ·P(t,f); Among them, λ max is the maximum pulse rate; the dimension is: number of channels C × time T × frequency F, then Poisson encoding is performed independently on each channel: S c (t,f)~Bernoulli(λ max ·P c (t,f)·Δt); Among them, P c (t,f) is the normalized intensity value of channel c; S c (t,f) indicates whether channel c generates a pulse in the time window (t·Δt, (t+1)·Δt) and frequency unit f.

10. The method for MI-EEG classification and recognition based on spiking neural network according to claim 1, characterized in that: In step S4), the pulses outputted in step S3) are classified using a voting method, which specifically includes the following steps: S41), pulse counting For each output neuron i, count the total number of pulses W in the time window T i ,Right now: Where S i ∈{0,1} indicates whether neuron i emits a pulse at time step t; S42) Voting decision The category corresponding to the neuron with the largest pulse count is selected as the prediction result, that is: Predicted Class=arg max(W i ) Where Predicted Class represents the final category label predicted by the voting decision mechanism.

Images