Brain activity state recognition method, device, equipment and storage medium

Abstract

The embodiment of the present invention provides a method, device, equipment and storage medium for identifying brain activity states, the method comprising: obtaining the user's EEG signal data; obtaining the characteristic information of the EEG signal according to the user's EEG signal data and the target artificial neural network model; the target artificial neural network model includes spiking neurons; determining the brain activity state boundary information and target integration information according to the characteristic information of the EEG signal and the target spiking neural network model; the target spiking neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; the brain activity state boundary information is used to represent the time information of the brain activity state switching; the target integration information represents the brain activity information between adjacent brain activity state boundaries; and identifying the user's brain activity state according to the brain activity state boundary information and the target integration information. The method of the embodiment of the present invention realizes accurate identification of brain activity states.

Description

Technical Field

[0001] The present invention relates to the field of medical signal processing technology, and in particular to a method, device, equipment and storage medium for identifying brain activity states. Background Art

[0002] In the field of medical anesthesia, monitoring of anesthesia depth and analgesia level is an important reference for drug administration, and detection of brain activity status is an important way to judge the patient's physical condition. Therefore, how to accurately identify the user's brain activity status is of great significance.

[0003] In the related technology, the user's brain activity state is identified based on the traditional artificial neural network model. However, due to the low-speed backpropagation method and high computing cost of the artificial neural network, the recognition efficiency and accuracy of the user's brain activity state are low, making it difficult to carry out large-scale clinical applications. Summary of the invention

[0004] In view of the problems in the prior art, embodiments of the present invention provide a method, apparatus, device and storage medium for identifying brain activity states.

[0005] Specifically, the embodiment of the present invention provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides a method for identifying a brain activity state, comprising:

[0007] Obtain the user's EEG signal data;

[0008] Acquiring characteristic information of the EEG signal according to the EEG signal data of the user and a target artificial neural network model; the target artificial neural network model includes spiking neurons;

[0009] According to the characteristic information of the EEG signal and the target pulse neural network model, the brain activity state boundary information and the target integration information are determined; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; the brain activity state boundary information is used to represent the time information of the brain activity state switching; the target integration information represents the brain activity information between adjacent brain activity state boundaries;

[0010] The brain activity state of the user is identified according to the brain activity state boundary information and the target integration information.

[0011] Furthermore, acquiring characteristic information of the EEG signal according to the EEG signal data of the user and the target artificial neural network model includes:

[0012] Encoding the EEG signal data to obtain Mel-frequency cepstrum data corresponding to the EEG signal data of the user;

[0013] The Mel-frequency cepstrum data is input into a target artificial neural network model to obtain characteristic information of the EEG signal.

[0014] Furthermore, the step of inputting the characteristic information of the EEG signal into a target spiking neural network model to determine brain activity state boundary information and target integration information includes:

[0015] Inputting characteristic information of the EEG signal into a target spiking neural network model to determine the membrane potential of neurons in the target spiking neural network model;

[0016] Determining a pulse firing position based on a membrane potential of the neuron;

[0017] Determining brain activity state boundary information according to the pulse emission position;

[0018] According to the brain activity state boundary information, target integration information between adjacent brain activity state boundaries is obtained.

[0019] Furthermore, inputting the characteristic information of the EEG signal into the target spiking neural network model to determine the membrane potential of neurons in the target spiking neural network model includes:

[0020] The membrane potential of the neuron in the target spiking neural network model is determined based on the following method:

[0021] V k,t =V k,t-1 +F(I k,t ,V k,t-1 ,…);

[0022] The V k,t represents the membrane potential of the kth neuron at the tth time; the V k,t-1 represents the membrane potential of the kth neuron at the t-1th moment; k,t ,V k,t-1 , ...) represents the target dynamics model in the target pulse neural network model; the I k,t The current sequence corresponding to the characteristic information of the EEG signal.

[0023] Furthermore, the target dynamics model in the target pulse neural network model includes at least one of the following: a first-order dynamics model, a first-order dynamics with a dynamic threshold, a second-order dynamics model, and a second-order dynamics model with a self-inhibition term.

[0024] Furthermore, the determining of the pulse emission position based on the membrane potential of the neuron includes:

[0025] When the membrane potential of the neuron is greater than a first threshold, pulse emission is performed and a pulse emission position is determined;

[0026] Determining brain activity state boundary information according to the pulse emission position includes:

[0027] The pulse emission position is used as brain activity state boundary information.

[0028] In a second aspect, an embodiment of the present invention further provides a brain activity state recognition device, comprising:

[0029] An acquisition module, used to acquire the user's EEG signal data;

[0030] A processing module, used for acquiring characteristic information of the EEG signal according to the EEG signal data of the user and a target artificial neural network model; the target artificial neural network model includes spiking neurons;

[0031] A determination module, used to determine brain activity state boundary information and target integration information according to the characteristic information of the EEG signal and the target pulse neural network model; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; the brain activity state boundary information is used to represent the time information of the brain activity state switching; the target integration information represents the brain activity information between adjacent brain activity state boundaries;

[0032] The identification module is used to identify the user's brain activity state according to the brain activity state boundary information and the target integration information.

[0033] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for identifying brain activity states as described in the first aspect is implemented.

[0034] In a fourth aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the brain activity state recognition method as described in the first aspect is implemented.

[0035] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the brain activity state recognition method as described in the first aspect.

[0036] The brain activity state recognition method, device, equipment and storage medium provided by the embodiment of the present invention reduce the processing overhead of EEG signal data and speed up the calculation speed of EEG signal data by pulsing the artificial neural network model, so that the characteristic information of EEG signal can be extracted more quickly, and thus the efficiency of brain activity state recognition can be effectively improved; further, through the pulse neural network model, that is, the more brain-like pulse neural network is applied to brain activity state recognition, the boundary information and target integration information of brain activity state can be quickly and accurately determined by processing EEG signals in a way that is more in line with biological characteristics, so that the brain activity state can be accurately determined based on the boundary information and target integration information of brain activity state, so that the processing efficiency of EEG signal is improved, and thus the efficiency and accuracy of brain activity recognition can be improved. In the embodiment of the present invention, the precise calculation advantage of artificial neural network and the temporal information processing advantage of pulse neural network are combined, so that the processing efficiency of EEG signal is effectively improved, and the recognition efficiency and accuracy of brain activity state are improved while reducing energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 This is one of the flowcharts of the method for identifying brain activity states provided by an embodiment of the present invention;

[0039] Figure 2 This is the second flow chart of the brain activity state recognition method provided by the embodiment of the present invention;

[0040] Figure 3 This is a third flow chart of the method for identifying brain activity states provided in an embodiment of the present invention;

[0041] Figure 4 This is a fourth flow chart of the brain activity state recognition method provided by an embodiment of the present invention;

[0042] Figure 5 is a schematic diagram of the structure of a brain activity state recognition device provided by an embodiment of the present invention;

[0043] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The method of the embodiment of the present invention can be applied in medical signal processing scenarios to achieve accurate identification of brain activity states.

[0046] In the related technology, the user's brain activity state is identified based on the traditional artificial neural network model. However, due to the low-speed backpropagation method and high computing cost of the artificial neural network, the recognition efficiency and accuracy of the user's brain activity state are low, making it difficult to carry out large-scale clinical applications.

[0047] The brain activity state recognition method of the embodiment of the present invention reduces the processing overhead of EEG signal data and speeds up the calculation speed of EEG signal data by pulsing the artificial neural network model, so that the characteristic information of the EEG signal can be extracted more quickly, and thus the efficiency of brain activity state recognition can be effectively improved; further, through the pulse neural network model, that is, the more brain-like pulse neural network is applied to brain activity state recognition, the boundary information and target integration information of the brain activity state can be quickly and accurately determined by processing the EEG signal in a more biologically consistent manner, so that the brain activity state can be accurately determined based on the boundary information and target integration information of the brain activity state, so that the processing efficiency of the EEG signal is improved, and thus the efficiency and accuracy of brain activity recognition can be improved. In the embodiment of the present invention, the precise calculation advantage of the artificial neural network and the temporal information processing advantage of the pulse neural network are combined, so that the processing efficiency of the EEG signal is effectively improved, and the energy consumption is reduced while improving the recognition efficiency and accuracy of the brain activity state.

[0048] Combine the following Figure 1-Figure 6 The technical solution of the present invention is described in detail with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0049] Figure 1 FIG. 1 is a flow chart of an embodiment of a method for identifying brain activity states provided by an embodiment of the present invention. Figure 1 As shown, the method provided in this embodiment includes:

[0050] Step 101: Obtaining the user's EEG signal data;

[0051] Specifically, in order to accurately identify the user's brain activity state, the user's electroencephalogram signal data is first acquired in the embodiment of the present invention.

[0052] Step 102: acquiring characteristic information of the EEG signal according to the EEG signal data of the user and the target artificial neural network model; the target artificial neural network model includes spiking neurons;

[0053] Specifically, after obtaining the user's EEG signal data, in an embodiment of the present invention, characteristic information of the EEG signal is extracted through a target artificial neural network model; optionally, the target artificial neural network model includes pulse neurons; optionally, LIF (Leaky integrate and fire) neurons are used to replace artificial neurons using ReLU activation function in traditional Transformer, and the neuron discharge frequency and the output value of the ReLU activation function are made equivalent, thereby obtaining a target artificial neural network model including pulse neurons, that is, by pulse-izing the Transformer model and using the pulsed Transformer as a sparse coding layer, the network is made more sparse, and all EEG signal data in the time dimension are calculated through a self-attention mechanism, so that the entire operation process and the process of extracting EEG signal features can be parallelized, and the encoding of EEG signals is realized, and the characteristic information of EEG signals is extracted, thereby reducing the processing overhead of EEG signal data and speeding up the calculation speed of EEG signal data, and the characteristic information of EEG signals can be extracted more quickly, thereby further effectively improving the efficiency of brain activity state recognition.

[0054] Optionally, a LIF neuron is constructed using the following formula:

[0055]

[0056] Where V k，t represents the membrane potential of the kth neuron at time t, C m is the membrane potential constant, V th is the release threshold, V reset is the reset potential, S pike It is the discharge sign, I k is the input current.

[0057] Alternatively, the equivalence relationship between artificial neurons and spiking neurons is described as:

[0058]

[0059] Where R(t) is the firing rate of the spiking neuron, is the output of the ReLU activation function, R max is the maximum discharge rate allowed, V iSNN is the membrane potential of the pulse neuron, and t is the time step at the current moment.

[0060] Step 103, determining brain activity state boundary information and target integration information according to the characteristic information of the EEG signal and the target pulse neural network model; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; the brain activity state boundary information is used to represent the time information of the brain activity state switching; the target integration information represents the brain activity information between adjacent brain activity state boundaries;

[0061] Specifically, after acquiring the characteristic information of the EEG signal based on the target artificial neural network model, the implementation of the present invention determines the brain activity state boundary information and the target integration information according to the characteristic information of the EEG signal and the target pulse neural network model; that is, the sparse structure and asynchronous processing characteristics of the pulse neural network model can more effectively process the characteristic information of the EEG signal containing spatiotemporal information and reduce energy consumption. By applying a more brain-like pulse neural network to brain activity state recognition and processing EEG signals in a manner that is more in line with biological characteristics, the processing efficiency of EEG signals can be effectively improved, thereby also improving the efficiency and accuracy of brain activity recognition.

[0062] Optionally, brain activity state boundary information is used to represent the time information of brain activity state switching, that is, the time information of brain activity state changes; the target integration information represents the brain activity information between adjacent brain activity state boundaries, that is, the front and back boundaries of a brain activity state are determined according to the adjacent brain activity state boundary information, and the spectrum information between the two boundaries is integrated as a characteristic description of the brain activity state, thereby obtaining the boundaries of different brain activity states and the integration of brain activity characteristic information corresponding to the corresponding brain activity states.

[0063] Step 104: Identify the user's brain activity state based on the brain activity state boundary information and the target integration information.

[0064] Specifically, after determining the target integrated information corresponding to the brain activity boundary information and the spectral information between two adjacent boundaries based on the characteristic information of the EEG signal and the target pulse neural network model, that is, after obtaining the boundaries of different brain activity states and the integration of the brain activity characteristic information corresponding to the corresponding brain activity states, the user's brain activity state can be accurately determined based on the brain activity boundary information and the spectral information between adjacent boundaries.

[0065] Optionally, the following autoregressive method is used to classify and output the recognition result of the brain activity state:

[0066]

[0067] Among them, P AR(y|l) indicates that the decoded output sequence y is obtained based on the input sequence l, the input sequence l includes the integrated information of the brain activity boundary information and the spectrum information between adjacent brain activity boundaries, and the output sequence y represents the recognition result of the brain activity state. Optionally, the recognition result y of the brain activity state at time u depends on the integrated information of the brain activity boundary information and the spectrum information between adjacent boundaries in the input sequence l and the decoder output before time u. That is, after determining the brain activity state boundary information and the target integrated information, the recognition result of the brain activity state can be accurately output by regression, thereby improving the accuracy of the brain activity recognition result.

[0068] The method of the above embodiment reduces the processing overhead of EEG signal data and speeds up the calculation speed of EEG signal data by pulsing the artificial neural network model, so that the characteristic information of EEG signal can be extracted more quickly, and thus the efficiency of brain activity state recognition can be effectively improved; further, through the pulse neural network model, that is, applying a more brain-like pulse neural network to brain activity state recognition, the boundary information and target integration information of brain activity state can be quickly and accurately determined by processing EEG signals in a more biologically consistent manner, so that the brain activity state can be accurately determined based on the boundary information and target integration information of brain activity state, so that the processing efficiency of EEG signal is improved, and thus the efficiency and accuracy of brain activity recognition can be improved. In the embodiment of the present invention, the precise calculation advantage of artificial neural network and the timing information processing advantage of pulse neural network are combined, so that the processing efficiency of EEG signal is effectively improved, and the recognition efficiency and accuracy of brain activity state are improved while reducing energy consumption.

[0069] In one embodiment, acquiring characteristic information of the EEG signal according to the user's EEG signal data and the target artificial neural network model includes:

[0070] Encode the EEG signal data to obtain the Mel-frequency cepstrum data corresponding to the user's EEG signal data;

[0071] The Mel-frequency cepstrum data is input into the target artificial neural network model to obtain the characteristic information of the EEG signal.

[0072] Specifically, after obtaining the user's EEG signal, in an embodiment of the present invention, the EEG signal data is encoded to obtain the Mel-cephalogram data corresponding to the user's EEG signal data; optionally, the user's original EEG signal is subjected to a fast Fourier transform under a given time window T and step size Δt, and then the spectrum is passed through a Mel filter group to obtain a Mel-cephalogram, and finally a cepstrum analysis is performed to convert it into a Mel-cephalogram that contains both dimensional information and time information, thereby achieving a more accurate and effective characterization of the EEG signal, thereby more accurately extracting the features of the EEG signal and identifying the brain activity state; further, the Mel-cephalogram data is input into a target artificial neural network model containing spiking neurons, thereby reducing the processing overhead of the EEG signal data and accelerating the calculation speed of the EEG signal data, thereby combining the precise calculation advantages of the artificial neural network with the timing information processing advantages of the spiking neural network, so that the processing efficiency of the EEG signal is effectively improved, the feature information of the EEG signal can be extracted more quickly, and the brain activity state can be quickly and accurately identified.

[0073] The method of the above embodiment, by encoding the EEG signal data, obtains the Mel-cephalogram inverse spectrum data corresponding to the user's EEG signal data, and thus obtains the EEG signal information containing both dimensional information and time information, thereby achieving a more accurate and effective characterization of the EEG signal, thereby making it possible to more accurately extract the features of the EEG signal and identify the brain activity state; further, by inputting the Mel-cephalogram inverse spectrum data into a target artificial neural network model containing spiking neurons, the processing overhead of the EEG signal data is reduced and the calculation speed of the EEG signal data is accelerated, thereby combining the precise calculation advantage of the artificial neural network with the timing information processing advantage of the spiking neural network, so that the processing efficiency of the EEG signal is effectively improved, and the feature information of the EEG signal can be extracted more quickly, thereby achieving rapid and accurate identification of the brain activity state.

[0074] In one embodiment, the characteristic information of the EEG signal is input into the target spiking neural network model to determine the boundary information of the brain activity state and the target integration information, including:

[0075] Inputting characteristic information of the EEG signal into the target spiking neural network model to determine the membrane potential of neurons in the target spiking neural network model;

[0076] Determine the location of pulse firing based on the neuron's membrane potential;

[0077] Determine the boundary information of brain activity state based on the pulse emission position;

[0078] According to the brain activity state boundary information, target integration information between adjacent brain activity state boundaries is obtained.

[0079] Specifically, after acquiring the characteristic information of the EEG signal according to the user's EEG signal data and the target artificial neural network model, in the embodiment of the present invention, the characteristic information of the EEG signal is input into the target pulse neural network model to determine the boundary information of the brain activity state and the target integration information, so that the recognition of the brain activity state can be accurately realized based on the boundary information of the brain activity state and the target integration information. Optionally, the characteristic information of the acquired EEG signal is converted into current and input into the target pulse neural network model, the neuron accumulates current forward and then causes the membrane potential to change, so as to determine the pulse emission position (pulse emission time) and brain activity boundary information based on the membrane potential of the neuron; finally, the front and rear boundaries of a certain brain activity state are determined according to the adjacent pulse emission positions, and the spectrum information between the two emission positions is integrated as the characteristic description of the brain activity state.

[0080] Optionally, the acquired characteristic information of the EEG signal is converted into electric current based on the following method:

[0081] I=Transformer out ×W

[0082] Where I is the current sequence, Transformer out is the characteristic information of the EEG signal obtained based on the target artificial neural network model, which contains the high-dimensional features of the Mel-frequency inverse spectrum. The Transformer is trained through the back-propagation algorithm to make the high-dimensional features temporally separable. W is the mapping weight.

[0083] Optionally, target integration information between adjacent brain activity state boundaries is determined based on:

[0084]

[0085] Among them, i and j are the time positions of two adjacent pulses, and Integrated information is the information integrated in this time period, that is, the target integrated information between adjacent brain activity state boundaries.

[0086] Optionally, the pulse firing position is determined based on the membrane potential of the neuron, comprising:

[0087] When the membrane potential of the neuron is greater than a first threshold, pulse emission is performed and a pulse emission position is determined;

[0088] According to the pulse emission position, the boundary information of brain activity state is determined, including:

[0089] The pulse emission position is used as the boundary information of brain activity state.

[0090] Specifically, after the characteristic information of the acquired EEG signal is converted into electric current and input into the target pulse neural network model, the neuron accumulates current forward and then causes a change in membrane potential. When the membrane potential reaches the threshold, the neuron will emit a pulse as an event and determine the pulse emission position. This event means that a certain boundary of the brain activity state is located. That is, after determining the pulse emission position (the pulse emission position is the pulse emission moment), the pulse sending moment is used as the time information of the brain activity state switching, thereby realizing the accurate determination of the brain activity state boundary information, and effectively improving the accuracy of the brain activity state recognition results.

[0091] The method of the above embodiment, by applying a more brain-like pulse neural network model to brain activity state recognition, uses a more biologically consistent method to process EEG signals, thereby achieving accurate determination of brain activity state boundary information and target integration information, and effectively improving the recognition efficiency and accuracy of brain activity states.

[0092] In one embodiment, inputting characteristic information of the EEG signal into a target spiking neural network model to determine the membrane potential of neurons in the target spiking neural network model includes:

[0093] The membrane potential of the neuron in the target spiking neural network model is determined based on the following method:

[0094] V k,t =V k,t-1 +F(I k,t ,V k,t-1 ,…);

[0095] V k,t represents the membrane potential of the kth neuron at the tth time; V k,t-1 represents the membrane potential of the kth neuron at the t-1th moment; F(I k,t ,V k,t-1 ,…) represents the target dynamics model in the target spiking neural network model; I k,t The current sequence corresponding to the characteristic information of the EEG signal.

[0096] Specifically, in the embodiment of the present invention, the characteristic information of the acquired EEG signal is converted into electric current and input into the target pulse neural network model. The neuron accumulates the current forward and then causes the membrane potential to change, thereby determining the pulse emission position and brain activity boundary information based on the membrane potential of the neuron. Optionally, the membrane potential of the neuron in the target pulse neural network model is determined in the following manner:

[0097] V k,t =V k,t-1 +F(I k,t ,V k,t-1 ,…);

[0098] V k,t represents the membrane potential of the kth neuron at the tth time; V k,t-1 represents the membrane potential of the kth neuron at the t-1th moment; F(I k,t ,V k,t-1 ,…) represents the target dynamics model in the target spiking neural network model; I k,t The current sequence corresponding to the characteristic information of the EEG signal; that is, the membrane potential of the neuron can be determined quickly and accurately through the target pulse neural network model, and then when the membrane potential of the neuron is greater than the first threshold, the boundary information of the brain activity state can be accurately determined, thereby realizing accurate identification of the brain activity state.

[0099] The method of the above embodiment can quickly and accurately determine the membrane potential of neurons through the target pulse neural network model, and then when the membrane potential of neurons is greater than the first threshold, the boundary information of brain activity state can be accurately determined, thereby realizing accurate identification of brain activity state.

[0100] Optionally, the target dynamics model in the target pulse neural network model includes at least one of the following: a first-order dynamics model, a first-order dynamics with a dynamic threshold, a second-order dynamics model, and a second-order dynamics model with an autoinhibition term.

[0101] Specifically, the target pulse neural network model in the embodiment of the present invention includes four selectable target dynamic models: a first-order dynamic model, a first-order dynamic model with a dynamic threshold, a second-order dynamic model, and a second-order dynamic model with a self-inhibition term. Optionally, each dynamic model enables neurons to adopt different threshold accumulation and pulse emission strategies for the input of the same EEG signal feature information, thereby improving the learning ability of the network through node optimization; that is, by adding a rich target dynamic model to the pulse neural network, the dynamic complexity of a single node of the network is improved, thereby improving the computing power of the network, and at the same time helping to improve robustness, which is particularly suitable for EEG signals with large noise, thereby accurately determining the membrane potential of neurons and the boundary information of brain activity states, and improving the accuracy of brain activity state recognition. Optionally, compared with artificial neural networks, pulse neural networks have more complex neuron and synaptic structures. In the embodiment of the present invention, by adding a neuron dynamic model to the neural network, the network obtains more powerful computing power and adaptability, so that the membrane potential of neurons and the boundary information of brain activity states can be quickly and accurately determined, and the efficiency and accuracy of brain activity state recognition can be improved.

[0102] Optionally, four target dynamic models are established using the following formulas:

[0103]

[0104] V th，k =αV th，k -1+Spike·Δh

[0105]

[0106]

[0107] Among them, V th，k is the neuron threshold at time k, α is the threshold attenuation constant, Δh is the threshold jump constant, a is the second-order term coefficient, b is the first-order term coefficient, c is the constant term coefficient, g is the second-order term coefficient of the excitatory neuron, m is the first-order term coefficient of the excitatory neuron, and U k，t is the membrane potential of inhibitory neuron k at time t, and n is the membrane potential coefficient of inhibitory neuron. Optionally, the first-order dynamics is the dynamics of IF (integrate and fire) neurons, and all input currents are processed linearly with a constant threshold; the strategy for membrane potential accumulation of first-order dynamics with dynamic threshold is consistent with that of first-order dynamics, but threshold plasticity is adopted, so that when the neuron discharges sparsely, the threshold is lowered to promote pulse discharge, and when the discharge is dense, the threshold is raised to inhibit the discharge of pulses; the second-order dynamics adopts history-dependent nonlinear membrane potential processing, so that the accumulation speed of the current membrane potential is not only related to the input at the current moment, but also to the membrane potential at the historical moment, and a fixed discharge threshold is adopted; the second-order dynamics with self-inhibition terms treats neurons with second-order dynamics as excitatory neurons, and adds inhibitory neurons, that is, the input current is received through the excitatory neurons, and the activity of the excitatory neurons is inhibited through the inhibitory neurons, so as to balance the states of the two, so that the discharge frequency of the excitatory neurons reaches a reasonable range.

[0108] The method of the above embodiment improves the computing power of the network by adding a rich target dynamics model to the pulse neural network, and at the same time helps to improve the robustness. It is particularly suitable for EEG signals with high noise, so as to accurately determine the membrane potential of neurons and the boundary information of brain activity states, and improve the efficiency and accuracy of brain activity state recognition.

[0109] For example, Figure 2The schematic diagram of brain activity state recognition shown in the figure encodes the EEG signal data into a Mel-frequency inverse spectrum, uses the target artificial neural network model (pulsed Transformer module) as a sparse coding layer to encode the EEG signal data, and uses the target pulse neural network model containing multiple neuron dynamics models to process the characteristic information of the EEG signal output by the pulse Transformer, thereby performing state segmentation in the time dimension of the Mel-frequency inverse spectrum, and obtaining the boundaries of different brain activity states and the characteristic integration of the corresponding states, and finally uses the autoregressive decoder to process the brain activity state boundary information and target integration information output by the target pulse neural network model to obtain the recognition result of the brain activity state. In the embodiment of the present invention, the hybrid neural network with sparse connection composed of artificial neural network and pulse neural network combined with neuron dynamics has stronger time series information processing capability, higher classification accuracy, stronger robustness, and lower computing energy consumption, and realizes a processing and recognition method of brain activity state that is more in line with biological characteristics, which effectively improves the efficiency and accuracy of brain activity state recognition.

[0110] For example, Figure 3 The schematic diagram of brain activity state recognition shown in the figure encodes the EEG signal data into a Mel-frequency inverse spectrum, encodes the input data through a pulsed Transformer module as a sparse coding layer, extracts the characteristic information of the EEG signal, maps the characteristic information of the EEG signal into a current sequence and inputs it into a pulse neuron model, triggers membrane potential accumulation and pulse sending position, determines the brain activity boundary information according to the pulse emission position, and integrates the information between two adjacent pulse positions into the characteristic representation of the corresponding state. Finally, the regression decoding model outputs the classification result of the brain activity state to achieve accurate recognition of the brain activity state.

[0111] For example, Figure 4 The schematic diagram of brain activity state recognition shown in the figure combines the precise calculation advantage of artificial neural network with the time series information processing advantage of pulse neural network, so that the processing efficiency of EEG signal is effectively improved, and the efficiency and accuracy of brain activity recognition are improved, so as to reduce energy consumption, improve efficiency and improve the recognition accuracy of brain activity state. Optionally, by pulsing the artificial neural network model, the processing overhead of EEG signal data is reduced and the calculation speed of EEG signal data is accelerated, so that the characteristic information of EEG signal can be extracted more quickly, and the efficiency of brain activity state recognition is improved; by adding rich target dynamics models to the pulse neural network, the dynamic complexity of a single node of the network is improved, so as to improve the computing power of the network, and at the same time help to improve the robustness, which is particularly suitable for EEG signals with large noise, so as to accurately determine the membrane potential of neurons and the boundary information of brain activity state, and improve the efficiency and accuracy of brain activity state recognition.

[0112] Exemplarily, in the embodiment of the present invention, the LJSpeech data set is selected for experimental verification, and the phoneme error rate (PER) is used as a metric for comparison with other methods, where the phoneme error rate is defined as the number of incorrectly recognized phonemes divided by the total number of phonemes in the sample. The average PER obtained by the method of the present invention is 9.31%, and the PER obtained by other methods is 11.7%. It can be seen that the brain activity state recognition method in the embodiment of the present invention has a higher accuracy advantage.

[0113] The brain activity state recognition device provided by the present invention is described below. The brain activity state recognition device described below and the brain activity state recognition method described above can be referenced to each other.

[0114] Figure 5 : is a schematic diagram of the structure of the brain activity state recognition device provided by the present invention. The brain activity state recognition device provided in this embodiment includes:

[0115] An acquisition module 710 is used to acquire the user's EEG signal data;

[0116] The processing module 720 is used to obtain characteristic information of the EEG signal according to the EEG signal data of the user and the target artificial neural network model; the target artificial neural network model includes spiking neurons;

[0117] The determination module 730 is used to determine the brain activity state boundary information and the target integration information according to the characteristic information of the EEG signal and the target pulse neural network model; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; the brain activity state boundary information is used to represent the time information of the brain activity state switching; the target integration information represents the brain activity information between adjacent brain activity state boundaries;

[0118] The identification module 740 is used to identify the user's brain activity state according to the brain activity state boundary information and the target integration information.

[0119] Optionally, the processing module 720 is specifically used to: encode the EEG signal data to obtain Mel-frequency cepstrum data corresponding to the user's EEG signal data;

[0120] The Mel-frequency cepstrum data is input into the target artificial neural network model to obtain the characteristic information of the EEG signal.

[0121] Optionally, the determination module 730 is specifically used to: input feature information of the EEG signal into the target spiking neural network model to determine the membrane potential of neurons in the target spiking neural network model;

[0122] Determine the location of pulse firing based on the neuron's membrane potential;

[0123] Determine the boundary information of brain activity state based on the pulse emission position;

[0124] According to the brain activity state boundary information, target integration information between adjacent brain activity state boundaries is obtained.

[0125] Optionally, the determination module 730 is specifically used to determine the membrane potential of a neuron in the target pulse neural network model based on the following method:

[0126] V k,t =V k,t-1 +F(I k,t ,V k,t-1 ,…);

[0127] V k,t represents the membrane potential of the kth neuron at the tth time; V k,t-1 represents the membrane potential of the kth neuron at the t-1th moment; F(I k,t ,V k,t-1 ,…) represents the target dynamics model in the target spiking neural network model; I k,t The current sequence corresponding to the characteristic information of the EEG signal.

[0128] Optionally, the target dynamics model in the target pulse neural network model includes at least one of the following: a first-order dynamics model, a first-order dynamics with a dynamic threshold, a second-order dynamics model, and a second-order dynamics model with an autoinhibition term.

[0129] Optionally, the determination module 730 is specifically configured to: when the membrane potential of the neuron is greater than a first threshold, perform pulse emission and determine the pulse emission position;

[0130] The pulse emission position is used as the boundary information of brain activity state.

[0131] The device of the embodiment of the present invention is used to execute the method in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0132] Figure 6The physical structure diagram of an electronic device is illustrated, and the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the brain activity state recognition method, which includes: obtaining the user's EEG signal data; obtaining the characteristic information of the EEG signal according to the user's EEG signal data and the target artificial neural network model; the target artificial neural network model includes a pulse neuron; determining the brain activity state boundary information and the target integration information according to the characteristic information of the EEG signal and the target pulse neural network model; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; the brain activity state boundary information is used to indicate the time information of the brain activity state switching; the target integration information indicates the brain activity information between adjacent brain activity state boundaries; and identifying the user's brain activity state according to the brain activity state boundary information and the target integration information.

[0133] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0134] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the brain activity state identification method provided by the above-mentioned methods, and the method includes: obtaining the user's EEG signal data; obtaining characteristic information of the EEG signal based on the user's EEG signal data and a target artificial neural network model; the target artificial neural network model includes pulse neurons; determining brain activity state boundary information and target integration information based on the characteristic information of the EEG signal and the target pulse neural network model; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; the brain activity state boundary information is used to represent the time information of brain activity state switching; the target integration information represents the brain activity information between adjacent brain activity state boundaries; and identifying the user's brain activity state based on the brain activity state boundary information and the target integration information.

[0135] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned brain activity state identification methods, the methods comprising: obtaining the user's EEG signal data; obtaining characteristic information of the EEG signal based on the user's EEG signal data and a target artificial neural network model; the target artificial neural network model comprises pulse neurons; determining brain activity state boundary information and target integration information based on the characteristic information of the EEG signal and the target pulse neural network model; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; the brain activity state boundary information is used to represent the time information of the brain activity state switching; the target integration information represents the brain activity information between adjacent brain activity state boundaries; and identifying the user's brain activity state based on the brain activity state boundary information and the target integration information.

[0136] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0137] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying brain activity states, characterized in that: include: Obtain the user's EEG signal data; Acquiring characteristic information of the EEG signal according to the EEG signal data of the user and the target artificial neural network model; The target artificial neural network model includes spiking neurons; Determining brain activity state boundary information and target integration information according to the characteristic information of the EEG signal and the target pulse neural network model; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; The brain activity state boundary information is used to represent the time information of the brain activity state switching; The target integrated information represents brain activity information between adjacent brain activity state boundaries; Determining brain activity state boundary information and target integration information based on the characteristic information of the EEG signal and the target spiking neural network model includes: Inputting characteristic information of the EEG signal into a target spiking neural network model to determine the membrane potential of neurons in the target spiking neural network model; Determining a pulse firing position based on a membrane potential of the neuron; Determining brain activity state boundary information according to the pulse emission position; According to the brain activity state boundary information, obtaining target integration information between adjacent brain activity state boundaries; The brain activity state of the user is identified according to the brain activity state boundary information and the target integration information.

2. The method for identifying brain activity states according to claim 1, characterized in that: The step of obtaining characteristic information of the EEG signal according to the EEG signal data of the user and the target artificial neural network model includes: Encoding the EEG signal data to obtain Mel-frequency cepstrum data corresponding to the EEG signal data of the user; The Mel-frequency cepstrum data is input into a target artificial neural network model to obtain characteristic information of the EEG signal.

3. The method for identifying brain activity states according to claim 1, characterized in that: The step of inputting the characteristic information of the EEG signal into a target spiking neural network model to determine the membrane potential of neurons in the target spiking neural network model comprises: The membrane potential of the neuron in the target spiking neural network model is determined based on the following method: V k,t =V k,t-1 +F(I k,t ,V k,t-1 ,…); Among them, V k,t represents the membrane potential of the kth neuron at the tth time; V k,t-1 represents the membrane potential of the kth neuron at the t-1th moment; F(I k,t ,V k,t-1 ,…) represents the target dynamics model in the target spiking neural network model; I k,t The current sequence corresponding to the characteristic information of the EEG signal.

4. The method for identifying brain activity states according to claim 3, characterized in that: The target dynamics model in the target pulse neural network model includes at least one of the following: a first-order dynamics model, a first-order dynamics model with a dynamic threshold, a second-order dynamics model, and a second-order dynamics model with a self-inhibition term.

5. The method for identifying brain activity states according to any one of claims 1 to 4, characterized in that: The step of determining the pulse emission position based on the membrane potential of the neuron comprises: When the membrane potential of the neuron is greater than a first threshold, pulse emission is performed and a pulse emission position is determined; Determining brain activity state boundary information according to the pulse emission position includes: The pulse emission position is used as brain activity state boundary information.

6. A brain activity state recognition device, characterized in that: include: An acquisition module, used to acquire the user's EEG signal data; A processing module, used for acquiring characteristic information of the EEG signal according to the EEG signal data of the user and a target artificial neural network model; the target artificial neural network model includes spiking neurons; A determination module, used to determine brain activity state boundary information and target integration information according to the characteristic information of the EEG signal and the target pulse neural network model; the target pulse neural network model is used to determine the pulse emission position based on the characteristic information of the EEG signal; The brain activity state boundary information is used to represent the time information of the brain activity state switching; The target integrated information represents brain activity information between adjacent brain activity state boundaries; Determining brain activity state boundary information and target integration information based on the characteristic information of the EEG signal and the target spiking neural network model includes: Inputting characteristic information of the EEG signal into a target spiking neural network model to determine the membrane potential of neurons in the target spiking neural network model; Determining a pulse firing position based on a membrane potential of the neuron; Determining brain activity state boundary information according to the pulse emission position; According to the brain activity state boundary information, obtaining target integration information between adjacent brain activity state boundaries; The identification module is used to identify the user's brain activity state according to the brain activity state boundary information and the target integration information.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the brain activity state recognition method as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the brain activity state recognition method as described in any one of claims 1 to 5 is implemented.

9. A computer program product having executable instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor implements the brain activity state recognition method as described in any one of claims 1 to 5.

Images