Human electroencephalogram signal identification method based on pulse neural network and related equipment

Abstract

The invention discloses a human electroencephalogram signal identification method based on a pulse neural network and related equipment, and relates to the technical field of human electroencephalogram signal processing. The method comprises the following steps: acquiring an electroencephalogram signal of a target person; inputting the electroencephalogram signal of the target person into a spiking neural network, and obtaining an identification result output by the spiking neural network; wherein the recognition result is obtained by decoding the spiking neural network based on a target pulse signal, the target pulse signal is generated by continuously stimulating neurons in the spiking neural network by synaptic current corresponding to the electroencephalogram signal of the target person, and the spiking neural network is obtained through training. By implementing the technical scheme provided by the invention, the calculated amount and energy consumption of human electroencephalogram signal processing can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of human electroencephalogram (EEG) signal processing, and in particular to a method for recognizing human electroencephalogram (EEG) signals based on a pulse neural network and related equipment. Background Art

[0002] With the rapid development of neuroscience and artificial intelligence technology, human-computer interaction systems based on human electroencephalography (EEG) signals have gradually become a research hotspot. Such systems can recognize and predict human intentions by analyzing and interpreting human EEG signals, and have broad application prospects in medical diagnosis, rehabilitation training, intelligent control and other fields. However, accurately identifying and interpreting complex EEG signals has always been a major challenge in this field.

[0003] At present, the common human EEG signal recognition method mainly relies on traditional deep neural networks. Usually, the human EEG signal is first preprocessed and feature extracted, and then the deep neural network is used to identify it based on the extracted features. However, traditional deep neural networks need to perform a large number of continuous operations regardless of the characteristics of the input human EEG signal, which leads to a large amount of calculation in processing human EEG signals, resulting in high energy consumption of the device, and is not suitable for real-time processing of human EEG signals on portable or low-power devices. Summary of the invention

[0004] The present invention provides a method for recognizing human electroencephalogram signals based on a pulse neural network and related equipment, which can reduce the amount of calculation and energy consumption of human electroencephalogram signal processing.

[0005] In a first aspect of the present invention, a method for recognizing human electroencephalogram signals based on a spiking neural network is provided, comprising: Obtaining the target person's EEG signal; Inputting the target person's electroencephalogram signal into a spiking neural network to obtain a recognition result output by the spiking neural network; Among them, the recognition result is obtained by decoding the pulse neural network based on the target pulse signal, and the target pulse signal is generated by the synaptic current corresponding to the target person's electroencephalogram signal continuously stimulating the neurons in the pulse neural network, and the pulse neural network is obtained through training.

[0006] In a second aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for recognizing human electroencephalogram signals based on a pulse neural network as described above is implemented.

[0007] In a third aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for recognizing human electroencephalogram signals based on a pulse neural network as described above is implemented.

[0008] In a fourth aspect of the present invention, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements any of the above-described methods for recognizing human electroencephalogram signals based on a pulse neural network.

[0009] In summary, one or more technical solutions provided in the present invention have at least the following technical effects or advantages: By inputting the target person's EEG signal into a trained spiking neural network, the synaptic current corresponding to the target person's EEG signal is used to continuously stimulate the neurons in the spiking neural network to generate a target pulse signal, and the recognition result is obtained based on the decoding of the target pulse signal. This method makes full use of the event-driven characteristics of the spiking neural network, and only performs calculations when neurons are stimulated to generate pulses, avoiding the problem of traditional deep neural networks that require a large number of continuous calculations regardless of the characteristics of the input signal.

[0010] Therefore, the present invention significantly reduces the amount of computation and energy consumption of human EEG signal processing, making it more suitable for real-time processing of human EEG signals on portable or low-power devices. At the same time, the working mechanism of the spiking neural network is closer to the biological nervous system, and can better capture and process the temporal dynamic features in human EEG signals, thereby improving the accuracy and efficiency of recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 It is a flow chart of a method for recognizing human electroencephalogram signals based on a pulse neural network provided in an embodiment of the present invention.

[0013] Figure 2 It is a flow chart of a pulse neural network training method provided by an embodiment of the present invention.

[0014] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Please refer to Figure 1 , Figure 1 The present invention provides a flowchart of a method for recognizing human EEG signals based on a pulse neural network, which can be implemented by a computer program, a single-chip microcomputer, or a human EEG signal recognition system based on a pulse neural network based on a von Neumann system. The computer program can be integrated into an application or run as an independent tool application. Specifically, the method may include the following steps: S101, obtaining an electroencephalogram signal of a target person.

[0017] The target person's EEG signal refers to the record of brain electrical activity from a specific target object to be identified. It is an electrophysiological signal that reflects the activity of a group of neurons in the brain, usually collected through an electrode array placed on the scalp. The target person's EEG signal can be understood as a set of multi-channel time series data, where each channel corresponds to an electrode at a specific location on the scalp, recording the potential difference at that location over time.

[0018] Furthermore, the target person's EEG signal is used to provide input data for the spiking neural network, which is the basis of the entire recognition process. It carries the target object's brain activity information in a specific time period, which may reflect a specific intention, cognitive state or emotional change. By analyzing and identifying these signals, the present invention aims to achieve accurate interpretation of human intentions or brain states, and provide key information input for various brain-computer interface-based applications.

[0019] For example, in a specific application scenario, the target person's EEG signals may come from a user who is trying to control an external device through his mind. These signals may contain the brain activity pattern of the user wanting to perform a specific action (such as moving the cursor to the left). By accurately identifying and decoding these signals, the system can convert the user's intentions into actual control commands, thereby realizing human-computer interaction based on EEG.

[0020] Furthermore, in order to more accurately capture and process the dynamic characteristics of human EEG signals, thereby providing higher quality input data for subsequent signal recognition processes. Based on the above embodiment, as an optional embodiment, in S101: the process of obtaining the target human EEG signal may also include the following steps: S201, obtaining n first time blocks obtained by integrating the electroencephalogram sequence of the target object collected by the sensor.

[0021] Specifically, the above-mentioned sensors can be multiple electrodes in a specialized EEG acquisition device, which are usually placed on the scalp of the target subject, covering different functional areas of the brain. The commonly used electrode placement scheme follows the international 10-20 system to ensure the standardization and comparability of data acquisition. During the acquisition process, these electrodes continuously record the potential changes in different areas of the brain to form multi-channel time series data. The sampling frequency is usually set between 250Hz and 1000Hz to ensure that the high-frequency components of brain activity are captured.

[0022] Since EEG signals are continuous time series data, directly processing these data may lead to complex calculations and analysis. Therefore, dividing the continuous EEG signal into n first time blocks of fixed length can help the system capture the dynamic changes of the signal on a local time scale.

[0023] Among them, the first time block refers to dividing the continuously collected human EEG sequence into data segments of fixed length. The above fixed length is an adjustable parameter, which is usually selected according to the specific application scenario and EEG signal characteristics, and the typical value is between 0.5 seconds and 2 seconds. Each first time block can be understood as an N×T multidimensional data matrix, where N represents the number of EEG electrodes and T is the number of sampling points within ν seconds, which depends on the sampling frequency.

[0024] Specifically, the design of the first time block is to achieve local analysis of EEG signals. By dividing the continuous signal into segments of fixed length, each first time block represents the EEG activity within a local time range. This method enables the system to analyze the changes of the signal over time more finely, especially to capture short-term changes in EEG activity, such as sudden changes in motor intention. At the same time, this block strategy also greatly reduces the complexity of processing large-scale data, because each time block can be regarded as an independent analysis unit.

[0025] S202: Determine mutation points in the n first time blocks according to the accumulated deviations between the frequency quantities in the n first time blocks.

[0026] Specifically, due to the high dynamics of EEG signals, there may be significant state changes at different time points, especially in tasks of mind control or motor intention. These changes may correspond to different brain activity states or intention transitions. In order to accurately capture significant changes in EEG signals, it is necessary to identify mutation points in EEG signals.

[0027] Among them, the mutation point refers to the time point or time period when a significant state change occurs in the EEG signal, marking a sharp shift in the pattern of EEG activity. These changes usually reflect significant changes in cognitive state, motor intention or emotional state. The mutation point can be understood as a special position in the EEG signal time series. Before and after this position, the statistical characteristics of the signal (such as frequency distribution, amplitude or phase) have changed significantly, beyond the range of normal fluctuations, indicating that a qualitative change has occurred in brain activity.

[0028] In addition, the identification of mutation points enables the system to dynamically adjust the time window of analysis to ensure that each analysis unit contains complete state information. This dynamic adjustment mechanism significantly improves the accuracy of signal processing and helps to distinguish between real changes in EEG activity and random noise. In real-time control applications, the detected mutation points can trigger a rapid response of the system, enabling timely capture and response to changes in intent, which is of great significance for building efficient and reliable brain-computer interface systems.

[0029] Optionally, the embodiment of the present invention adopts a test statistic method based on cumulative sum (CUSUM) to realize mutation point detection. CUSUM is a classic and effective technique, which is particularly suitable for detecting mutations in time series. Its core principle is to discover changes by calculating the cumulative deviation of the signal.

[0030] Specifically, it is necessary to apply the CUSUM test to the spectrum quantity in each time block and frequency band. The CUSUM statistics are accumulated over time. When the accumulated value of the statistics deviates significantly from the reference value, it indicates that there may be a mutation point.

[0031] Because the cumulative sum statistics of CUSUM will gradually accumulate over time, if it is not restricted, too many unnecessary or false mutation points may be detected. In order to improve the accuracy and reliability of detection, the embodiment of the present invention introduces the concept of thresholded CUSUM. By setting a predefined threshold, only when the CUSUM statistic exceeds the threshold, the time point is considered as a candidate mutation point. The above thresholding method significantly improves the accuracy and sensitivity of mutation point detection and effectively avoids false detection caused by noise interference.

[0032] However, relying solely on thresholded CUSUM may not be enough to completely avoid false detections. To this end, the embodiment of the present invention also introduces a further evaluation and confirmation mechanism. Around the candidate mutation points, the system will further evaluate the CUSUM statistics to confirm the authenticity of these mutation points. Thereby reducing false mutation points caused by noise or other non-brain electrical activity interference. Only after further verification, it is confirmed that the change in the CUSUM statistic does represent the real change of the signal, and the time point will be finally marked as the actual mutation point.

[0033] S203. Integrate n first time blocks into m second time blocks based on the mutation point to obtain an electroencephalogram signal of the target person, wherein n and m are positive integers, and m is less than or equal to n.

[0034] Specifically, in order to optimize the temporal structure of EEG signals so that they can better reflect the real dynamic changes of brain activity, the first time block is integrated based on the mutation point to obtain a set of more accurate and meaningful time units, providing a more reliable data basis for subsequent feature extraction and intention recognition.

[0035] Specifically, the mutation point can be used as a dividing line to reorganize the originally fixed-length n first time blocks into m variable-length second time blocks. Here, n and m are both positive integers, and m is less than or equal to n, which means that the number of time blocks after integration will not increase, but usually decrease.

[0036] Optionally, the integration process follows the following principle: all first time blocks between two adjacent mutation points are merged into a second time block; if the interval between two mutation points is less than a predefined minimum time block length, multiple such small intervals may be merged into a larger second time block.

[0037] By adopting the above technical solution, the state changes in EEG signals can be captured more accurately. The traditional fixed-length segmentation method may artificially cut a complete state change process, while the integration based on mutation points can ensure that each second time block contains a relatively complete and consistent EEG activity pattern. Secondly, the above method improves the signal-to-noise ratio of the data. Since each second time block corresponds to a relatively stable brain state, the noise interference in the state transition process is reduced, making the signal characteristics in each time block purer and more significant.

[0038] In addition, the above-mentioned dynamic time block integration strategy enhances the system's adaptability to EEG activities of different time scales. Some brain states may last for a short time, while others may last for a long time. By allowing the length of the second time block to change dynamically according to the actual EEG activity, the system can more flexibly capture brain states of different durations, avoiding the information loss or redundancy that may be caused by a fixed time window.

[0039] The m second time blocks obtained after integration constitute the final target person's EEG signal. The target person's EEG signal not only retains the key information of the original EEG data, but also enhances the significance of its time structure through intelligent segmentation. Each second time block now represents a relatively independent and complete brain state or intention stage, which provides an ideal data unit for subsequent feature extraction and classification tasks.

[0040] Furthermore, since the EEG signal contains not only the activity information of a single channel, but also the interactions between different brain regions. In order to further extract and analyze the complex information contained in the EEG signal, the embodiment of the present invention introduces coherence analysis related technology. By performing coherence analysis on the EEG signal, these interactions can be captured, providing a richer information basis for subsequent feature extraction and classification tasks. Especially in complex tasks such as mind control, the coordinated activities of different brain regions can often better reflect the user's true intentions than the activities of a single region.

[0041] Based on the above embodiment, as an optional embodiment, after S201, obtaining n first time blocks obtained by integrating the human EEG sequence of the target object collected by the sensor, the following steps may also be included: S301 : Determine coherence values ​​of n first time blocks according to a ratio between auto-spectra and cross-spectra corresponding to the n first time blocks.

[0042] Specifically, the Welch periodogram method is first applied to each first time block to estimate the self-spectrum and cross-spectrum. As an improved power spectral density (PSD) estimation technique, the Welch method further divides the time block into multiple overlapping small segments, and calculates the periodogram after applying a window function (such as a Hanning window) to each segment, and finally averages these periodograms to obtain the power spectrum estimate of the entire time block. The advantage of the above method is that it can significantly reduce random fluctuations and improve the stability and reliability of spectrum estimation.

[0043] Based on the Welch method, the autospectrum and cross-spectrum are calculated separately. The autospectrum reflects the power distribution of a single EEG channel and provides important information for analyzing the activity intensity of a specific brain area. The cross-spectrum is used to evaluate the relationship between two different EEG channels, which helps to capture the information exchange and coordinated activities between different brain areas. By comparing the autospectrum and the cross-spectrum, a coherence value can be obtained, which is a value ranging from 0 to 1, which is used to quantify the degree of synchronization between different EEG channels.

[0044] In order to further improve the reliability of coherence estimation, the embodiment of the present invention adopts an average strategy of multiple estimates. Specifically, for each first time block, multiple overlapping short time period analyses are performed, each analysis obtains a coherence estimate, and finally the average value of these estimates is taken as the final result. This method effectively reduces the variance of the estimate and improves the stability of the result.

[0045] S302: Perform standardization processing on n first time blocks based on coherence values.

[0046] Specifically, in order to make the comparison between different spectra and coherence more consistent and reliable, the embodiment of the present invention also introduces Fisher-z transformation to standardize the coherence value. The core idea of ​​Fisher-z transformation is to convert the coherence value into data with approximate normal distribution.

[0047] Through the above technical solution, the embodiment of the present invention significantly enhances the ability to describe the spatiotemporal characteristics of EEG signals. Coherence analysis not only provides a method for quantifying the strength of functional connections between different brain regions, but also enables the system to capture more subtle changes in brain activity patterns.

[0048] S102. Input the target person's electroencephalogram signal into a spiking neural network to obtain a recognition result output by the spiking neural network, wherein the recognition result is obtained by decoding the spiking neural network based on the target pulse signal, the target pulse signal is generated by the synaptic current corresponding to the target person's electroencephalogram signal continuously stimulating the neurons in the spiking neural network, and the spiking neural network is obtained through training.

[0049] Among them, the spiking neural network (SNN) refers to an artificial neural network model that imitates the working principle of the biological nervous system. In the embodiment of the present invention, it can be understood as a computing model that uses discrete pulse signals as information carriers and processes and transmits information by simulating the dynamic process of biological neurons. SNN is used to convert complex EEG signals into interpretable control instructions to achieve efficient and accurate mind control.

[0050] Compared with traditional deep learning neural networks, SNNs have the following significant features and advantages: First, in terms of information encoding, deep learning networks usually use continuous numerical values ​​to represent the activation state of neurons, while SNNs use discrete pulse sequences. This pulse encoding is closer to the working mode of biological nervous systems and can more naturally process time series data, such as electroencephalogram signals. Second, in terms of computational efficiency, the event-driven nature of SNNs means that calculations are performed only when neurons emit pulses, which is in sharp contrast to the continuous matrix operations in deep learning networks, greatly reducing energy consumption and is particularly suitable for implementing real-time brain-computer interfaces on low-power devices.

[0051] In addition, SNN has a natural advantage in processing temporal dynamic information. Deep learning networks usually require additional structures to handle temporal dependencies, while SNN can directly integrate temporal information in its dynamics and better capture instantaneous changes and long-term dependencies in EEG signals. This feature enables SNN to perform well in processing highly dynamic EEG signals and to more accurately identify and interpret complex brain activity patterns.

[0052] Specifically, after acquiring the target person's EEG signal, the next step is to input it into the spiking neural network to obtain the recognition result. When the target person's EEG signal is input into the SNN, it is first converted into a synaptic current. This conversion process simulates the mechanism by which biological neurons receive external stimuli. As a continuous source of stimulation, synaptic current continuously acts on neurons in the network. In the biological nervous system, neurons accumulate the received current, and when the accumulated potential exceeds a certain threshold, the neuron will emit an action potential, that is, generate a target pulse signal.

[0053] Unlike traditional deep neural networks, SNNs only perform calculations when neurons fire spikes. This means that the network's computing resources are effectively allocated to the information that really needs to be processed, rather than requiring a large number of continuous matrix operations regardless of the characteristics of the input signal, as in traditional networks. This computing mode greatly reduces the amount of computation and energy consumption of the processing process, making the system particularly suitable for real-time EEG signal processing on portable or low-power devices.

[0054] In addition, the working mechanism of the spiking neural network is closer to the biological nervous system. This similarity enables the network to better process the complex temporal dynamic features in EEG signals. In the biological nervous system, the encoding and transmission of information often depends on the temporal pattern of neuron firing, not just the firing frequency. By simulating this temporal encoding mechanism, the spiking neural network can capture subtle temporal features that traditional deep neural networks may ignore, thereby improving the accuracy and efficiency of recognition.

[0055] Based on the above embodiment, as an optional embodiment, in S102: the step of inputting the target person's EEG signal into the spiking neural network to obtain the recognition result output by the spiking neural network may further include the following steps: S401. Input the target person's EEG signal into a spiking neural network, and convert the target person's EEG signal into a synaptic current based on the spatial features in the target person's EEG signal through the spiking neural network, wherein the spatial features are used to characterize the dependency relationship between different brain regions.

[0056] Specifically, EEG signals are collected from different areas of the brain through multiple electrodes. Each electrode corresponds to a spatial location, and the collected signal represents the brain activity at that location. In order to maintain the spatial dependence of these channels, the signal can be mapped into a two-dimensional matrix. The benefit of this mapping is that it maintains the topological structure of the electrodes in physical space, so that subsequent convolution operations can capture the spatial relationship between electrodes.

[0057] Specifically, the multi-channel EEG signals are mapped to a two-dimensional matrix, maintaining the topological structure of the electrodes on the scalp. For example, if the international 10-20 system is used for electrode placement, the signals collected by these electrodes will be mapped into a two-dimensional matrix, so that the signals of adjacent electrodes also maintain a neighboring relationship in the matrix. This mapping method lays the foundation for subsequent spatial feature extraction, allowing the network to capture the spatial correlation between electrodes.

[0058] Next, the mapped two-dimensional EEG signal is input into a convolutional pulse coding layer. The design of this layer cleverly combines convolution operation and pulse coding to achieve automatic conversion from continuous EEG signals to discrete pulse signals. The essence of the convolution operation is to extract features by sliding the kernel function in a local area, which is just suitable for capturing local spatial dependencies in EEG signals. For example, a motor imagery task may activate adjacent motor cortical areas, and this local activity pattern can be effectively captured by the convolution operation.

[0059] Exemplarily, the postsynaptic potential function of the convolutional layer can be expressed as: ; In the formula, Represents the postsynaptic potential of the neuron at position (i, j) in the convolutional layer, that is, the output of the neuron. This neuron belongs to the convolution channel f of the kth layer. represents the pulse signal of the cth channel of the k-1th layer at time step t, at the position (i-m, j-n), represents the weight of the convolution kernel, which represents the convolution kernel weight connecting the cth channel of the k-1th layer to the fth channel of the kth layer. The size of the convolution kernel is M×N, that is, it performs weighted calculation on the local area of ​​M×N. Represents the bias term of the convolution kernel, which is used to adjust the result of the convolution.

[0060] The above operations enable the network to extract more complex spatial features, such as coordinated activity patterns between different brain regions. For example, in complex cognitive tasks, the prefrontal and parietal lobes may show synchronous activity, and this cross-regional correlation can be captured by multi-channel convolution operations.

[0061] In order to further enhance the network's feature extraction capabilities and computational efficiency, an average pooling layer is inserted between the convolutional layers. The pooling operation not only reduces the computational burden of subsequent layers, but also enhances the network's robustness to small changes in the EEG signal. This is especially important because EEG signals are susceptible to various noises and interferences, and the translation invariance characteristics of the pooling layer can offset these interferences to a certain extent.

[0062] The synaptic currents generated by the above method not only contain the temporal information of the original EEG signal, but also encode complex spatial dependencies. For example, if the prefrontal and parietal regions of the brain show strong coordinated activity when performing a task, the synaptic currents corresponding to these regions may show similar pulse patterns or intensity changes.

[0063] In addition, brain activity is not only spatially complex, but also temporally continuous and dependent. Therefore, it is not enough to only consider the spatial features of the current moment, because brain activity is a continuous process, and the activity at the current moment is often affected by the previous state. Therefore, the embodiment of the present invention also proposes to capture the temporal features while extracting the spatial features, so that the information contained in the EEG signal can be interpreted more accurately.

[0064] Based on the above embodiment, as an optional embodiment, the step of converting the target person's EEG signal into a synaptic current by using a spiking neural network based on the spatial features in the target person's EEG signal in S401 may further include the following steps: S501: Extracting the spatial features of the target person's EEG signal at the current moment through a spiking neural network.

[0065] S502: Integrate the spatial features of the current moment with the spatial features of the previous moment to obtain spatiotemporal features.

[0066] Specifically, the above integration is mainly achieved through the postsynaptic potential function of the recurrent layer, where the recurrent layer (also called the loop layer) can capture more complex time dependencies, especially nonlinear relationships at larger time scales. The recurrent layer further integrates time information by taking the neuron state at the previous moment as the input at the current moment.

[0067] Exemplarily, the postsynaptic potential function of the recurrent layer can be expressed as: ; In the formula, represents the weight matrix of the feedforward connection in the recurrent layer, which acts on the input pulse at the current time step t , represents the bias term of the feedforward connection, represents the weight matrix of the recursive connection, which converts the previous time step Pulse Integrate into the current time step, Represents the bias term of the recursive connection.

[0068] Among them, the weight matrix of the recursive connection and the bias term are trainable parameters. By optimizing these parameters through gradient descent, the network is able to automatically learn how to best combine current and historical information. This self-learning mechanism enables the model to adapt to different types of EEG signal patterns and handle a variety of complex time dependencies without human intervention.

[0069] By adopting the above-mentioned spatiotemporal feature integration method, the model's ability to understand continuous EEG signals can be improved, enabling the system to capture complex brain activity patterns, such as continuous thinking processes or gradually changing cognitive states. Secondly, by considering temporal continuity, this method enhances the model's robustness to noise and short-term fluctuations, because it not only relies on information at a single moment, but also comprehensively considers activity patterns over a period of time. In addition, the above method also improves the model's performance in dealing with long-term dependent tasks, such as identifying continuous intentions or complex cognitive processes.

[0070] S503: Convert the target person's EEG signal into synaptic current based on the spatiotemporal characteristics.

[0071] Exemplarily, the generation formula of the prominent current can be expressed as: ; In the formula, Represents the synaptic current of the k-th layer neuron at time step t. This is the current signal obtained by integrating the input signal received by the neuron; represents a non-negative vector, called the current decay factor. It indicates the degree of decay of the current over time. The synaptic current of a neuron will not remain constant, but will gradually decay over time. Each layer of neurons will have its own decay factor; represents the synaptic current of the k-th layer neuron at the previous time step t-1. The current synaptic current depends not only on the input pulse of the current time step, but also on the synaptic current of the previous moment. Therefore, the current of the neuron has time dependence and memory.

[0072] S402, continuously stimulating neurons in the pulse neural network through synaptic current, so that a target pulse signal is output when the membrane voltage of the neuron accumulates to a threshold value.

[0073] In order to further enhance the network's ability to capture information at different time scales, the embodiment of the present invention also introduces an adaptive emission threshold mechanism: ; ; In the formula, Indicates that when the neuron fired a pulse in the previous time step , the threshold will increase , which means that the neuron becomes more difficult to activate after firing a pulse, thereby temporarily reducing the firing frequency. This mechanism simulates the post-pulse hyperpolarization phenomenon of biological neurons, that is, the neuron becomes less excitable after firing; if the neuron did not fire a pulse in the previous time step, the threshold will gradually decay according to Attenuation is performed, but it cannot fall below a minimum threshold This ensures that neurons become more susceptible to firing after a long period without firing.

[0074] The above mechanism enables neurons to dynamically adjust their sensitivity based on their recent activity. When neurons fire frequently, the threshold increases, making them less likely to activate, which simulates the fatigue effect of biological neurons. Conversely, the threshold of neurons that have been inactive for a long time decreases, making them more likely to be activated. This adaptive mechanism enables the network to flexibly integrate information at different time scales, capturing both rapidly changing instantaneous features and slowly evolving long-term patterns.

[0075] To reduce the reliance on manual tuning of hyperparameters, some key parameters in the model are set to be trainable and optimized via gradient descent: and : are the decay factors for synaptic current and membrane voltage, respectively. These parameters control how neurons decay past activity over time. By setting them as trainable parameters, the model can automatically adjust these values ​​to suit the needs of different tasks.

[0076] Threshold related parameters: It can also be optimized through training, thus reducing the reliance on manual parameter settings.

[0077] S403, decoding the target pulse signal into a recognition result through a pulse neural network, and outputting the decoding result.

[0078] Specifically, the target pulse signal is implemented as follows: First, the target pulse signal output by the last layer (K-1 layer) of the SNN is input into the pulse fully connected layer of the decoding module. This layer integrates the input pulse signal through the postsynaptic potential function and performs a linear transformation, the formula is as follows: ; In the formula, represents the output pulse signal of the K-1th layer at time step t. It is a binary vector indicating whether each neuron in this layer fired a pulse at time step t. represents the weight matrix of the spike fully connected layer from the K-1th layer to the Kth layer, which maps the output signal of the K-1th layer to the neurons of the next layer, Represents the bias term of the spiking fully connected layer, which is used to adjust the output of each neuron.

[0079] Next, the neural network passes the output of the spiking fully connected layer to a non-spiking fully connected layer. The output of the non-spiking fully connected layer is a real-valued vector that represents the probability of each category at time step t. The process is as follows: ; In the formula, represents the output pulse signal or postsynaptic potential of the Kth layer at time step t. This output has been processed by the pulse fully connected layer. represents the weight matrix of the non-pulse fully connected layer. It maps the pulse signal of the K layer to the category space, and the output is the score of each category (unnormalized class probability). Represents the bias term of the non-pulse fully connected layer. Through the above process, the pulse signal of each time step is converted into a vector , where each element represents the unnormalized probability score for each class at that time step.

[0080] At each time step t, the output of the non-spiking fully connected layer is is a class probability vector. This vector represents the predicted probability distribution of each class by the neural network at time step t. This process can be converted to probabilities by applying the softmax function.

[0081] In order to comprehensively consider the information of the entire time series, the final classification result is obtained by weighted summing the outputs of the predicted probability distribution of each category: ; In the formula, Represents the decoding weight at each time step t. This weight is used to weight the category probability at each time step .

[0082] The advantage of the above decoding method is that it can effectively utilize the advantages of SNN in processing time series data, while achieving flexible classification through non-spiking layers. The spiking layer can preserve the temporal dynamic characteristics of the input signal and capture the instantaneous neural activity pattern, while the non-spiking layer can integrate these temporal features to make more complex decisions.

[0083] Please refer to Figure 2 , Figure 2 A flow chart of a pulse neural network training method provided in an embodiment of the present application is shown. The above embodiment describes the application process of the pulse neural network. Based on the above embodiment, the training process of the pulse neural network will be introduced below. Specifically, the process may further include the following steps: S601, constructing an initial spiking neural network.

[0084] Specifically, the construction of the initial network needs to comprehensively consider the spatial distribution and temporal dynamic characteristics of the EEG signal, so a multi-layer structure design is required. First, the input layer is designed as a two-dimensional matrix to maintain the spatial topological structure of the EEG electrodes, which helps the subsequent convolutional layer to extract spatial features.

[0085] Subsequently, multiple convolutional layers and average pooling layers are added to capture spatial dependencies at different scales. In order to process the temporal dynamic characteristics of EEG signals, a recursive layer is introduced into the network, after which the synaptic potential function can integrate the current moment and historical information to capture complex temporal dependencies.

[0086] In addition, considering the nonlinear characteristics of EEG signals, a neuron model with adaptive firing threshold is added to the network, and its threshold update mechanism allows neurons to dynamically adjust their sensitivity according to activity.

[0087] Finally, a decoding module is designed, including a pulse fully connected layer and a non-pulse fully connected layer, to convert the extracted features into the final classification results. The design of the above initial network structure can not only effectively process the spatiotemporal characteristics of EEG signals, but also has low computational complexity and energy consumption, which is particularly suitable for implementing real-time brain-computer interface systems on portable devices.

[0088] S602, obtaining a sample human electroencephalogram signal, and forward-propagating the initial spiking neural network through the sample human electroencephalogram signal to obtain an output result.

[0089] Specifically, a large amount of labeled EEG sample data is first collected, which should contain EEG activity patterns of different mind control tasks. The obtained sample human EEG signals undergo the same preprocessing steps as the target human EEG signals, including filtering, artifact removal, and normalization.

[0090] Subsequently, the processed sample signal is input into the constructed initial spike neural network. The forward propagation process of the network first converts the EEG signal into synaptic current through the convolutional spike coding layer, and calculates the synaptic current of each layer. Then, the synaptic current continues to stimulate the neurons in the network, and when the membrane voltage accumulates to the threshold, the neuron outputs the target pulse signal. The above process gradually extracts the spatial and temporal features of the EEG signal through multiple convolutional layers and recursive layers. The post-synaptic potential function of the recursive layer integrates the current moment and historical information to capture complex time dependencies. Finally, the decoding module converts the extracted features into a category probability vector for each time step, and obtains the final classification result through weighted summation.

[0091] The above forward propagation process can not only evaluate the network's initial recognition ability of different EEG patterns, but also provide the necessary intermediate calculation results for subsequent back propagation and parameter updates. By forward propagating a large amount of sample data, the performance of the network under various mind control tasks can be comprehensively evaluated, potential problems in the network structure or parameter settings can be identified, and a clear direction can be provided for the subsequent optimization process.

[0092] S603: Obtain the loss value between the output result and the label result.

[0093] Specifically, the present invention uses the cross entropy loss function as the evaluation indicator. This choice is based on the fact that the cross entropy can effectively measure the difference between the predicted distribution and the true distribution. When calculating the loss value, the class probability vector output by the network is first compared with the corresponding true label. For each sample, the cross entropy between its predicted probability distribution and the true label is calculated, and then the samples of the entire batch are averaged to obtain the loss value at the batch level.

[0094] S604: Based on the loss value, back-propagate the initial spiking neural network to obtain a gradient value.

[0095] Specifically, after obtaining the loss value, in order to optimize the performance of the spiking neural network, it is necessary to calculate the gradient of the network parameters through the back propagation algorithm to guide the update of the parameters. The traditional back propagation algorithm is difficult to be directly applied to the spiking neural network. This is because the output of the spiking neuron is a discrete, non-continuous pulse signal that cannot be directly derived. In order to solve this problem, the embodiment of the present invention adopts an improved spatiotemporal back propagation algorithm, which can realize the effective calculation of the gradient while retaining the temporal dynamic characteristics of the spiking neural network.

[0096] Specifically, a rectangular function is first used to approximate the gradient of the pulse to the membrane voltage. This process can be expressed as: ; Where z(v) represents the pseudo gradient function, which is used to approximate the gradient of the pulse to the membrane voltage. is the amplifier, used to control the strength of the pseudo gradient, , is a trapezoidal threshold window used to adjust the range of the pseudo gradient. The pseudo gradient function approximates the change in membrane voltage v during pulse firing by a rectangular or trapezoidal function. This allows the approximate gradient to be used to update parameters in the traditional back-propagation algorithm.

[0097] In a feasible implementation, the gradient value includes at least one of a gradient value of a neuron state, a spatial convolution layer gradient value, a linear layer gradient value, a temporal convolution layer gradient value, a cyclic layer gradient value, and an attenuation factor gradient value.

[0098] The gradient value of the neuron state refers to the partial derivative of the voltage, current and output pulse of the neuron to the loss function. In the embodiment of the present invention, it can be understood as quantifying the degree of influence of the change of the internal state of the neuron on the overall performance of the network. It is used to guide the network on how to adjust the dynamic characteristics of the neuron to improve the recognition accuracy.

[0099] In SNN, the state of a neuron includes voltage (v) and current (c) as well as output pulses (o). The gradient of these states can be calculated by the following formula: Exemplarily, the voltage gradient can be expressed as: ; In the formula, represents the gradient of the membrane voltage v with respect to the loss L in the kth layer at the tth time step, It is a pseudo-gradient function, which approximates the derivative of the membrane voltage v with respect to the pulse o. represents the gradient of the output pulse o with respect to the loss L, Represents the voltage attenuation factor, which is used to control the influence between the two time steps. This formula shows that the voltage gradient is determined by the pseudo gradient of the current time step plus the attenuation of the voltage gradient in the future time step.

[0100] Exemplarily, the current gradient can be expressed as: ; In the formula, represents the gradient of the current c with respect to the loss L in the kth layer at the tth time step, Represents the current attenuation factor. This formula indicates that the current gradient is not only determined by the voltage gradient at the current time step, but also affected by the current gradient at the future time step.

[0101] Exemplarily, the output pulse gradient can be expressed as:

[0102] In the formula, represents the transpose of the weight matrix from the kth layer to the next layer, It represents the gradient of the output pulse o with respect to the loss L in the kth layer at the t−1th time step. The formula shows that the gradient of the output pulse is determined by the current gradient at the current time step and the connection weights between neurons.

[0103] The spatial convolution layer gradient value refers to the partial derivative of the convolution kernel weight and bias to the loss function. In the embodiment of the present invention, it can be understood as a measure of the contribution of the spatial feature extraction capability to the network performance. It is used to optimize the convolution layer parameters and improve the network's ability to capture the spatial features of the EEG signal.

[0104] Correspondingly, the linear layer gradient value refers to the partial derivative of the fully connected layer weights and biases to the loss function. In the embodiment of the present invention, it can be understood as evaluating the degree of influence of the linear transformation on the final classification result. It is used to adjust the fully connected layer parameters and improve the feature mapping and classification capabilities of the network.

[0105] Exemplarily, the spatial convolutional layer gradient value and the linear layer gradient value can be expressed as: ; In the formula, Representation weight The gradient of the loss L is Indicates bias These gradients are calculated by summing up the errors over all time steps.

[0106] The temporal convolution layer gradient value refers to the partial derivative of the weight and bias of the convolution operation in the time dimension to the loss function. In the embodiment of the present invention, it can be understood as the importance of quantifying the temporal feature extraction to the overall performance of the network. It is used to optimize the temporal convolution layer and enhance the network's ability to capture the temporal dynamic features of the EEG signal.

[0107] Exemplarily, the temporal convolutional layer gradient value can be expressed as:

[0108] In the formula, It represents the gradient of the weight of the temporal convolutional layer to the loss L. represents the gradient of the bias of the temporal convolutional layer with respect to the loss L.

[0109] The recurrent layer gradient value refers to the partial derivative of the weight and bias of the recurrent connection to the loss function. In the embodiment of the present invention, it can be understood as evaluating the impact of long-term time-dependent modeling on network performance. It is used to adjust the recurrent layer parameters and improve the network's ability to process long-sequence EEG data.

[0110] Exemplarily, the gradient value of the circulation layer can be expressed as: ; In the formula, represents the gradient of the weight of the loop layer to the loss L, represents the gradient of the bias of the recurrent layer with respect to the loss L.

[0111] The attenuation factor gradient value refers to the partial derivative of the voltage and current attenuation factors to the loss function. In the embodiment of the present invention, it can be understood as the degree of influence of the quantified neuron information retention ability on the network performance. It is used to optimize the memory characteristics of neurons and balance the modeling ability of short-term response and long-term dependence.

[0112] Exemplarily, the attenuation factor gradient value can be expressed as: ; S605. Update the weights and biases of the initial spiking neural network based on the gradient value to obtain a spiking neural network.

[0113] Specifically, the update process uses the gradient descent method to adjust the network parameters according to the calculated gradient values. For the spatial convolution layer, linear layer, temporal convolution layer and recurrent layer, the update formula for their weights and biases can be expressed as: ；

[0114] Where W and b represent weight and bias respectively, η is the learning rate, ∇WL and ∇bL are the gradient values ​​of weight and bias respectively. At the same time, the dynamic parameters of neurons, such as the attenuation factors of voltage and current, are also updated in a similar way. The above comprehensive parameter update strategy ensures that the network can effectively capture the characteristics of EEG signals in both spatial and temporal dimensions.

[0115] In addition, the embodiment of the present invention introduces an adaptive learning rate mechanism to dynamically adjust the learning rate according to the change of the gradient to balance the learning speed and stability during the training process. In this way, the network can converge to the optimal solution faster while avoiding falling into the local optimum. After multiple rounds of iterative updates, the initial pulse neural network gradually evolves into a highly optimized model that can not only accurately identify different mind control modes, but also has a strong generalization ability and can adapt to various complex situations in practical applications.

[0116] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a method for recognizing human electroencephalogram signals based on a pulse neural network.

[0117] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a human electroencephalogram signal recognition method based on a pulse neural network provided by the above methods.

[0119] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute a method for identifying human electroencephalogram signals based on a pulse neural network provided by the above-mentioned methods.

[0120] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing human electroencephalogram signals based on a pulse neural network, characterized in that: include: Obtaining the target person's EEG signal; Inputting the target person's electroencephalogram signal into a spiking neural network to obtain a recognition result output by the spiking neural network; Among them, the recognition result is obtained by decoding the pulse neural network based on the target pulse signal, and the target pulse signal is generated by the synaptic current corresponding to the target person's electroencephalogram signal continuously stimulating the neurons in the pulse neural network, and the pulse neural network is obtained through training.

2. The method for recognizing human electroencephalogram signals based on a spiking neural network according to claim 1, characterized in that: The step of obtaining the target person's electroencephalogram signal comprises: Obtain n first time blocks obtained by integrating the human electroencephalogram sequence collected by the sensor; Determining a mutation point in the n first time blocks according to the accumulated deviations between the frequency quantities in the n first time blocks; The n first time blocks are integrated into m second time blocks based on the mutation point to obtain the target person's electroencephalogram signal, wherein n and m are positive integers, and m is less than or equal to n.

3. The method for recognizing human electroencephalogram signals based on a spiking neural network according to claim 2, characterized in that: After acquiring the n first time blocks obtained by integrating the human electroencephalogram sequence collected by the sensor, the method further includes: determining coherence values ​​of the n first time blocks according to a ratio between the self-spectra and the cross-spectra corresponding to the n first time blocks; The n first time blocks are normalized based on the coherence value.

4. The method for recognizing human electroencephalogram signals based on a spiking neural network according to claim 1, characterized in that: The step of inputting the target person's electroencephalogram signal into a spiking neural network and obtaining a recognition result output by the spiking neural network includes: Inputting the target person's electroencephalogram signal into a spiking neural network, and converting the target person's electroencephalogram signal into a synaptic current based on the spatial features in the target person's electroencephalogram signal by the spiking neural network, wherein the spatial features are used to characterize the dependency relationship between different brain regions; Continuously stimulating neurons in the spiking neural network by the synaptic current so that a target pulse signal is output when the membrane voltage of the neuron accumulates to a threshold value; The target pulse signal is decoded into a recognition result by the pulse neural network, and the decoding result is output.

5. The method for recognizing human electroencephalogram signals based on a pulse neural network according to claim 4, characterized in that: The step of converting the target person's electroencephalogram signal into a synaptic current based on the spatial features in the target person's electroencephalogram signal by the spiking neural network comprises: Extracting the spatial features of the target person's electroencephalogram signal at the current moment through the spiking neural network; Integrate the spatial features of the current moment with the spatial features of the previous moment to obtain spatiotemporal features; The target person's electroencephalogram signal is converted into a synaptic current based on the spatiotemporal characteristics.

6. The method for recognizing human electroencephalogram signals based on a spiking neural network according to any one of claims 1 to 5, characterized in that: Before obtaining the target person's electroencephalogram signal, the method further includes: Construct an initial spiking neural network; Acquire a sample human electroencephalogram signal, and forward-propagate the initial spiking neural network through the sample human electroencephalogram signal to obtain an output result; Obtaining a loss value between the output result and the label result; Based on the loss value, back-propagating the initial spiking neural network to obtain a gradient value; The weights and biases of the initial spiking neural network are updated based on the gradient values ​​to obtain the spiking neural network.

7. The method for recognizing human electroencephalogram signals based on a spiking neural network according to claim 6, characterized in that: The gradient value includes at least one of a gradient value of a neuron state, a spatial convolution layer gradient value, a linear layer gradient value, a temporal convolution layer gradient value, a circulation layer gradient value, and an attenuation factor gradient value.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for recognizing human electroencephalogram signals based on a pulse neural network as described in any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for recognizing human electroencephalogram signals based on a pulse neural network as claimed in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for recognizing human electroencephalogram signals based on a pulse neural network as claimed in any one of claims 1 to 7 is implemented.

Images