Brain-computer neurofeedback system based on attention frequency division convolutional neural network

Abstract

The present invention relates to an electroencephalogram neurofeedback system based on an attention frequency division convolutional neural network, comprising: an offline electroencephalogram signal acquisition module for acquiring electroencephalogram signals of a subject induced under specific cue conditions; an electroencephalogram signal preprocessing module for preprocessing the electroencephalogram signals to obtain preprocessed electroencephalogram data; an offline model training module for training the preprocessed electroencephalogram data to obtain an optimal network model; and a neurofeedback real-time adjustment module for, during neurofeedback, loading the optimal network model to obtain a prediction result of the electroencephalogram signals, and enabling the subject to perform real-time self-adjustment based on the prediction result so as to achieve adjustment of the brain state. The system provided by the present invention improves the accuracy of online real-time adjustment of neurofeedback, realizes high-precision recognition of the brain cognitive state, and thus achieves more precise treatment.

Description

Technical Field

[0001] The present invention relates to the fields of cognitive neuroscience and clinical psychology, and particularly relates to an electroencephalogram neurofeedback system based on an attention frequency division convolutional neural network. Background Art

[0002] Electroencephalogram neurofeedback (EEG Neurofeedback), also known as neurofeedback therapy, is a treatment method that helps people learn to regulate their own electroencephalogram activities by monitoring electroencephalogram activities. By using biofeedback technology, patients can see their electroencephalogram activities and try to improve their physiological and psychological states through self-regulation. This technology processes electroencephalogram signals and implements electroencephalogram neurofeedback technology according to different analysis methods and algorithms, mainly including electroencephalogram neurofeedback based on machine learning and electroencephalogram neurofeedback based on deep learning. The history of electroencephalogram neurofeedback can be traced back to the early 1960s. At that time, Professor Barry Sterman accidentally discovered during the study of how NASA astronauts stayed awake that cats could control epileptic seizures through electroencephalogram neurofeedback after training. Since then, electroencephalogram neurofeedback has been gradually applied to a variety of clinical situations, including the treatment of attention deficit hyperactivity disorder, learning disabilities, depression, anxiety, insomnia, etc. In recent years, with the development of technology, electroencephalogram neurofeedback technology has been more widely applied, and various variants have also been developed, such as electroencephalogram-based psychological stress response training, neuroplasticity training, cortical activation training, etc. In addition, electroencephalogram neurofeedback has also been widely studied and applied in aspects such as improving sports performance, improving academic performance, and improving depression and anxiety.

[0003] Traditional electroencephalogram neurofeedback technology based on machine learning mainly classifies and predicts based on traditional machine learning algorithms such as support vector machines and naive Bayes. It often requires manual feature extraction and model selection, and requires more manual intervention. At the same time, it is often easily affected by noise and interference when processing complex data, thereby reducing its performance. These may be the existing reasons for the poor effect of electroencephalogram neurofeedback technology in treatment. Therefore, how to improve the classification effect of electroencephalogram neurofeedback has become an urgent problem to be solved. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides an electroencephalogram neurofeedback system based on an attention frequency division convolutional neural network.

[0005] The technical solution of the present invention is as follows: An electroencephalogram neurofeedback system based on an attention frequency division convolutional neural network includes:

[0006] An offline electroencephalogram signal acquisition module: used to acquire electroencephalogram signals of a subject induced under specific cue conditions;

[0007] EEG signal preprocessing module: used to preprocess the EEG signal to obtain preprocessed EEG data;

[0008] Offline model training module: used to train the preprocessed EEG data to obtain an optimal network model;

[0009] Neurofeedback real-time regulation module: used to load the optimal network model during neurofeedback to obtain a prediction result of the EEG signal, and the subject performs real-time self-regulation based on this prediction result to achieve the regulation of the brain state.

[0010] Compared with the prior art, the present invention has the following advantages:

[0011] The present invention discloses an EEG neurofeedback system based on an attention frequency-divided convolutional neural network, which solves the problem of poor algorithm classification performance in existing traditional EEG neurofeedback based on machine learning, effectively extracts and integrates multi-band EEG features, and further improves the accuracy of online real-time regulation of neurofeedback, realizes high-precision recognition of the brain cognitive state, and thus achieves more precise treatment. Brief Description of the Drawings

[0012] Figure 1 It is a structural block diagram of an EEG neurofeedback system based on an attention frequency-divided convolutional neural network in an embodiment of the present invention;

[0013] Figure 2 It is a schematic diagram of the architecture of an attention frequency-divided convolutional neural network in an embodiment of the present invention;

[0014] Figure 3 It is a schematic diagram of EEG neurofeedback in an embodiment of the present invention. Detailed Description of the Embodiments

[0015] The present invention provides an EEG neurofeedback system based on an attention frequency-divided convolutional neural network, which improves the accuracy of online real-time regulation of neurofeedback, realizes high-precision recognition of the brain cognitive state, and thus achieves more precise treatment.

[0016] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0017] Embodiment 1

[0018] As Figure 1 shown, an EEG neurofeedback system based on an attention frequency-divided convolutional neural network provided by an embodiment of the present invention includes the following modules:

[0019] Offline EEG signal acquisition module: used to acquire the EEG signal of the subject induced under specific cue conditions;

[0020] EEG signal preprocessing module: used to preprocess EEG signals to obtain preprocessed EEG data;

[0021] Offline model training module: used to train the preprocessed EEG data to obtain an optimal network model;

[0022] Neurofeedback real-time regulation module: used during neurofeedback to load the optimal network model to obtain the prediction result of EEG signals, and the subject makes real-time self-regulation based on this prediction result to achieve the regulation of the brain state.

[0023] In one embodiment, the above offline EEG signal acquisition module: used to acquire EEG signals of the subject induced under specific cue conditions, specifically including:

[0024] The EEG signal induced under specific cue conditions refers to taking smoking addiction patients as an example. During the actual treatment process, by showing different smoking pictures that appear on the computer screen to smoking addiction patients in real time, the effect of inducing the patients' addiction craving is achieved. The activity state of the EEG signal when smoking addiction patients see smoking pictures is different from the EEG activity state when normal people view smoking pictures. Therefore, better differentiation between the two can be achieved through specific cue conditions;

[0025] In the embodiment of the present invention, the psychological task paradigm is written based on the Psychophysics Toolbox of Matlab, and the acquisition and recording of EEG data are based on the SynAmps RT2 amplifier provided by NeuroScan Corporation to acquire EEG signals under specific cue induction conditions.

[0026] In one embodiment, the above EEG signal preprocessing module: used to preprocess EEG signals to obtain preprocessed EEG data, specifically including:

[0027] This module needs to preprocess the acquired EEG signals. The purpose is to remove environmental noise during the EEG signal acquisition process and artifacts caused by the subject's eye movement, head movement, etc., so as to improve the signal-to-noise ratio of the EEG data and obtain high-quality EEG signals to meet the requirements of subsequent model training.

[0028] The preprocessing process in the embodiment of the present invention is based on the EEGLAB software package. The main operations include: performing 0.5Hz high-pass filtering, using independent component analysis (ICA) or least squares regression to remove artifacts, removing trials with amplitudes exceeding ±100uV, and manually removing bad segments.

[0029] In one embodiment, the above offline model training module specifically includes:

[0030] Step S1: Divide the preprocessed EEG data into a training set, a validation set, and a test set; input the training set into the Frequency Band Convolutional Neural Network with Attention (FBCNET) for training, which specifically includes the following steps:

[0031] Step S11: The input layer is used to receive the preprocessed EEG data;

[0032] Step S12: The convolutional layer uses the FBCSP algorithm to extract the feature matrix y of each category from the preprocessed EEG data:

[0033]

[0034] Among them, f is the preprocessed EEG data, with a size of t×b, where t is the time point of the signal and n is the number of electrodes of the signal; h is a three-dimensional filter array with a size of m×t×n, where m is the number of filters;

[0035] The FBCSP algorithm adopted in the embodiment of the present invention is a filter bank-based method, which can automatically extract the most discriminative features from EEG signals; then, for the output of each filter, it is necessary to calculate the common spatial pattern, and the specific steps are as follows:

[0036] For the feature matrix y of each category, calculate its covariance matrix σ, add up the covariance matrices of all categories to obtain the total covariance matrix; calculate the orthogonal projection matrix w of the total covariance matrix, so that the projected feature variance is the largest, and at the same time, this feature can distinguish this category from other categories, where the calculation formula of w is as follows:

[0037]

[0038] Among them, c1 and c2 respectively represent the class covariance matrices of two categories, σ c1 and are the corresponding covariance matrices respectively, and are their means;

[0039] Use the spatial calculation pattern method to obtain the eigenvectors and eigenvalues, sort the obtained eigenvectors according to the eigenvalue size, select the first n eigenvectors to obtain the feature map;

[0040] Step S13: Normalize the feature map through the Batch Normalization layer to obtain the normalized feature map;

[0041] The Batch Normalization layer normalizes the feature maps, which can accelerate the convergence speed, reduce the dependence of the model on parameter initialization, accelerate the training of the neural network and improve the accuracy of the model. It normalizes the input data of each layer in the neural network, making the distribution of the input data of each layer more uniform, reducing the internal covariate shift, thereby accelerating the training of the network and improving the generalization ability of the model;

[0042] Step S14: The normalized feature maps are non-linearly transformed through an activation layer, introducing non-linear factors, and the output result is a non-linearly activated feature map:

[0043] Step S15: Downsample the non-linearly activated feature maps to reduce the size of the feature maps, obtaining the pooled feature maps;

[0044] Step S16: Pass the pooled feature maps through a Dropout layer, randomly setting the outputs of some neurons to 0, thereby reducing the mutual dependence between neurons and avoiding overfitting, and outputting multi-channel feature maps;

[0045] Each position of the pooled feature maps corresponds to a set of weights and biases (which can be regarded as neurons). The role of the dropout layer is to randomly set the outputs of some neurons to 0, thereby reducing the mutual dependence between neurons and avoiding overfitting. Specifically, the probability that the output of each neuron in the dropout layer is set to 0 is a hyperparameter p. The larger p is, the weaker the effect of dropout. During the training process, the dropout layer will randomly set the outputs of some neurons to 0, while during testing, the outputs of all neurons will be retained. Commonly used dropout algorithms include standard dropout and variational dropout. The output result of the dropout layer is multi-channel feature maps with some neurons randomly "discarded".

[0046] Step S17: Pass the multi-channel feature maps through an Attention layer, a mechanism for calculating the relationships between different positions of the multi-channel feature maps, assigning different importance at different time steps, obtaining weighted feature maps; The attention output of each position of the weighted feature maps is calculated through the following formula:

[0047]

[0048] where Q, K, and V respectively represent the Query, Key, and Value corresponding to each position in the multi-channel feature maps, and d k represents the dimension of the Key vector of each position;

[0049] The embodiment of the present invention adopts the attention mechanism algorithm Self - Attention, also known as Scaled Dot - Product Attention, which is a mechanism for calculating the relationships between different positions in the feature map. It can assign different importance at different time steps, thus helping the model better capture information.

[0050] Step S18: Flatten the weighted feature map through a fully - connected layer, and the output is a one - dimensional vector, where each value represents the probability estimate of a category, and the length of the one - dimensional vector is equal to the total number of categories involved;

[0051] Step S19: Convert the output of the fully - connected layer into a classification result;

[0052] As Figure 2 shown, it shows the structural schematic diagram of the attention - frequency - division convolutional neural network;

[0053] Step S2: In the offline model training, through adjusting the learning rate and weight initial value for iterative training, an optimal network model is obtained to achieve the prediction and classification of EEG signals in different states, specifically including:

[0054] During the training process, the present invention utilizes a series of training techniques and optimization algorithms, uses the validation set to adjust the learning rate, initialize the weights, etc., to obtain the optimal network model parameters. In addition, the test set is also used to perform cross - validation on the FBCNET model to ensure the robustness and stability of the FBCNET model.

[0055] Through the training of the offline model training module, finally, the trained FBCNET model with the optimal performance is saved, and this model can achieve the accurate prediction and classification of EEG signals in different states.

[0056] In one embodiment, the above - mentioned neural feedback real - time regulation module: is used to load the optimal network model during neural feedback to obtain the prediction result of the EEG signal, and the subject makes real - time self - regulation based on this prediction result to achieve the regulation of the brain state, specifically including:

[0057] By loading the trained FBCNET model obtained in the offline model training module, read the EEG signal in real - time for 1 s and calculate it for 1 s. According to the classification result and the pre - set feedback strategy, the probability value corresponding to the current brain activity state is fed back to the subject as the prediction result. The subject can view the effect of the current regulation on the computer screen, and the subject makes corresponding self - regulation according to the feedback result to achieve the purpose of up - regulation or down - regulation, as Figure 3 shown.

[0058] The real-time regulation of neurofeedback in the present invention is based on the real-time analysis and processing of electroencephalogram (EEG) signals. An attention frequency-division convolutional neural network model is used to predict EEG signals, and the subject can perform real-time self-regulation of the EEG signals according to the prediction results to achieve the regulation of the brain state. The method of the present invention can more accurately regulate EEG signals, thereby better realizing the therapeutic effect of neurofeedback.

[0059] The above embodiments are provided only for the purpose of describing the present invention, and are not intended to limit the scope of the present invention. The scope of the present invention is defined by the appended claims. All equivalent substitutions and modifications made without departing from the spirit and principle of the present invention shall be covered within the scope of the present invention.

Claims

1. A brain-computer neurofeedback system based on an attention frequency division convolutional neural network, characterized in that Including: Offline EEG signal acquisition module: used to acquire EEG signals of subjects induced under specific cue conditions; EEG signal preprocessing module: used to preprocess the EEG signals to obtain preprocessed EEG data; Offline model training module: used to train the preprocessed EEG data to obtain an optimal network model. The offline model training module specifically includes: Step S1: Divide the preprocessed EEG data into a training set, a validation set, and a test set; input the training set into a convolutional neural network based on attention frequency division for training, specifically including the following steps: Step S11: The input layer is used to receive the preprocessed EEG data; Step S12: The convolutional layer uses the FBCSP algorithm to extract the feature matrix of each category from the preprocessed EEG data : = Among them, is the preprocessed EEG data, and its size is , where is the time point of the signal, is the number of electrodes of the signal; is a three-dimensional filter array with a size of , where is the number of filters; For each category's feature matrix y, calculate its covariance matrix , add the covariance matrices of all categories to obtain the total covariance matrix; calculate the orthogonal projection matrix of the total covariance matrix, such that the projected feature variance is maximized and the feature can distinguish this category from other categories, where is calculated as follows: Among them, and respectively represent the class covariance matrices of two categories, and are the corresponding covariance matrices respectively, and are their means; Use the spatial computing mode method to obtain feature vectors and eigenvalues, sort the obtained feature vectors according to the eigenvalue size, select the top n feature vectors to obtain a feature map; Step S13: Normalize the feature map through a Batch Normalization layer to obtain a normalized feature map; Step S14: Perform non-linear transformation on the normalized feature map through an activation layer, introduce non-linear factors, and the output result is a non-linearly activated feature map: Step S15: Downsample the non-linearly activated feature map to reduce the size of the feature map and obtain a pooled feature map; Step S16: Pass the pooled feature map through a Dropout layer, randomly set the outputs of some neurons to 0, thereby reducing the mutual dependence between neurons and avoiding overfitting, and output a multi-channel feature map; Step S17: Pass the multi-channel feature map through an Attention layer, a mechanism for calculating the relationship between different positions of the multi-channel feature map, assign different importance at different time steps, and obtain a weighted feature map; calculate the attention output of each position of the weighted feature map through the following formula: where Q, K, and V respectively represent the Query, Key, and Value corresponding to each position in the multi-channel feature map, represents the dimension of the Key vector for each position; Step S18: Flatten the weighted feature map through a fully connected layer, and the output is a one-dimensional vector, where each value represents the probability estimate value of a category, and the length of the one-dimensional vector is equal to the total number of categories involved; Step S19: Convert the output of the fully connected layer into a classification result; Step S2: In offline model training, perform iterative training by adjusting the learning rate and weight initial value to obtain an optimal network model, and realize the prediction and classification of the EEG signals in different states; Neurofeedback real-time adjustment module: used to load the optimal network model during neurofeedback to obtain the prediction result of the EEG signal, and the subject performs real-time self-adjustment based on this prediction result to achieve the adjustment of the brain state.

Images