Continuous attention nerve feedback training method and system based on brain-computer interface

Abstract

The invention discloses a continuous attention neural feedback training method and system based on a brain-computer interface, and relates to the technical field of neural feedback, and the method comprises the steps: collecting a multi-channel electroencephalogram signal of a user in visual task training in real time; extracting power spectral density characteristics of the multi-channel electroencephalogram signals in a beta frequency band, classifying the power spectral density characteristics by adopting a support vector machine algorithm, and outputting a judgment result of an alert or non-alert state; and according to a judgment result, dynamically adjusting an information fusion proportion alpha value in the visual task through a reward-punishment mechanism, updating image information feedback in the visual task in real time, and adjusting the attention state of the user through an image information feedback result. Neural feedback and a dynamic reward and punishment system are fused, real-time excitation feedback is obtained by autonomously adjusting electroencephalogram activity, the problem of insufficient training power caused by traditional static tasks or single positive feedback is solved, and the long-term training effect is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neurofeedback, in particular to a continuous attention neurofeedback training method and system based on a brain-computer interface. BACKGROUND

[0002] Neurofeedback is a biofeedback technology that real-time feedbacks the brain activity information to the subject in the form of vision, hearing or touch, so as to guide the subject to regulate the brain activity, and finally achieve the behavior change. The feedback signal gives the subject a hint of whether the current brain cortex activity is further away from the target or closer to the target, and the neurofeedback enables the subject to tap the self-regulation ability, and achieve the optimal cognitive level by reaching the target brain activity. The goal of neurofeedback training is to enable the subject to learn how to improve their regulation of certain aspects of brain activity, so as to make such activity sustainable reorganization. Different paradigms are used in neurofeedback training for training different brain patterns, such as training of continuous attention, stress resistance, motor imagination or visualization of positive memory. Neurofeedback technology has broad application prospects, and can not only be used for brain exercise and enhancement of cognitive behavior performance of healthy people, but also be used for rehabilitation training of brain disease patients. Neurofeedback relies on closed loop real-time use of brain activity. Specifically, it is to collect signals from the brain, signal preprocessing (denoising, filtering), extract relevant features to identify the state or intention of the subject, and convert it into feedback, thereby forming a closed loop to enable the subject to adapt in real time.

[0003] Electroencephalogram power spectral density is a relatively sensitive attention fluctuation index, and electroencephalogram can not only be used for monitoring of continuous attention, but also be used as a neurofeedback signal for improvement of continuous attention cognitive ability. Neurofeedback training based on electroencephalogram as a low-cost, side-effect-free and stable intervention method has great application prospects in the improvement of continuous attention. However, the neural mechanism of continuous attention decline is not clear, and the limitations of the neurofeedback training method for continuous attention include: due to the differences in individual physiological signals and self-regulation ability, the system has poor universality, lacks feedback indicators and effective directional guidance methods that objectively reflect the attention state, and is prone to have insufficient training motivation and poor training effect. SUMMARY

[0004] The present application aims at the deficiencies of the prior art, and provides a continuous attention neurofeedback training method and system based on a brain-computer interface to solve the problems in the prior art.

[0005] The present application specifically provides the following technical solutions: The continuous attention neurofeedback training method based on a brain-computer interface comprises: real-time acquisition of multi-channel electroencephalogram signals of a user in a visual task training; extracting power spectral density features of the multi-channel electroencephalogram signals in the beta frequency band, classifying the power spectral density features by using a support vector machine algorithm, and outputting a determination result of an alert state or a non-alert state; the alert state corresponds to a state of a reaction time lower than a threshold value and a fluctuation index CP value of attention higher than a threshold value, and the non-alert state corresponds to a state of a reaction time higher than a threshold value and a fluctuation index CP value of attention lower than a threshold value; According to the determination result, the information fusion ratio a value in the visual task is dynamically adjusted by a reward-punishment mechanism, the image information feedback in the visual task is updated in real time, and the attention state of the user is adjusted through the image information feedback result.

[0006] Preferably, after the multi-channel electroencephalogram signals of the user in the visual task training are collected in real time, the method further comprises: The eye movement and electromyographic artifacts in the multi-channel electroencephalogram signals are removed by including band-pass filtering and independent component analysis; For the multi-channel electroencephalogram signals from which the eye movement and electromyographic artifacts are removed, the segmented signals are intercepted based on a stimulus event, and the segmented signals are updated by using an exponential weighted moving average method, and the updated multi-channel electroencephalogram signals are taken as the extracted multi-channel electroencephalogram signals; the stimulus event is an external input that generates a stimulus signal.

[0007] Preferably, the power spectral density features of the multi-channel electroencephalogram signals in the beta frequency band are extracted, and specifically comprise: The updated multi-channel electroencephalogram signals are subjected to discrete Fourier transform, and the specific expression is: ; In the formula, , , is a complex exponential function on a unit circle, indicating the rotation direction and speed of a frequency component on a complex plane, is a discrete electroencephalogram signal after the discrete Fourier transform; The square of the amplitude is calculated and divided by the period to obtain the power spectral density in the beta frequency band, and the specific expression is: ; In the formula, is a power spectral density feature.

[0008] Preferably, the information fusion ratio a value is obtained by the following process: For the updated segmented signals, the confidence degree prediction of the attention state corresponding to the current trial belonging to the task-related category is performed, and a prediction probability is obtained; The information fusion ratio alpha value is obtained by a difference between a predicted probability of a task-related category and a predicted probability of a task-unrelated category.

[0009] Preferably, the information fusion ratio alpha value in the visual task is dynamically adjusted by a reward-punishment mechanism according to the determination result, and specifically: When the alert state is determined, the reward is performed, the alpha value is increased, and the visual task difficulty is reduced; When the non-alert state is determined, the punishment is performed, the alpha value is reduced, and the visual task difficulty is increased.

[0010] Preferably, the hyperparameters of the support vector machine algorithm are selected by a Bayesian optimization method, including a kernel function type, a penalty factor C and a kernel scale parameter, and an optimization target is to maximize a ten-fold cross-validation classification accuracy.

[0011] Preferably, the determination result of the alert or non-alert state is specifically: A difference between a sum of all correct response trial numbers and a sum of all error response trial numbers in the power spectral density feature is a numerator, and a sum of all correct response trial numbers is a denominator to obtain an attention fluctuation index CP value; A ratio of a reaction time of all correct response trials to a number of target trials with correct responses by all subjects in the power spectral density feature is a reaction time; The attention fluctuation index CP value and the reaction time are used as threshold values, and the alert or non-alert state is determined according to the threshold values.

[0012] The application provides a continuous attention neural feedback training system based on a brain-computer interface, which comprises: An electroencephalogram signal acquisition module is configured to acquire multi-channel electroencephalogram signals of a user in a visual task training in real time. An attention classification module is configured to extract a power spectral density feature of the multi-channel electroencephalogram signals in a beta frequency band, classify the power spectral density feature by using a support vector machine algorithm, and output a determination result of an alert state or a non-alert state; the alert state corresponds to a state with a reaction time lower than a threshold value and an attention fluctuation index CP value higher than a threshold value, and the non-alert state corresponds to a state with a reaction time higher than a threshold value and an attention fluctuation index CP value lower than a threshold value. A feedback signal generation module is configured to dynamically adjust an information fusion ratio alpha value in a visual task by a reward-punishment mechanism according to the determination result, update image information feedback in the visual task in real time, and adjust an attention state of the user through the image information feedback result.

[0013] The application provides a computer device, comprising a memory and a processor, the memory stores a program, and the program is executed by the processor to make the processor execute the steps of the brain-computer interface-based sustained attention neural feedback training method.

[0014] The application provides a storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the brain-computer interface-based sustained attention neural feedback training method.

[0015] Compared with the prior art, the application has the following advantages: The application extracts the power spectral density features of the multi-channel electroencephalogram signals in the beta frequency band, classifies the power spectral density features by using a support vector machine algorithm, and outputs the determination results of the alert or non-alert state, thereby realizing the power spectral density (PSD) feature analysis technology, capturing the electroencephalogram activity changes in real time, identifying the signs of attention decline earlier than the traditional eye movement, skin electricity and other physiological signal monitoring methods, realizing high-precision early warning and immediate intervention, and determining the alert state or non-alert state through the state of the attention fluctuation index CP value, and dynamically adjusting the feedback strength and the task difficulty (such as the reward-punishment mechanism driving the task difficulty to rise and fall) according to the determination results, adjusting the attention state of the user, breaking through the limitation of the standardized training mode, significantly improving the training efficiency, innovatively integrating the neural feedback and the dynamic reward-punishment system, realizing that the user obtains real-time incentive feedback by self-regulating the electroencephalogram activity, solving the problem of insufficient training motivation caused by the traditional static task or single positive feedback, and enhancing the long-term training effect. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 It is a schematic diagram of the electroencephalogram neural feedback system in the embodiment of the application; Figure 2 It is a schematic diagram of the stimulation signal of the two types of information fusion in the embodiment of the application; Figure 3 It is a schematic diagram of the stimulation flow of each group of tasks in the embodiment of the application; Figure 4 It is a schematic diagram of the presentation of the stimulation signal picture in the embodiment of the application; Figure 5 It is a schematic diagram of the reward-punishment feedback mechanism in the embodiment of the application; Figure 6 It is a flowchart of the neural feedback training paradigm in the embodiment of the application; Figure 7 It is a sustained attention neural feedback training enhancement experiment flowchart in the embodiment of the application; Figure 8 The application provides a brain-computer interface-based sustained attention neural feedback training method. DETAILED DESCRIPTION

[0017] The technical solutions of the embodiments of the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should belong to the protection scope of the present application.

[0018] Please refer to Figure 1 , the embodiment provides a neural feedback system for smart elderly care based on federated learning and differential privacy, which includes a user end, a client end and a server end. The user end is a terminal device located in the home of an elderly person, used for data collection, mainly collecting voice data of the elderly person with Internet of Things devices. The client end is a terminal device of each community participating in federated learning training, such as desktop computers, notebook computers, etc. The server end is a medical institution or nursing home center, used for remote monitoring of the health of the elderly.

[0019] As shown in Figure 1 and Figure 8 , the embodiment provides a continuous attention neural feedback training method based on brain-computer interface, which includes the following steps: Step S1: Real-time acquisition of multi-channel electroencephalogram signals of the user in the visual task training through a multi-channel electroencephalogram device (such as a 64 / 128 lead electroencephalogram cap).

[0020] The electroencephalogram signal acquisition module uses a Brain Product 128 magnetic resonance compatible electroencephalogram acquisition system to acquire electroencephalogram data in training. The electroencephalogram acquisition system includes an electroencephalogram cap conforming to the internationally accepted 10-20 standard lead system, an electroencephalogram signal acquisition amplifier, an optical-electric converter, an acquisition computer and acquisition software. The acquisition channel selects 32 channels, and the sampling frequency is 500 Hz. The system uses Brain Vision Recorder software to record the electroencephalogram signals, and the data is transmitted through the remote data access (RDA) mode, which is convenient for connection with other systems for preprocessing. The collected data includes 32 electroencephalogram channels and 1 electrocardiogram signal channel, using Ag / AgCl electrodes, and the signal acquisition quality is ensured through conductive paste. The process of acquiring the stimulation signal is as shown in Figure 3 .

[0021] Step S2: After real-time acquisition of multi-channel electroencephalogram signals of the user in the visual task training, it further includes: Denoising and artifact processing of multi-channel electroencephalogram signals, i.e. removing eye movement and electromyographic artifacts in multi-channel electroencephalogram signals through band-pass filtering (0.1-40 Hz) and independent component analysis ICA.

[0022] For the multi-channel electroencephalogram signal removing eye movement and electromyographic artifacts, the segmented signal is based on the stimulus event, and the exponential weighted moving average method is used to update the segmented signal, and the updated multi-channel electroencephalogram signal is used as the extracted multi-channel electroencephalogram signal; the stimulus event is an external input generating a stimulus signal.

[0023] By processing the EEG data in real time, the collected multi-channel electroencephalogram signal is denoised, including removing eye movement, electromyographic and artifact noise, extracting effective electroencephalogram activity signal, and decoding the individual's attention state. Among them, Figure 4 It is a stimulus signal diagram. By adjusting the task difficulty through real-time feedback, the individual's attention level is maintained or improved. The system can provide personalized training feedback according to the individual's reaction ability and state, and ensure the maximization of training effect.

[0024] In order to ensure the accuracy of the signal, the signal needs to be further preprocessed. The specific steps of preprocessing are as follows:

[0025] Signal segmentation: based on the stimulus starting point (stimulus ms) of each trial, the multi-channel electroencephalogram signal is segmented, and each segment is 900 ms long, including the signal 100 ms before the stimulus to 800 ms after the stimulus.

[0026] Data format conversion: convert the segmented two-dimensional electroencephalogram data into a data format recognizable by the MNE toolkit.

[0027] Channel selection: remove the electrocardiogram channel (ECG) and the 7 electroencephalogram channels of the frontal lobe and temporal lobe (FP1, FP2, F7, F8, TP9, TP10), and the remaining 25 channels are used for subsequent processing.

[0028] Linear trend removal: remove the linear drift in the signal.

[0029] Band-pass filtering: use a double-zero-phase FIR filter to perform 0.1 ~ 40 Hz band-pass filtering on the signal to avoid phase distortion.

[0030] Downsampling: reduce the sampling frequency from 500 Hz to 100 Hz.

[0031] Baseline correction: use the signal from -200 ms to 0 time point for baseline correction.

[0032] Artifact removal: use the signal space projection (SSP) method to remove artifact components.

[0033] Step S3: Extract the power spectral density (PSD) features of the multi-channel electroencephalogram signal in the beta band (13-30 Hz).

[0034] The alertness level of sustained attention reflects the interaction state between various brain regions, and the electroencephalogram can detect the interaction state and quantitatively evaluate the alertness level. Many electroencephalogram features can potentially be used to detect the alertness level of the subject during the task process. The power spectral density (PSD) feature is selected as the main feature of the present application, which is used to represent the power of each frequency band of the multi-channel electroencephalogram signal. The unit of PSD is V² / Hz, and the analog signal is converted into a discrete digital signal by the analog-to-digital conversion (AD conversion) of the electroencephalogram device.

[0035] The power spectral density feature of the multi-channel electroencephalogram signal in the beta frequency band is extracted, specifically: The PSD is calculated by using the periodogram method. For a discrete electroencephalogram signal of a finite length, i.e., the updated multi-channel electroencephalogram signal , first perform discrete Fourier transform, and the formula is: ; In the formula, , , is a complex exponential function on the unit circle, which represents the rotation direction and speed of the frequency component in the complex plane, is the discrete electroencephalogram signal after the discrete Fourier transform. Among them has periodicity, so also has periodicity, and the period is , and finally amplitude square and divide by the period to obtain the power spectral density feature of the beta frequency band, and the specific expression is:

[0036] ; Among them, is the power spectral density feature.

[0037] The power spectral density describes the power distribution of the signal along the frequency, which can be divided into several specific frequency bands: delta (0.1~4Hz), theta (4~8 Hz), alpha (8~13 Hz), beta (13~30 Hz), and gamma (>30 Hz). Among the different frequency bands of the multi-channel electroencephalogram signal, the multi-channel electroencephalogram signal in the beta frequency band has more obvious correlation in cognitive ability and attention level than other frequency bands. Therefore, the power spectral density feature of the beta frequency band is used as the model input in the present application.

[0038] Step S4: classifying the power spectral density feature by using the support vector machine algorithm, and outputting the determination result of the alertness state or the non-alertness state; the alertness state corresponds to the state of the reaction time being lower than the threshold value and the attention fluctuation index CP value being higher than the threshold value, and the non-alertness state corresponds to the state of the reaction time being higher than the threshold value and the attention fluctuation index CP value being lower than the threshold value.

[0039] The hyperparameters of the support vector machine (SVM) are selected by a Bayesian optimization method, including a kernel function type, a penalty factor C and a kernel scale parameter, and an optimization target is to maximize a ten-fold cross-validation classification accuracy.

[0040] The present application defines alert state and alert decline state by analyzing the behavioral indicators of the subjects in the task: reaction time (RT) and concentration fluctuation index (CP). The alert state (high concentration) corresponds to lower reaction time and higher CP value, while the alert decline state (low concentration) shows higher reaction time and lower CP value. These behavioral indicators serve as a reference to help label different segments of the multi-channel electroencephalogram signal as alert or alert decline state, forming the basis of the alert state classification model data set.

[0041] The concentration fluctuation index CP value is obtained by taking the difference between the number of all correct response trials and the number of all error response trials as the numerator, and taking the number of all correct response trials as the denominator; the concentration performance (CP) index can comprehensively reflect the concentration performance, and the calculation formula is as follows: ; In the formula, represents the number of all correct response trials (including target stimuli and non-target stimuli); is the number of all error responses (non-response targets).

[0042] The reaction time of all correct response trials in the power spectrum density feature and the ratio of the number of all correct response trials to the number of target trials performed by the subjects are taken as the reaction time; RT is defined as the average reaction time of the subjects to the target trials, and the calculation formula is as follows: ; In the formula, represents the number of target trials performed by the subjects; represents the reaction time of the i-th correct response trial.

[0043] The concentration fluctuation index CP value and the reaction time are taken as thresholds to determine the alert or non-alert state.

[0044] The present application establishes an alert state classification model based on a support vector machine (SVM) algorithm. Under each frequency band, the PSD features of the samples are normalized and input into the SVM model. , i =1,…,m, m is the number of samples; wherein the sample features ​n is the feature dimension, and in this invention, n=63; tags (Corresponding to alertness / low alertness state). The SVM algorithm solves the optimization problem, with the objective function being:

[0045] ; , ; In the formula, the eigenvector The kernel function Φ maps the data to a high-dimensional space. The SVM binary classification algorithm then searches for a hyperplane, a boundary in this high-dimensional space, that divides the data into two classes, maximizing the margin between the samples and the hyperplane. The hyperplane can be represented as where... It is a normal vector. It is a bias term; >0 is the penalty factor for the error term. It is a slack variable, a penalty term. The addition of this effectively prevents overfitting of the model and enhances its generalization ability.

[0046] The choice of SVM kernel function plays a crucial role in model performance. Available kernel types include linear kernels, polynomial kernels, and radial basis kernels. This invention treats the kernel function type as an adjustable hyperparameter, and the kernel function formula used is as follows: ; Besides the kernel function type, other hyperparameters that need optimization include: The penalty factor C is searched among positive values, with a logarithmic scale ranging from 10⁻³ to 10³.

[0047] The kernel scale parameter is searched for in positive values, with a logarithmic scale range of 10⁻³ to 10³.

[0048] The order of the polynomial kernel is searched among integers in the range of 2 to 4 (this hyperparameter is optimized only if a polynomial kernel function is used).

[0049] This invention employs Bayesian optimization to search for the optimal combination of the four hyperparameters mentioned above (kernel function type, penalty factor C, kernel scale parameter, and polynomial kernel order). During the modeling process, a Bayesian optimizer is used for multiple random sampling initializations to obtain the distribution results of hyperparameters and errors. Based on this, the optimal hyperparameters are approximated using Bayesian optimization. After 30 iterations, the hyperparameter combination with the highest accuracy in 10-fold cross-validation is selected as the optimal hyperparameters. Finally, based on these optimal hyperparameters, classification models are trained on four frequency band datasets, resulting in four alertness state classification models based on frequency band power spectral density features.

[0050] Step S5: According to the determination result, the information fusion ratio a value in the visual task is dynamically adjusted through the reward-punishment mechanism, the image information feedback in the visual task is updated in real time, and the attention state of the user is adjusted through the image information feedback result.

[0051] According to the classification result, a feedback signal is generated in real time. As shown in Figure 5 The present application adjusts the task difficulty through the reward-punishment mechanism to guide the attention state of the individual. When the system decodes that the attention state of the individual decreases, the task difficulty increases (punishment); when the attention of the individual improves, the task difficulty decreases (reward). This mechanism stimulates the individual to maintain high vigilance and improves his sustained attention through real-time adjustment of the task.

[0052] When it is determined that the vigilance state, reward is performed, the a value is increased to reduce the difficulty of the visual task; when it is determined that the non-vigilance state, punishment is performed, the a value is reduced to increase the difficulty of the visual task; the adjustment range of the a value is limited to 0.17 to 0.98, and supports real-time processing, and the total delay from multi-channel electroencephalogram signal acquisition to feedback generation is not more than 200 ms. As shown in Figure 2 .

[0053] The process of obtaining the information fusion ratio a value is specifically: For the updated segmented signal, the prediction of the confidence degree of the attention state of the current trial belonging to the task-related category is performed, and the prediction probability is obtained.

[0054] The information fusion ratio a value is obtained by the difference between the prediction probability of the task-related category and the prediction probability of the task-unrelated category.

[0055] Specifically: The present application decodes the attention state of the subject in real time to determine whether he is focusing on the face information or the scene information. The decoding model is a binary classification model based on logistic regression. The model is trained by multi-channel electroencephalogram signals (converted to column vectors after dimension transformation) to predict the probability of each segmented signal belonging to the task-related category. The model uses the logistic regression algorithm and is realized by the scikit-learn library, adopts L1 regularization and “saga” optimization algorithm.

[0056] As shown in Figure 6 The decoding model is iteratively updated based on the data of multiple “collection groups” and “feedback groups” during the training process. From the second round, the data of the first 12 “collection groups” is used to train the classification model, and the subsequent rounds use the data of the closer “collection groups” and the data of the “feedback groups” of the last round for training. During the whole training process, the multi-channel electroencephalogram signals are processed by artifact rejection, SSP method, and sliding window average.

[0057] Based on the stimulation event intercepts the segmented signal, and then includes: For the artifact-removed segmented signal e, an exponential weighted moving average method is used to update, and the updated segmented signal is obtained; The update formula of the current segmented multi-channel electroencephalogram signal is as follows: ; In the formula, i represents the current trial index; j represents the trial index before i; The updated segmented signal is e'. The decoding model is based on the updated segmented signal e' for real-time classification. The system completes the pre-processing, artifact removal and decoding of the segmented signal e within 200 ms of the corresponding experimental trial.

[0058] The classification model predicts the updated segmented multi-channel electroencephalogram signal e' to obtain the prediction probability pc, that is, the degree of confidence that the attention state of the current trial belongs to the "task-related category". The prediction probability of the classification model is converted into the "classification output" of each segmented multi-channel electroencephalogram signal through the classification output formula. According to the nature of the prediction probability of the binary classification model, it can be inferred that the range of "classification output" is -1~1.

[0059] For real-time classification, the artifact-removed segmented multi-channel electroencephalogram signal is used for decoding, and the classification model outputs the prediction probability, which is then converted into the proportion of the task-related category (alpha value) through the sigmoid function, which is used for real-time updating of the visual feedback. The feedback result is visualized and presented to the subject. The system ensures the identifiability of the task-related information, and adjusts the output value to change between 0.17 and 0.98, ensuring that the visual feedback is sensitive and intuitive.

[0060] In order to correct the bias of the model, especially the phenomenon that the model output is biased towards the "face" category, the system introduces a bias correction parameter. This parameter is obtained through the training data set and is gradually adjusted in subsequent rounds. Through these mechanisms, the system can adaptively adjust the sustained attention level of the brain and provide real-time and effective neural feedback.

[0061] As Figure 7 shown, in order to verify the effectiveness of the neural feedback system, a series of double-blind control experiments were conducted. In the experiment, 14 subjects were randomly divided into true feedback group and false feedback group. All subjects completed the behavior test and feedback training task. The double-blind control experiment is as follows: On the first day, the behavior test rules are explained, and the behavior test is performed by collecting behavior data. On the second day, based on the behavior test, the feedback training rules are explained, and the feedback training is performed by collecting behavior and electroencephalogram data.

[0062] The classification accuracy of the attention state decoding model is analyzed offline to verify that the system significantly improves the sustained attention level of the subject in the short term. The specific results show that the true feedback group improves the attention decoding accuracy by an average of 8.92% after training, and the decoding accuracy is significantly correlated with the behavioral indicators (such as reaction time RT and error rate). The decoding ability improvement rate from the 5th round of feedback training to the 1st round is shown in Table 1.

[0063] Table 1 Decoding ability improvement rate from the 5th round of feedback training to the 1st round A series of double-blind control experiments are conducted to verify the effectiveness of the neural feedback system. The specific results show that through double-blind experiments, the brain attention state recognition accuracy of the user group that actually received feedback training improved by 8.92% compared to before training, while the brain attention state recognition accuracy of the user group that received false feedback improved by 0.68% compared to before training, indicating that the method can effectively improve attention regulation ability and is suitable for attention training of special workers (such as unmanned aerial vehicle operators), and the decoding accuracy is significantly correlated with the behavioral indicators (reaction time RT, error rate RERR) (p<0.05).

[0064] Based on the above method, the application provides a sustained attention neural feedback training system based on a brain-computer interface, which comprises an electroencephalogram signal acquisition module, a classification module and a feedback signal generation module.

[0065] The electroencephalogram signal acquisition module is used to acquire multi-channel electroencephalogram signals of a user in real time during visual task training; the classification module is used to extract power spectral density features of the multi-channel electroencephalogram signals in the beta frequency band, classify the power spectral density features using a support vector machine algorithm, and output a determination result of an alert state or a non-alert state; the alert state corresponds to a state with a reaction time lower than a threshold value and an attention fluctuation index CP value higher than a threshold value, and the non-alert state corresponds to a state with a reaction time higher than a threshold value and an attention fluctuation index CP value lower than a threshold value; and the feedback signal generation module is used to dynamically adjust an information fusion ratio a value in the visual task through a reward-punishment mechanism according to the determination result, update image information feedback in the visual task in real time, and adjust the attention state of the user through the image information feedback result.

[0066] The application also provides a computer device comprising a memory and a processor, the memory storing a program, and the program being executed by the processor to make the processor execute the steps of the sustained attention neural feedback training method based on a brain-computer interface.

[0067] According to disclosed embodiments, a computer device can communicate with one or more external devices (e.g., a keyboard, a pointing device, a Bluetooth communication, etc.), or with any devices (e.g., a router, a modem, etc.) that enable the computer device to communicate with one or more other computer devices.

[0068] The application further provides a storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the brain-computer interface-based sustained attention neural feedback training method.

[0069] According to disclosed embodiments, the storage medium can be a non-volatile computer-readable storage medium, which can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this application, a storage medium can be any tangible media that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.

[0070] The above description is further detailed in combination with specific preferred embodiments, and for those skilled in the art, without departing from the concept of the present application, a number of simple deductions or replacements can be made, which should be considered as falling within the protection scope of the present application.

Claims

1. A method for continuous attention neural feedback training based on a brain-computer interface, characterized in that, The method comprises the following steps: real-time acquisition of multi-channel electroencephalogram signals of a user in visual task training; extracting power spectral density features of the multi-channel electroencephalogram signals in a beta frequency band, classifying the power spectral density features by using a support vector machine algorithm, and outputting a determination result of an alert state or a non-alert state; the alert state corresponds to a state of a reaction time lower than a threshold value and a concentration fluctuation index CP value higher than a threshold value, and the non-alert state corresponds to a state of a reaction time higher than a threshold value and a concentration fluctuation index CP value lower than a threshold value; based on the determination result, dynamically adjusting an information fusion ratio α value in the visual task by using a reward-punishment mechanism, real-time updating image information feedback in the visual task, and adjusting the attention state of the user through the image information feedback result. 2.The brain-computer interface based sustained attention neurofeedback training method of claim 1, wherein, After the real-time acquisition of the multi-channel electroencephalogram signals of the user in the visual task training, the method further comprises the following steps: removing eye movement and electromyographic artifacts in the multi-channel electroencephalogram signals by using a band-pass filter and independent component analysis; for the multi-channel electroencephalogram signals from which the eye movement and electromyographic artifacts are removed, segmenting the signals based on a stimulus event, and updating the segmented signals by using an exponential weighted moving average method, and taking the updated multi-channel electroencephalogram signals as the extracted multi-channel electroencephalogram signals; the stimulus event is an external input that generates a stimulus signal. 3.The brain-computer interface based sustained attention neurofeedback training method of claim 2, wherein, The power spectral density features of the multi-channel electroencephalogram signals in the beta frequency band are as follows: Updated multi-channel electroencephalogram signals a discrete Fourier transform is performed, specifically expressed as: ； wherein , , is a complex exponential function on the unit circle, indicating the direction and speed of rotation of the frequency component in the complex plane, is the discrete electroencephalogram signal after the discrete Fourier transform; Computing the square of the amplitude and dividing by the period The power spectral density characteristics of the β band are obtained, and the specific expression is as follows: ； wherein is the power spectral density characteristic. 4.The brain-computer interface based sustained attention neurofeedback training method of claim 2, wherein, The process of obtaining the information fusion ratio α value is as follows: for the updated segmented signals, predicting the reliability of the attention state of the current trial belonging to a task-related category, and obtaining a prediction probability; obtaining the information fusion ratio α value by using the difference between the prediction probability of the task-related category and the prediction probability of a task-unrelated category.

5. The brain-computer interface-based sustained attention neurofeedback training method of claim 4, wherein, The dynamic adjustment of the information fusion ratio α value in the visual task by using the reward-punishment mechanism based on the determination result is as follows: when the determination result is the alert state, reward is given, and the α value is increased, so as to reduce the difficulty of the visual task; when the determination result is the non-alert state, punishment is given, and the α value is reduced, so as to increase the difficulty of the visual task. 6.The brain-computer interface based sustained attention neurofeedback training method of claim 1, wherein, The hyperparameters of the support vector machine algorithm are selected by using a Bayesian optimization method, including a kernel function type, a penalty factor C, and a kernel scale parameter, and the optimization target is to maximize the classification accuracy rate of ten-fold cross-validation.

7. The brain computer interface-based sustained attention neurofeedback training method of claim 1, wherein, The determination result of the alert or non-alert state is as follows: obtaining a concentration fluctuation index CP value by using the difference between the number of all correct response trials and the number of all error response trials in the power spectral density features as the numerator, and using the number of all correct response trials as the denominator; taking the ratio of the reaction time of all correct response trials to the number of target trials in which the subject makes a correct response as the reaction time; taking the concentration fluctuation index CP value and the reaction time as threshold values, and determining the alert or non-alert state based on the threshold values.

8. A sustained attention neurofeedback training system based on brain-computer interface, characterized in that, The method comprises the following steps: an electroencephalogram signal acquisition module, configured to acquire multi-channel electroencephalogram signals of a user in visual task training in real time; The attention classification module is used for extracting power spectrum density features of the multi-channel electroencephalogram signals in the beta frequency band, classifying the power spectrum density features by using a support vector machine algorithm, and outputting a determination result of an alert state or a non-alert state; the alert state corresponds to a state of a reaction time lower than a threshold value and an attention fluctuation index CP value higher than a threshold value, and the non-alert state corresponds to a state of a reaction time higher than a threshold value and an attention fluctuation index CP value lower than a threshold value; The feedback signal generation module is used for dynamically adjusting an information fusion ratio a value in a visual task by a reward-punishment mechanism according to the determination result, updating image information feedback in the visual task in real time, and adjusting the attention state of the user through the image information feedback result.

9. A computer device, comprising: The computer program is executed by the processor to implement the steps of the brain-computer interface-based persistent attention neural feedback training method in any one of claims 1 to 7.

10. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the brain-computer interface-based persistent attention neural feedback training method in any one of claims 1 to 7.

Images