Visual cognition training method based on electroencephalogram-behavior multi-mode signal feedback regulation

Abstract

The invention belongs to the technical field of visual cognition training, and discloses a visual cognition training method based on electroencephalogram-behavior multi-mode signal feedback regulation. The method comprises the steps that a computer presents a visual cognition training task, meanwhile, an electroencephalogram signal collecting device synchronously collects electroencephalogram signals in the process that a user executes the training task, and the collected electroencephalogram signals are transmitted to the computer in a data stream mode; the computer calculates electroencephalogram signal features and behavioral signal features when the user executes the training task, and feature values and standard conditions are fed back and presented in a screen of the computer in a bar-shaped indicator and character form according to comparison results of the features and a preset threshold value; and after the user completes all visual cognition training tasks, the computer counts the number of modules reaching the standard according to the electroencephalogram signal characteristic ratio and the behavioral characteristic ratio so as to measure the quality of the user completing the visual cognition training.

Description

Technical Field

[0001] The present invention relates to the technical field of visual cognition training, and in particular to a visual cognition training method based on electroencephalogram-behavior multimodal signal feedback regulation. Background Art

[0002] Practitioners in many industries not only have extraordinary winning abilities and professional skills, but also need to reach a high level of perception and cognition. Cognitive ability refers to the ability of the human brain to process, store and extract information. Cognitive level is one of the important psychological conditions for measuring learning ability and evaluating development potential. How to train cognitive ability is one of the hot topics that cognitive researchers are concerned about.

[0003] In recent years, neurofeedback technology has been considered as one of the effective means to improve cognitive ability. Neurofeedback technology refers to the measurement of brain nerve activity related to specific functions, and feedback to users in the form of vision or hearing, to help brain nerve activity to adjust and improve autonomously. Combining EEG neurofeedback technology with traditional cognitive training methods can simultaneously achieve multi-dimensional adjustment of brain state and cognitive ability level, and improve the efficiency of cognitive training.

[0004] At present, visual cognitive training is still mainly based on traditional methods based on repetitive task training, lacking the combination with new cognitive ability level assessment and enhancement methods. In addition, in task-based neurofeedback training tasks, EEG characteristics are only a measure of the brain state. Using EEG characteristics as the only feedback control method cannot reflect the dynamic changes of the user's cognitive ability during training. Summary of the invention

[0005] The present invention provides a visual cognition training method based on EEG-behavior multimodal signal feedback regulation, which combines visual cognition tasks and joint feedback regulation based on EEG signal and behavioral signal characteristics, and synchronously realizes the improvement of user's visual cognition level and the regulation of user's brain activity state through training. The brain regulation method that users gradually master by feedback of EEG signal characteristics is conducive to maintaining a more positive and excited state during the training process; the user maintains a moderate sense of tension by feedback of behavioral signal characteristics, which is conducive to more concentration and focus on completing the training task goals, thereby helping to improve the efficiency of visual cognition training.

[0006] The technical solution of the present invention is as follows: a visual cognition training method based on EEG-behavior multimodal signal feedback regulation, comprising:

[0007] The computer presents a visual cognition training task, which is divided into a test part and a formal training part, and records the accuracy of task judgment and task completion time as the user's behavioral data features; at the same time, the EEG signal acquisition device synchronously acquires the EEG signals of the user during the visual cognition training task, and transmits the acquired EEG signal features to the computer in the form of a data stream;

[0008] The computer calculates the EEG signal characteristics and behavioral data characteristics of the user when performing the visual cognition training task, and presents the characteristic values ​​and the standard-reaching status feedback in the form of a bar indicator and text on the screen of the computer according to the comparison results of the EEG signal characteristics and the behavioral data characteristics with the preset threshold values;

[0009] After the user completes all visual cognition training tasks, the computer counts the number of training task modules that meet the set requirements according to the EEG signal feature ratio and the behavioral feature ratio to measure the quality of the user's completion of this visual cognition training.

[0010] The test part includes five test task modules; the formal training part includes forty-five training task modules.

[0011] The EEG signal feature ratio is the ratio of the EEG signal feature to the individualized baseline of the EEG feature; the behavioral feature ratio is the ratio of the behavioral feature to the individualized baseline of the behavioral feature; the individualized baseline of the EEG feature is the mean of the EEG signal feature obtained in the test part; the individualized baseline of the behavioral feature is the mean of the behavioral feature obtained in the test part.

[0012] The EEG signal feature acquisition process is as follows: the computer intercepts the EEG signal data segment corresponding to the user's visual cognitive training task from the original EEG signal data stream according to the timestamp, preprocesses the EEG signal data segment, and extracts the EEG signal features related to concentration, that is, the relative power of the β-band of the electrode channel in the parietal lobe area, as the EEG signal features.

[0013] The process of acquiring the behavioral signal feature is as follows: the computer selects effectively completed tasks based on the accuracy rate, and calculates the average completion time of the visual cognitive training tasks corresponding to the effectively completed tasks as the behavioral signal feature.

[0014] After the user completes each training task module, the computer calculates the ratio of the EEG signal feature corresponding to the training task module to the individualized baseline value of the EEG feature, and simultaneously calculates the ratio of the behavioral signal feature corresponding to the training task module to the individualized baseline value of the behavioral feature, and presents the two types of ratios in the form of bar indicators on both sides of the screen of the computer;

[0015] The two types of ratios are compared with corresponding thresholds, and corresponding text feedback is presented on the screen of the computer according to the comparison results.

[0016] The computer counts the number of training task modules that meet the set requirements according to the EEG signal feature ratio and the behavioral feature ratio, including:

[0017] The computer counts the task performances corresponding to the forty-five training task modules of the formal training part, and counts the EEG feature ratio and the behavioral feature ratio corresponding to each training task module; based on the comparison between the two ratios and the corresponding thresholds, the training task module is labeled with a corresponding task completion status; the number of training task modules corresponding to different labels in the forty-five groups of training task modules executed by the user is counted, and the training task modules whose task completion status meets the set requirements are qualified modules, thereby measuring the training quality of the user.

[0018] Beneficial effects of the present invention:

[0019] First, based on the neurofeedback technology, the present invention introduces behavioral indicators as independent feedback parameters, which together with EEG characteristic indicators form a dual-modal feedback system. By real-time detection and feedback of the user's EEG characteristics and behavioral performance, the user can simultaneously grasp his or her own brain activity state and task execution efficiency, thereby providing more comprehensive training information and enhancing the multidimensionality and practicality of training.

[0020] Second, the present invention can help users more clearly identify the relationship between their own state and task performance through the EEG-behavior dual-modal independent feedback mechanism. Users can adjust their concentration according to EEG characteristics (i.e., the relative power of the β band), and optimize the task execution strategy in combination with the changing trend of task completion time, thereby achieving the simultaneous improvement of cognitive ability and behavioral efficiency.

[0021] Third, the present invention is applicable to fields that require quick decision-making and high-precision operations, such as competitive sports, e-sports, etc. By independently feeding back EEG characteristics and behavioral performance, it can help users improve their concentration and task execution efficiency during training, thereby providing more comprehensive support for users' performance in actual combat scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1An overall flow chart of a visual cognition training method based on EEG-behavior multimodal signal feedback regulation provided by an embodiment of the present invention, including the overall step division of the training and the general process and correlation between the training steps;

[0024] Figure 2 This is a module-level flow chart of the training program described in the present invention, which shows the detailed process between the task modules in visual cognitive training and the calculation and presentation method of EEG and behavioral signal feature feedback. DETAILED DESCRIPTION

[0025] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0026] To facilitate understanding of the embodiments of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] It should be noted that the following embodiments and features in the embodiments may be combined with each other in the absence of conflict; and, based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making any creative work are within the scope of protection of the present disclosure.

[0028] In general, the visual cognition training method based on EEG signal neurofeedback proposed in the present invention is divided into a testing part and a formal training part;

[0029] The test part includes five training task modules in total; the formal training part includes forty-five training task modules;

[0030] While the user is performing a series of visual cognitive training tasks presented by the computer, the EEG signal acquisition device worn by the user synchronously acquires the user's EEG signals and transmits the EEG signals to the computer in the form of a data stream;

[0031] The computer performs data segment interception and preprocessing of the received EEG signal data stream, and extracts specific EEG signal features related to concentration from the preprocessed data, namely, the relative β-band power of the electrode channel in the parietal lobe area;

[0032] At the same time, the visual cognition training task software automatically generates and stores behavioral signal data of completed task modules, including task accuracy and task completion time;

[0033] In the test part, the average value of the user's EEG signal characteristics corresponding to the five training task modules contained therein is regarded as the individualized baseline value of the user's EEG characteristics in this visual cognition training, which represents the user's brain concentration state at the beginning of the training;

[0034] In the test part, the average values ​​of the user behavioral characteristics corresponding to the five training task modules, that is, the average value of the task completion time, are regarded as the individualized benchmark values ​​of the user's behavioral characteristics in this visual cognition training, which represent the user's visual cognition ability level at the beginning of the training;

[0035] In the formal training part, after the user completes any training task module, the EEG signal features extracted by the computer are compared with the individualized benchmark values ​​of the EEG features, and the ratio of the two is presented in the form of a bar indicator on the left side of the computer screen; at the same time, the computer calculates the mean task completion time corresponding to the task module and compares it with the individualized benchmark value of the behavioral characteristics, and the ratio of the two is presented in the form of a bar indicator on the right side of the computer screen;

[0036] The EEG signal characteristic ratio also needs to be compared with a preset EEG characteristic threshold interval to judge the adjustment of the current user's brain state. Specifically, if the EEG signal characteristic ratio is higher than the preset EEG characteristic threshold upper limit, it is considered that the user's brain concentration has improved, and positive feedback is presented in the form of text in the computer; if the EEG signal characteristic ratio is lower than the preset EEG characteristic threshold lower limit, it is considered that the user's brain concentration has decreased, and negative feedback is presented in the form of text in the computer; if the EEG signal characteristic ratio is between the lower limit and the upper limit of the preset threshold, it is considered that the user's brain concentration has not changed significantly, and neutral feedback is presented in the form of text in the computer;

[0037] At the same time, the behavioral feature ratio must also be compared with a preset behavioral feature threshold to evaluate the current user's visual cognitive ability level. Specifically, if the behavioral feature ratio is lower than the preset behavioral feature threshold, that is, the user takes a shorter time to complete the task, then the user is considered to have a higher visual cognitive ability, and positive feedback is presented in the form of text in the computer; if the behavioral feature ratio is higher than the preset behavioral feature threshold, that is, the user takes a longer time to complete the task, then the user is considered to have a lower visual cognitive ability, and negative feedback is presented in the form of text in the computer;

[0038] The user gradually learns to adjust the brain state to improve the brain concentration according to the characteristic ratio of the electroencephalogram signal presented in visual feedback and text feedback in the computer;

[0039] At the same time, the user obtains the dynamic changes of his / her current visual cognitive ability level according to the behavioral characteristic ratio presented in visual feedback form and text feedback in the computer, and improves the visual cognitive ability level through repetitive training based on visual cognitive tasks;

[0040] In addition, after the user completes all visual cognitive training tasks, the computer counts the number of qualified modules according to the EEG signal feature ratio and the behavioral feature ratio to measure the quality of the user's completion of this visual cognitive training.

[0041] The overall flow chart of a visual cognition training method based on EEG-behavior multimodal signal feedback regulation provided by an embodiment of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0042] The first step is that before the training begins, the user needs to wear the EEG signal acquisition device and correctly connect the EEG signal acquisition device to the computer;

[0043] The second step is to start the training. The user first completes the test part of the training. The computer counts the EEG signal characteristics and behavioral data characteristics of the user in the test part and constructs the individualized characteristic benchmark value of this training. The specific steps include the following:

[0044] 2.1) The user completes the five modules of visual cognition training tasks included in the test part;

[0045] 2.2) The computer intercepts the EEG signal data segments corresponding to the five module tasks according to the timestamps; the data segments are preprocessed, including 50 Hz notch filtering and 1-30 Hz bandpass filtering;

[0046] 2.3) The computer calculates the power density spectrum of the EEG data corresponding to the electrode channel in the parietal region; calculates the power of the β (13-30 Hz) frequency band and the power of the 1-30 Hz frequency band; calculates the ratio of the β frequency band power to the 1-30 Hz frequency band power to obtain the key EEG signal feature: the relative power of the β frequency band; the β rhythm is generally believed to be highly correlated with the concentration state of the brain and is a key indicator for evaluating the concentration of the brain in the present invention;

[0047] 2.4) The average value of the EEG signal characteristics of the statistical test users is the individualized EEG signal characteristic benchmark value of the user in this training;

[0048] 2.5) The computer selects the effectively completed tasks according to the accuracy of the five module tasks of the test part completed by the user, and calculates the average value of the task completion time corresponding to the effectively completed tasks, which is the individualized behavioral characteristic benchmark value of the user in this training.

[0049] In the third step, when the user performs the training tasks in the formal training part, each time the user completes a task module, the computer calculates the EEG signal characteristics and behavioral data characteristics corresponding to the user when executing the module, and gives the user corresponding text and graphic feedback based on the comparison results of the EEG signal characteristics with the individualized EEG characteristic benchmark values, and the comparison results of the behavioral characteristics with the individualized behavioral characteristic benchmark values, including the following: Figure 2 Steps shown:

[0050] 3.1) The user completes the training task of any module in the formal training part;

[0051] 3.2) The computer intercepts the EEG signal data segment corresponding to the module training task according to the timestamp; performs the above-mentioned preprocessing on the data segment; and extracts the above-mentioned EEG signal features from the data segment;

[0052] 3.3) Calculate the ratio of the EEG signal feature value corresponding to the module to the user's individualized EEG signal feature baseline value for this training, and present the ratio in the form of a bar indicator on the left side of the computer screen, that is, provide visual feedback of the EEG signal feature to the user;

[0053] 3.4) Compare the ratio of the above EEG features to the individualized EEG feature baseline with the pre-set EEG feature threshold interval, and present the corresponding text feedback on the user screen according to the comparison result. Specifically, if the ratio of the two is lower than the pre-set lower threshold, negative feedback is presented in text on the screen; if the ratio of the two is higher than the pre-set upper threshold, positive feedback is presented in text on the screen; if the ratio of the two is between the upper and lower limits of the pre-set threshold, neutral feedback is presented in text on the screen;

[0054] 3.5) The computer calculates the behavioral characteristics corresponding to the training tasks of the module, i.e., the average value of the task completion time, and calculates the ratio of the average value of the task completion time to the individualized behavioral characteristic baseline value, and presents the ratio in the form of a bar indicator on the right side of the screen of the computer, i.e., providing the behavioral characteristics to the user in a visual form;

[0055] 3.6) Compare the ratio with a preset behavioral feature threshold to evaluate the current user's visual cognitive ability level, and present corresponding text feedback on the user screen based on the comparison result. Specifically, if the behavioral feature ratio is lower than the preset behavioral feature threshold, that is, the user takes a shorter time to complete the task, then the user is considered to have a higher visual cognitive ability, and positive feedback is presented in the form of text on the computer; if the behavioral feature ratio is higher than the preset behavioral feature threshold, that is, the user takes a longer time to complete the task, then the user is considered to have a lower visual cognitive ability, and negative feedback is presented in the form of text on the computer.

[0056] In the fourth step, after the user completes all visual cognition training tasks, the computer counts the number of qualified modules according to the EEG signal feature ratio and the behavioral feature ratio to measure the quality of the user's completion of this visual cognition training, which specifically includes the following steps:

[0057] 4.1) The computer counts the task performance corresponding to the 45 task modules included in the formal training part, that is, counts the ratio of EEG characteristics to behavioral characteristics corresponding to each module;

[0058] 4.2) According to the comparison between the two ratios and the corresponding thresholds, the module is labeled with the corresponding task completion status. Specifically, if the EEG feature ratio is lower than the lower limit of the EEG feature threshold and the behavioral feature ratio is lower than the behavioral threshold, it is defined as label "1"; if the EEG feature ratio is between the upper and lower limits of the EEG feature threshold and the behavioral feature ratio is lower than the behavioral threshold, it is defined as label "2"; if the EEG feature ratio is higher than the upper limit of the EEG feature threshold and the behavioral feature ratio is lower than the behavioral threshold, it is defined as label "3"; if the EEG feature ratio is lower than the lower limit of the EEG feature threshold and the behavioral feature ratio is higher than the behavioral threshold, it is defined as label "-1"; if the EEG feature ratio is between the upper and lower limits of the EEG feature threshold and the behavioral feature ratio is higher than the behavioral threshold, it is defined as label "-2"; if the EEG feature ratio is higher than the upper limit of the EEG feature threshold and the behavioral feature ratio is higher than the behavioral threshold, it is defined as label "-3";

[0059] 4.3) In particular, the label "3" represents that in the corresponding module, the user's behavioral characteristics and EEG characteristics have reached expectations; the sum of the number of labels "3" and "-3" represents the total number of EEG signal characteristics that reach expectations; the sum of the number of labels "1", "2" and "3" represents the total number of behavioral characteristics that reach expectations;

[0060] 4.4) Counting the number of modules corresponding to different labels in the forty-five groups of task modules in the formal training phase performed by the user, so as to measure the training quality of the user.

[0061] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A visual cognition training method based on EEG-behavior multimodal signal feedback regulation, characterized in that: include: The computer presents a visual cognition training task, which is divided into a test part and a formal training part. The accuracy of task judgment and task completion time are recorded as the user's behavioral data features; At the same time, the EEG signal acquisition device synchronously acquires the EEG signal of the user during the visual cognitive training task, and transmits the acquired EEG signal features to the computer in the form of a data stream; The computer calculates the EEG signal characteristics and behavioral data characteristics of the user when performing the visual cognition training task, and presents the characteristic values ​​and the standard-reaching status feedback in the form of a bar indicator and text on the screen of the computer according to the comparison results of the EEG signal characteristics and the behavioral data characteristics with the preset threshold values; After the user completes all visual cognition training tasks, the computer counts the number of training task modules that meet the set requirements according to the EEG signal feature ratio and the behavioral feature ratio to measure the quality of the user's completion of this visual cognition training.

2. The visual cognition training method based on EEG-behavior multimodal signal feedback control according to claim 1, characterized in that: The test part includes five test task modules; the formal training part includes forty-five training task modules.

3. The visual cognition training method based on EEG-behavior multimodal signal feedback control according to claim 2 is characterized in that: The EEG signal feature ratio is the ratio of the EEG signal feature to the individualized baseline of the EEG feature; the behavioral feature ratio is the ratio of the behavioral feature to the individualized baseline of the behavioral feature; the individualized baseline of the EEG feature is the mean of the EEG signal feature obtained in the test part; the individualized baseline of the behavioral feature is the mean of the behavioral feature obtained in the test part.

4. The visual cognition training method based on EEG-behavior multimodal signal feedback control according to claim 3 is characterized in that: The EEG signal feature acquisition process is as follows: the computer intercepts the EEG signal data segment corresponding to the user's visual cognitive training task from the original EEG signal data stream according to the timestamp, preprocesses the EEG signal data segment, and extracts the EEG signal features related to concentration, that is, the relative power of the β-band of the electrode channel in the parietal lobe area, as the EEG signal features.

5. The visual cognition training method based on EEG-behavior multimodal signal feedback control according to claim 3, characterized in that: The process of acquiring the behavioral signal feature is as follows: the computer selects effectively completed tasks based on the accuracy rate, and calculates the average completion time of the visual cognitive training tasks corresponding to the effectively completed tasks as the behavioral signal feature.

6. The visual cognition training method based on EEG-behavior multimodal signal feedback control according to any one of claims 2 to 5, characterized in that: After the user completes each training task module, the computer calculates the ratio of the EEG signal feature corresponding to the training task module to the individualized baseline value of the EEG feature, and simultaneously calculates the ratio of the behavioral signal feature corresponding to the training task module to the individualized baseline value of the behavioral feature, and presents the two types of ratios in the form of bar indicators on both sides of the screen of the computer; The two types of ratios are compared with corresponding thresholds, and corresponding text feedback is presented on the screen of the computer according to the comparison results.

7. The visual cognition training method based on EEG-behavior multimodal signal feedback control according to any one of claims 2 to 5, characterized in that: The computer counts the number of training task modules that meet the set requirements according to the EEG signal feature ratio and the behavioral feature ratio, including: The computer counts the task performances corresponding to the forty-five training task modules of the formal training part, and counts the EEG feature ratio and the behavioral feature ratio corresponding to each training task module; based on the comparison between the two ratios and the corresponding thresholds, the training task module is labeled with a corresponding task completion status; the number of training task modules corresponding to different labels in the forty-five groups of training task modules executed by the user is counted, and the training task modules whose task completion status meets the set requirements are qualified modules, thereby measuring the training quality of the user.

Images