Group multi-scene cognitive training method based on MR and multi-modal brain-computer interface

Abstract

The invention provides a group multi-scene cognitive training method and device based on MR and a multi-modal brain-computer interface, and relates to the technical field of cognitive training. The method comprises the following steps: performing cognitive competence evaluation based on a cognitive competence evaluation scale to obtain a cognitive competence evaluation result; based on a cognitive ability database, a preset group number of people and a candidate group database, performing cognitive ability matching according to a cognitive ability evaluation result; based on a cognitive task library, performing task matching according to the group cognitive ability vector; under the MR scene, based on the cooperative training group, the competition training group and the matched cognition task, group cognition training is carried out, signal collection is carried out, and multi-modal physiological signals are obtained; and performing cognitive competence evaluation by using the cognitive competence evaluation model to obtain a trained cognitive competence evaluation result. The group cognition training method based on the BCI multiple scenes is high in training efficiency, large in generalization degree and high in applicability.

Description

Technical Field

[0001] The present invention relates to the field of cognitive training technology, and in particular to a group multi-scenario cognitive training method and device based on MR and multimodal brain-computer interface. Background Art

[0002] Mild cognitive impairment (MCI) is a cognitive state between normal aging and early dementia. It is mainly manifested as memory loss or mild decline in other cognitive functions, but these changes are not enough to significantly affect daily life activities. The main characteristics of mild cognitive impairment include: memory loss, language problems, weakened judgment, inattention, and spatial perception problems.

[0003] Cognitive training is a training method that aims to improve an individual's thinking and perception abilities. It can enhance cognitive functions such as memory, attention, logical reasoning, language skills, and spatial perception. Cognitive training can be achieved through different activities and exercises to help individuals better process information, solve problems, make decisions, and improve brain function.

[0004] Compared with traditional cognitive training, Mixed Reality (MR) training can bring better training results. MR seamlessly integrates virtual content into the real world through sensors, display devices and computer algorithms, providing an immersive interactive experience, combining physical reality with virtual reality, allowing users to see and interact with the real environment and virtual objects at the same time. MR can improve spatial cognitive ability and provide more possibilities for patients with cognitive impairment to improve their spatial orientation ability.

[0005] In order to accurately evaluate the effect of cognitive training, many studies have used brain-computer interface (BCI) technology as an introduction. BCI is a system that directly connects the brain to external devices without relying on peripheral nerves and muscle tissue, allowing the brain to control computers or other electronic devices through scalp electrical signals. BCI can achieve communication between the brain and devices and evaluate cognitive abilities, so it has great potential in medical rehabilitation, neuroscience research and other fields. Group BCI supports multi-user interaction, allowing multiple users to participate in tasks or training together through competition or collaboration. Therefore, group BCI adds elements such as social interaction and teamwork, making up for the lack of training motivation and persistence in single-person cognitive training.

[0006] At present, BCI cognitive training tasks are mostly limited to a single scenario and can only improve a certain cognitive ability. Multi-scenario BCI adds training scenarios for cognitive abilities and supports a variety of cognitive training tasks for users. The advantage of multi-scenario BCI is that it can promote the generalization of cognitive abilities and improve the user's adaptability in different situations. This method places users in a variety of different scenarios, so that cognitive training is not limited to a single task or environment, thereby increasing the practical applicability and effectiveness of the training.

[0007] The mechanism of cognitive dysfunction is very complex, and it is often difficult to characterize the pattern of cognitive dysfunction using single-modal data. Multimodal data has the advantage of being more accurate in assessing the user's cognitive function level and diagnosing the user's cognitive impairment level. For example, electroencephalogram (EEG) can record the brain's electrical activity, electromyography (EMG) can detect muscle fiber status, and eye movement signals (EMS) can track and record eye movements.

[0008] In the process of cognitive training, the assessment of the cognitive ability of the subject is an important part. Traditional cognitive ability assessment methods mainly rely on questionnaires and EEG. However, this method has many limitations, such as: the physiological signal used is single, the user is easily disturbed by the environment during the assessment process, and the accuracy of the assessment results is insufficient. Existing cognitive ability assessment methods can only be used in a specific single scenario. This cognitive ability assessment algorithm is usually trained in a specific environment and task, so it is difficult to show the same accuracy and effectiveness in different scenarios. Since it cannot cover all possible situations and tasks, these algorithms are prone to failure when facing new environments. Cognitive ability assessment in a single scenario relies on data in a specific scenario. These data often lack diversity and are difficult to reflect the complex and changeable situations in the real world. The limitations of the data make it difficult for the algorithm to adapt to different application scenarios.

[0009] Secondly, existing cognitive ability training is usually designed for a single individual. Single-person cognitive training lacks elements of social interaction and teamwork, which may lead to insufficient training motivation and persistence. Studies have shown that group training can enhance participants' motivation and persistence through social interaction and cooperation, making the training effect more lasting. At the same time, group training can provide emotional support and encouragement.

[0010] In the prior art, there is a lack of a group cognitive training method based on BCI in multiple scenarios with high training efficiency, high generalization degree and strong applicability. Summary of the invention

[0011] In order to solve the technical problems of single-person cognitive training and single training scene in the prior art, the embodiment of the present invention provides a group multi-scene cognitive training method and device based on MR and multimodal brain-computer interface. The technical solution is as follows:

[0012] On the one hand, a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface is provided, the method is implemented by a group multi-scenario cognitive device, and the method includes:

[0013] S1. Conduct cognitive ability assessment based on the cognitive ability assessment scale to obtain cognitive ability assessment results;

[0014] S2, based on the cognitive ability database, the preset group size and the candidate group database, cognitive ability matching is performed according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training groups and competitive training groups;

[0015] S3, based on the cognitive task library, performing task matching according to the group cognitive ability vector to obtain matching cognitive tasks;

[0016] S4. In the MR scenario, performing group cognitive training based on the cooperative training group, the competitive training group and the cognitive task, and performing signal collection to obtain multimodal physiological signals;

[0017] S5. Perform cognitive ability assessment using a cognitive ability assessment model based on the multimodal physiological signal to obtain a post-training cognitive ability assessment result.

[0018] On the other hand, a group multi-scenario cognitive training device based on MR and multimodal brain-computer interface is provided, and the device is applied to a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, and the device comprises:

[0019] A pre-training cognitive ability assessment module is used to conduct cognitive ability assessment based on a cognitive ability assessment scale and obtain cognitive ability assessment results;

[0020] A group matching module is used to perform cognitive ability matching according to the cognitive ability evaluation results based on the cognitive ability database, the preset group size and the candidate group database, and obtain group cognitive ability vectors, cooperative training groups and competitive training groups;

[0021] A cognitive task matching module, used to perform task matching based on the cognitive task library and the group cognitive ability vector to obtain matching cognitive tasks;

[0022] A signal acquisition module, used for performing group cognitive training based on the cooperative training group, the competitive training group and the cognitive task, and performing signal acquisition to obtain multimodal physiological signals;

[0023] The post-training ability assessment module is used to perform cognitive ability assessment based on the multimodal physiological signals through a cognitive ability assessment model to obtain a post-training cognitive ability assessment result.

[0024] On the other hand, a group multi-scenario cognitive device is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned group multi-scenario cognitive training methods based on MR and multimodal brain-computer interface is implemented.

[0025] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned group multi-scenario cognitive training methods based on MR and multimodal brain-computer interface.

[0026] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0027] The present invention proposes a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, which can comprehensively and accurately reflect the physiological state of the subject by combining EEG signals, electromyographic signals and eye movement signals. The fusion of multimodal data can effectively improve the accuracy of cognitive ability assessment, overcome the limitations of single signal assessment, and provide more detailed and comprehensive assessment results. Through the group cognitive training mode, the interaction and collaboration between subjects are promoted, the diversity and fun of training are enhanced, and the enthusiasm of subjects to participate is stimulated. Team training can provide richer cognitive training tasks, improve the comprehensiveness of training, and overcome the lack of motivation caused by lack of interaction and team goals in single-person training. The cross-scenario and cross-subject cognitive ability assessment algorithm can accurately assess the cognitive ability levels of different users in different cognitive training scenarios, and improve the applicability and generalization of the cognitive ability assessment algorithm model. The present invention is a group cognitive training method based on BCI with high training efficiency, high generalization and strong applicability in multiple scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 It is a flow chart of a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface provided by an embodiment of the present invention;

[0030] Figure 2It is a block diagram of a group multi-scenario cognitive training device based on MR and multimodal brain-computer interface provided by an embodiment of the present invention;

[0031] Figure 3 It is a structural diagram of a group multi-scenario cognitive device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0033] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0034] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0035] In the embodiments of the present invention, sometimes the subscripts such as W 1 It may be written in non-subscript form such as W1. When the difference is not emphasized, the meaning is the same.

[0036] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0037] The embodiment of the present invention provides a group multi-scenario cognitive training method based on MR and multi-modal brain-computer interface, which can be implemented by a group multi-scenario cognitive device, which can be a terminal or a server. Figure 1 The flowchart of the group multi-scenario cognitive training method based on MR and multimodal brain-computer interface is shown. The processing flow of the method may include the following steps:

[0038] S1. Conduct cognitive ability assessment based on the cognitive ability assessment scale to obtain cognitive ability assessment results.

[0039] In a feasible implementation, the present invention provides multi-modal brain-computer interface group multi-scenario cognitive training in MR scenarios, and the cognitive ability assessment scale used for new users involves multiple technical fields.

[0040] For old users, retrieve the cognitive ability assessment results after the most recent cognitive training stored in the user system. If it is a new user, switch to the scale interface, and the user uses the cognitive assessment scale to implement cognitive ability assessment and obtain cognitive ability assessment results. The cognitive ability assessment results include four cognitive aspects: "memory", "spatial cognitive ability", "attention" and "execution". The assessment results of each cognitive aspect are (1~4). The cognitive ability assessment result is a one-dimensional vector of size 4, and the content in the vector is (1~4).

[0041] Electronic cognitive ability assessment scales are displayed in the MR device, and users complete these electronic scale tests to assess their cognitive abilities. Electronic cognitive ability assessment scales include: Montreal Cognitive Assessment (MoCA), International Cognitive Ability Resource (ICAD), Mental Rotation Test (MRT), and Cold Cognitive Control Test (CCCT).

[0042] S2. Based on the cognitive ability database, the preset group size and the candidate group database, cognitive ability matching is performed according to the cognitive ability assessment results to obtain the group cognitive ability vector, the cooperative training group cognitive ability vector and the competitive training group cognitive ability vector.

[0043] Optionally, based on the cognitive ability database, the preset group size and the candidate group database, cognitive ability matching is performed according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training group cognitive ability vectors and competitive training group cognitive ability vectors, including:

[0044] S21, based on the cognitive ability database and the preset group size, according to the cognitive ability assessment results, using a greedy strategy based on complementarity evaluation to perform cognitive ability matching to obtain a cooperative training group;

[0045] S22. Based on the cognitive ability database and the candidate group database, cognitive ability matching is performed according to the cooperative training group to obtain the group cognitive ability vector and the competitive training group.

[0046] Optionally, based on the cognitive ability database and the preset group size, according to the cognitive ability assessment results, a greedy strategy based on complementarity evaluation is used to perform cognitive ability matching to obtain a cooperative training group, including:

[0047] S211, constructing a current training group for the corresponding users based on the cognitive ability assessment results;

[0048] S212, based on the current training group, calculate the average ability value according to the ability assessment results to obtain the average cognitive ability value of the group;

[0049] S213, calling the cognitive ability database to obtain a set of candidate user cognitive ability assessment results;

[0050] S214, calculating the difference degree according to the cognitive ability evaluation results of the candidate users and the cognitive ability mean of the group, and obtaining a difference degree set of the candidate users;

[0051] S215, based on the complementarity evaluation, selecting a candidate user corresponding to the maximum value in the candidate user difference set; and adding the candidate user to the current training group;

[0052] S216. Repeat steps S212-S215. When the number of group members reaches the preset number of group members, the current training group is determined as a cooperative training group.

[0053] In a feasible implementation, the process is implemented based on "complementarity evaluation" and "greedy algorithm". Input the user's cognitive ability vector, such as [3, 2, 4, 1], which represents the cognitive ability of each user in four dimensions. Output a group containing a preset number of people (such as 3 people).

[0054] Based on the current user's cognitive ability vector, such as [3, 2, 4, 1]. Calculate the current average ability value of the group. If there is only one member in the group [3, 2, 4, 1], the average value vector is [3, 2, 4, 1]. Calculate the difference between each candidate user. For each candidate user (a user who has not yet joined the group), calculate the difference between its cognitive ability vector and the current average ability value of the group. The calculation process is as follows (1):

[0055] (1);

[0056] in, Is the candidate user in The ability value of each dimension, The current group is in The average ability value of each dimension. The greater the difference, the better the complementarity between the user and the current group.

[0057] Among all candidate users, select the user with the greatest difference and add him to the group. Update the cognitive ability vector of the group and recalculate the average value of each dimension in the group. Repeat the above steps until the group reaches the preset size (for example, 3 people).

[0058] Optionally, based on the cognitive ability database and the candidate group database, cognitive ability matching is performed according to the cooperative training group to obtain the group cognitive ability vector and the competitive training group, including:

[0059] S221, based on the cognitive ability database, performing cognitive ability evaluation calculation according to the cooperative training group and the ability assessment results, and obtaining the group cognitive ability vector;

[0060] S222, calling the candidate group database to obtain a candidate group set;

[0061] S223, based on the cognitive ability database, similarity calculation is performed according to the candidate group set and the group cognitive ability vector to obtain a candidate group cognitive ability similarity set;

[0062] S224. Select the candidate group corresponding to the minimum value in the candidate group cognitive ability similarity set as the competitive training group.

[0063] In a feasible implementation, the process is implemented based on "similarity evaluation of Euclidean distance". The input is a set of cognitive ability vectors of groups, such as [12, 10, 13, 8], which represents the comprehensive ability of each dimension in the group. The output is the competitive training group that is closest to the current group's ability.

[0064] Calculate the cognitive ability vector of the current group. For a given group, sum the cognitive ability vectors of all its members. For each candidate competitive training group, calculate their Euclidean distance with the current group. The calculation process is as follows:

[0065] (2);

[0066] in, The current group is in The ability value of each dimension, Is the candidate competition training group in the first The ability value of each dimension.

[0067] Among all candidate groups, the group with the smallest similarity is selected, indicating that the ability level of this group is closest to that of the current group; the matched group serves as the competitive training group of the current group.

[0068] S3. Based on the cognitive task library, task matching is performed according to the group cognitive ability vector to obtain matching cognitive tasks.

[0069] Optionally, based on the cognitive task library, task matching is performed according to the group cognitive ability vector to obtain matching cognitive tasks, including:

[0070] S31, calling the cognitive task library to obtain the difficulty level of the cognitive task;

[0071] S32, calculating a matching score according to the group cognitive ability vector and the cognitive task difficulty level to obtain a matching score set;

[0072] S33: Select the cognitive task corresponding to the maximum matching score in the matching score set as the matching cognitive task.

[0073] In a feasible implementation, according to the cognitive ability assessment results, a cooperative training group is matched with a competitive training group among online users of cognitive tasks, and cognitive training tasks of appropriate difficulty are matched in a cognitive task library.

[0074] The present invention is implemented using the "minimum difference algorithm based on Euclidean distance". The input is the comprehensive cognitive ability vector of the group, such as [12, 10, 13, 8], and a preset cognitive task library. Each cognitive task has a difficulty vector, such as [20, 10, 10, 10], which represents the difficulty of the cognitive task in each dimension. The difficulty vector of cognitive task 1 is [10, 9, 12, 7], and that of cognitive task 2 is [14, 11, 13, 9].

[0075] Calculate the difference between the current group's cognitive ability vector and the difficulty vector of each cognitive task, use the Euclidean distance formula, and use the inverse of the difference to represent the matching degree. The calculation process is as follows (3):

[0076] (3);

[0077] in, The group is in The ability value of each dimension, The cognitive task is The difficulty value of each dimension.

[0078] Among all cognitive tasks, the cognitive task with the largest matching score was selected, indicating that the difficulty of the cognitive task was more suitable for the ability of the current group.

[0079] A total of 24 cognitive training tasks are selected to form a cognitive training task library, in which every 6 cognitive tasks are training cognitive tasks in four cognitive aspects of "memory", "spatial cognitive ability", "attention" and "execution" at different training levels.

[0080] S4. In the MR scenario, group cognitive training is conducted based on cooperative training groups, competitive training groups, and cognitive tasks, and signal collection is performed to obtain multimodal physiological signals.

[0081] In a feasible implementation, multiple physiological signals are collected, including EEG signals, EMG signals, and eye movement signals. Various signals are collected and bandpass filtered by using high-density electrode arrays, electrode patches or surface EMG sensors, and eye tracking equipment to ensure the accuracy and effectiveness of the signals.

[0082] The present invention uses a standard 64-lead non-invasive EEG cap to collect EEG signals, uses an 8-channel discrete EMG signal collector to collect EMG signals, and uses an eye movement capturer to capture eye movement signals.

[0083] S5. According to the multimodal physiological signals, cognitive ability assessment is performed through a cognitive ability assessment model to obtain the post-training cognitive ability assessment results.

[0084] In a feasible implementation manner, the screen of the cognitive task of the present invention will be presented to the user through the MR device, and the player matching and cognitive task matching will be automatically completed by the system. Cognitive tasks are selected from the cognitive task library. The cognitive task library contains a variety of training tasks for different cognitive domains, aiming to exercise and improve the user's cognitive function in an interesting way. Each cognitive domain has a corresponding training cognitive task containing different cognitive task scenarios in the cognitive task library. These cognitive tasks help users improve their cognitive abilities in specific areas through repeated practice and challenging tasks. Cognitive training cognitive tasks not only provide diversified tasks and cross-scenario training cognitive tasks, but also motivate users to make continuous progress, thereby effectively enhancing their cognitive level.

[0085] Optionally, based on the multimodal physiological signals, the ability assessment is performed through a cognitive ability assessment model to obtain a post-training ability assessment result, including:

[0086] S51, preprocessing the multimodal physiological signal to obtain a processed physiological signal;

[0087] S52, extracting features from the processed physiological signals to obtain multimodal physiological signal features;

[0088] S53, inputting the multimodal physiological signal features into the scene discrimination model to obtain a training scene;

[0089] S54, according to the training scenario, selecting a cognitive ability assessment sub-model corresponding to the training scenario in the cognitive ability assessment model;

[0090] S55. Input the multimodal physiological signal features into the cognitive ability assessment sub-model to obtain the post-training ability assessment results.

[0091] In a feasible implementation, the collected EEG is bandpass filtered, and the filter range is set to 0.5Hz to 40Hz, so as to remove low-frequency drift and high-frequency noise and retain effective information on brain electrical activity. Electrode patches or surface electromyography sensors are used to collect electromyographic signals of the right arm. The collected EMG is bandpass filtered, and the filter range is set to 20Hz to 450Hz, so as to remove low-frequency baseline drift and high-frequency noise and retain effective signals generated by muscle contraction. Eye tracking equipment is used to collect eye movement signals, including horizontal and vertical movement data of the eyeballs. The collected eye movement signals are bandpass filtered, and the filter range is set to 0.1Hz to 30Hz, so as to remove low-frequency and high-frequency noise and retain effective signals generated by eye movements.

[0092] Extract EEG features, EMG features and eye movement signal features, such as: average value, absolute average value, time-domain integrated electromyography, median frequency, mean frequency, standard deviation, and skewness. Extract correlation features between the three physiological signals, such as: linear correlation index between EEG average value and EMG average value, and time delay between EEG peak value and EMG peak value. Based on the above signal features, form a feature vector.

[0093] Among them, it is characterized in that the scene discrimination model is constructed according to the encoder structure of the Transformer model;

[0094] The cognitive ability assessment model includes n cognitive ability assessment sub-models; the cognitive ability assessment sub-models are constructed based on the encoder structure of the Transformer model; n is the total number of preset training scenarios.

[0095] In a feasible implementation, the scene discrimination model uses the encoder part of the Transformer, which has powerful feature extraction and global information capture capabilities. The model is trained using a pre-existing historical information dataset. The input of the model is the user's multimodal signal feature vector, and the output is the digital label of the scene where the user is located.

[0096] The cognitive ability assessment model uses the encoder part of Transformer. The model is trained using pre-existing data, with the input being the multimodal signal feature vector of the user in the scenario, and the output of the user's post-training cognitive ability assessment result being a one-dimensional cognitive ability vector of size 4.

[0097] The present invention proposes a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, which can comprehensively and accurately reflect the physiological state of the subject by combining EEG signals, electromyographic signals and eye movement signals. The fusion of multimodal data can effectively improve the accuracy of cognitive ability assessment, overcome the limitations of single signal assessment, and provide more detailed and comprehensive assessment results. Through the group cognitive training mode, the interaction and collaboration between subjects are promoted, the diversity and fun of training are enhanced, and the enthusiasm of subjects to participate is stimulated. Team training can provide richer cognitive training tasks, improve the comprehensiveness of training, and overcome the lack of motivation caused by lack of interaction and team goals in single-person training. The cross-scenario and cross-subject cognitive ability assessment algorithm can accurately assess the cognitive ability levels of different users in different cognitive training scenarios, and improve the applicability and generalization of the cognitive ability assessment algorithm model. The present invention is a group cognitive training method based on BCI with high training efficiency, high generalization and strong applicability in multiple scenarios.

[0098] Figure 2 The block diagram of a group multi-scenario cognitive training device based on MR and multi-modal brain-computer interface according to an exemplary embodiment is shown. The device is used for a group multi-scenario cognitive training method based on MR and multi-modal brain-computer interface. Figure 2 The device includes a pre-training capability assessment module 210, a group matching module 220, a cognitive task matching module 230, a signal acquisition module 240, and a post-training capability assessment module 250. Among them:

[0099] A pre-training ability assessment module 210 is used to perform cognitive ability assessment based on a cognitive ability assessment scale to obtain cognitive ability assessment results;

[0100] A group matching module 220 is used to perform cognitive ability matching based on the cognitive ability database, the preset group size and the candidate group database according to the cognitive ability evaluation results to obtain group cognitive ability vectors, cooperative training groups and competitive training groups;

[0101] A cognitive task matching module 230 is used to perform task matching based on the cognitive task library and the group cognitive ability vector to obtain matching cognitive tasks;

[0102] A signal acquisition module 240 is used to perform group cognitive training based on a cooperative training group, a competitive training group and cognitive tasks, and to acquire signals to obtain multimodal physiological signals;

[0103] The post-training ability assessment module 250 is used to perform cognitive ability assessment based on multimodal physiological signals through a cognitive ability assessment model to obtain a post-training cognitive ability assessment result.

[0104] Optionally, the group matching module 220 is further configured to:

[0105] S21, based on the cognitive ability database and the preset group size, according to the ability assessment results, a greedy strategy based on complementarity evaluation is used to match cognitive abilities to obtain a cooperative training group;

[0106] S22. Based on the cognitive ability database and the candidate group database, cognitive ability matching is performed according to the cooperative training groups to obtain a competitive training group.

[0107] Optionally, the group matching module 220 is further configured to:

[0108] S211, constructing a current training group for the corresponding users based on the cognitive ability assessment results;

[0109] S212, based on the current training group, calculating the average cognitive ability according to the cognitive ability assessment results to obtain the group cognitive ability mean;

[0110] S213, calling the cognitive ability database to obtain a set of candidate user cognitive ability assessment results;

[0111] S214, calculating the difference degree according to the cognitive ability evaluation results of the candidate users and the cognitive ability mean of the group, and obtaining a difference degree set of the candidate users;

[0112] S215, based on the complementarity evaluation, selecting a candidate user corresponding to the maximum value in the candidate user difference set; and adding the candidate user to the current training group;

[0113] S216. Repeat steps S212-S215. When the number of group members reaches the preset number of group members, the current training group is determined as a cooperative training group.

[0114] Optionally, the group matching module 220 is further configured to:

[0115] S221, based on the cognitive ability database, performing cognitive ability evaluation calculation according to the cooperative training group and cognitive ability evaluation results to obtain a group cognitive ability vector;

[0116] S222, calling the candidate group database to obtain a candidate group set;

[0117] S223, based on the cognitive ability database, similarity calculation is performed according to the candidate group set and the group cognitive ability vector to obtain a candidate group cognitive ability similarity set;

[0118] S224. Select the candidate group corresponding to the minimum value in the candidate group cognitive ability similarity set as the competitive training group.

[0119] Optionally, the cognitive task matching module 230 is further configured to:

[0120] S31, calling the cognitive task library to obtain the difficulty level of the cognitive task;

[0121] S32, calculating a matching score according to the group cognitive ability vector and the cognitive task difficulty level to obtain a matching score set;

[0122] S33: Select the cognitive task corresponding to the maximum matching score in the matching score set as the matching cognitive task.

[0123] Optionally, the post-training capability assessment module 250 is further configured to:

[0124] S51, preprocessing the multimodal physiological signal to obtain a processed physiological signal;

[0125] S52, extracting features from the processed physiological signals to obtain multimodal physiological signal features;

[0126] S53, inputting the multimodal physiological signal features into the scene discrimination model to obtain a training scene;

[0127] S54, according to the training scenario, selecting a cognitive ability assessment sub-model corresponding to the training scenario in the cognitive ability assessment model;

[0128] S55. Input the multimodal physiological signal features into the cognitive ability assessment sub-model to obtain the post-training cognitive ability assessment results.

[0129] Among them, the scene discrimination model is built based on the encoder structure of the Transformer model;

[0130] The cognitive ability assessment model includes n cognitive ability assessment sub-models; the cognitive ability assessment sub-models are constructed based on the encoder structure of the Transformer model; n is the total number of preset training scenarios.

[0131] The present invention proposes a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, which can comprehensively and accurately reflect the physiological state of the subject by combining EEG signals, electromyographic signals and eye movement signals. The fusion of multimodal data can effectively improve the accuracy of cognitive ability assessment, overcome the limitations of single signal assessment, and provide more detailed and comprehensive assessment results. Through the group cognitive training mode, the interaction and collaboration between subjects are promoted, the diversity and fun of training are enhanced, and the enthusiasm of subjects to participate is stimulated. Team training can provide richer cognitive training tasks, improve the comprehensiveness of training, and overcome the lack of motivation caused by lack of interaction and team goals in single-person training. The cross-scenario and cross-subject cognitive ability assessment algorithm can accurately assess the cognitive ability levels of different users in different cognitive training scenarios, and improve the applicability and generalization of the cognitive ability assessment algorithm model. The present invention is a group cognitive training method based on BCI with high training efficiency, high generalization and strong applicability in multiple scenarios.

[0132] Figure 3 is a schematic diagram of the structure of a group multi-scenario cognitive device provided by an embodiment of the present invention, such as Figure 3 As shown, the group multi-scenario cognitive device may include the above Figure 2 The group multi-scenario cognitive training device based on MR and multi-modal brain-computer interface is shown. Optionally, the group multi-scenario cognitive device 310 may include a first processor 2001.

[0133] Optionally, the group multi-scenario recognition device 310 may also include a memory 2002 and a transceiver 2003 .

[0134] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0135] Combine the following Figure 3 The components of the group multi-scenario cognitive device 310 are introduced in detail:

[0136] The first processor 2001 is the control center of the group multi-scenario cognitive device 310, and may be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be application specific integrated circuits (ASICs), or may be configured to implement one or more integrated circuits of the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0137] Optionally, the first processor 2001 can perform various functions of the group multi-scenario recognition device 310 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0138] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in FIG.

[0139] In a specific implementation, as an embodiment, the group multi-scenario recognition device 310 may also include multiple processors, such as Figure 3 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0140] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0141] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently and access the first processor 2001 through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0142] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0143] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 3 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0144] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0145] It should be noted that Figure 3 The structure of the group multi-scenario recognition device 310 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0146] In addition, the technical effects of the group multi-scenario cognitive device 310 can refer to the technical effects of the group multi-scenario cognitive training method based on MR and multimodal brain-computer interface described in the above method embodiment, and will not be repeated here.

[0147] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0148] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0149] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0150] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0151] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0152] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0153] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0154] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0155] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0158] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0159] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, characterized in that: The method comprises: S1. Conduct cognitive ability assessment based on the cognitive ability assessment scale to obtain cognitive ability assessment results; S2, based on the cognitive ability database, the preset group size and the candidate group database, cognitive ability matching is performed according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training groups and competitive training groups; S3, based on the cognitive task library, performing task matching according to the group cognitive ability vector to obtain matching cognitive tasks; S4. In the MR scenario, performing group cognitive training based on the cooperative training group, the competitive training group and the cognitive task, and performing signal collection to obtain multimodal physiological signals; S5. Perform cognitive ability assessment using a cognitive ability assessment model based on the multimodal physiological signal to obtain a post-training cognitive ability assessment result.

2. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 1 is characterized in that: The method of performing cognitive ability matching based on the cognitive ability database, the preset number of groups and the candidate group database according to the cognitive ability assessment results to obtain the group cognitive ability vector, the cooperative training group cognitive ability vector and the competitive training group cognitive ability vector includes: S21, based on the cognitive ability database and the preset group size, according to the cognitive ability assessment results, using a greedy strategy based on complementarity evaluation to perform cognitive ability matching to obtain a cooperative training group; S22. Based on the cognitive ability database and the candidate group database, cognitive ability matching is performed according to the cooperative training group to obtain a group cognitive ability vector and a competitive training group.

3. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 2 is characterized in that: The method of matching cognitive abilities based on the cognitive ability database and the preset number of group members according to the cognitive ability assessment results using a greedy strategy based on complementarity evaluation to obtain a cooperative training group includes: S211, constructing a current training group based on the corresponding users of the cognitive ability assessment results; S212, based on the current training group, calculating the average cognitive ability according to the cognitive ability assessment result to obtain the group cognitive ability mean; S213, calling the cognitive ability database to obtain a set of candidate user cognitive ability assessment results; S214, calculating the difference degree based on the cognitive ability evaluation results of the candidate users and the cognitive ability mean of the group to obtain a candidate user difference degree set; S215, based on the complementarity evaluation, selecting a candidate user corresponding to the maximum value in the candidate user difference set; and adding the candidate user to the current training group; S216. Repeat steps S212-S215. When the number of group members reaches the preset number of group members, the current training group is determined as a cooperative training group.

4. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 2 is characterized in that: The method of performing cognitive ability matching based on the cooperative training group based on the cognitive ability database and the candidate group database to obtain the group cognitive ability vector and the competitive training group includes: S221, based on the cognitive ability database, performing cognitive ability evaluation calculation according to the cooperative training group and the ability evaluation result to obtain a group cognitive ability vector; S222, calling the candidate group database to obtain a candidate group set; S223, based on the cognitive ability database, performing similarity calculation according to the candidate group set and the group cognitive ability vector, to obtain a candidate group cognitive ability similarity set; S224, selecting the candidate group corresponding to the minimum value in the candidate group cognitive ability similarity set as the competitive training group.

5. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 1 is characterized in that: The step of performing task matching based on the cognitive task library and the group cognitive ability vector to obtain matching cognitive tasks includes: S31, calling the cognitive task library to obtain the difficulty level of the cognitive task; S32, calculating a matching score according to the group cognitive ability vector and the cognitive task difficulty level to obtain a matching score set; S33: Select the cognitive task corresponding to the maximum matching score in the matching score set as the matching cognitive task.

6. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 1 is characterized in that: According to the multimodal physiological signal, cognitive ability assessment is performed through a cognitive ability assessment model to obtain a post-training cognitive ability assessment result: S51, preprocessing the multimodal physiological signal to obtain a processed physiological signal; S52, performing feature extraction on the processed physiological signal to obtain multimodal physiological signal features; S53, inputting the multimodal physiological signal features into a scene discrimination model to obtain a training scene; S54, according to the training scenario, selecting a cognitive ability assessment sub-model corresponding to the training scenario in the cognitive ability assessment model; S55. Input the multimodal physiological signal features into the cognitive ability assessment sub-model to obtain a post-training cognitive ability assessment result.

7. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 6 is characterized in that: The scene discrimination model is constructed based on the encoder structure of the Transformer model; The cognitive ability assessment model includes n cognitive ability assessment sub-models; the cognitive ability assessment sub-models are constructed according to the encoder structure of the Transformer model; and n is the total number of preset training scenarios.

8. A group multi-scenario cognitive training device based on MR and multi-modal brain-computer interface, the group multi-scenario cognitive training device based on MR and multi-modal brain-computer interface is used to implement the group multi-scenario cognitive training method based on MR and multi-modal brain-computer interface as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A pre-training ability assessment module is used to conduct cognitive ability assessment based on a cognitive ability assessment scale to obtain cognitive ability assessment results; A group matching module, for performing cognitive ability matching according to the cognitive ability evaluation results based on the cognitive ability database, the preset group size and the candidate group database, to obtain group cognitive ability vectors, cooperative training groups and competitive training groups; A cognitive task matching module, used to perform task matching based on the cognitive task library and the group cognitive ability vector to obtain matching cognitive tasks; A signal acquisition module, used for performing group cognitive training based on the cooperative training group, the competitive training group and the cognitive task, and performing signal acquisition to obtain multimodal physiological signals; The post-training ability assessment module is used to perform cognitive ability assessment based on the multimodal physiological signals through a cognitive ability assessment model to obtain a post-training cognitive ability assessment result.

9. A group multi-scenario cognitive device, characterized in that: The group multi-scenario cognitive device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images