A group multi-scenario cognitive training method based on MR and multimodal brain-computer interface

Abstract

The present invention provides a group multi-scenario cognitive training method and device based on MR and multimodal brain-computer interface, relating to the field of cognitive training technology. The method includes: performing cognitive ability assessment based on a cognitive ability assessment scale to obtain cognitive ability assessment results; performing cognitive ability matching based on the cognitive ability assessment results based on a cognitive ability database, a preset group size, and a candidate group database; performing task matching based on the group cognitive ability vector based on a cognitive task library; performing group cognitive training based on cooperative training groups, competitive training groups, and matching cognitive tasks in an MR scenario, and performing signal acquisition to obtain multimodal physiological signals; and performing cognitive ability assessment using a cognitive ability assessment model to obtain post-training cognitive ability assessment results. The present invention is a group cognitive training method based on BCI in multiple scenarios with high training efficiency, high generalization, and strong applicability.

Description

Technical Field

[0001] The present invention relates to the field of cognitive training technology, and in particular to a group multi-scenario cognitive training method and device based on MR and a multimodal brain-computer interface. Background Art

[0002] Mild cognitive impairment (MCI) is a cognitive condition that lies between normal aging and early dementia. It manifests primarily as memory loss or mild decline in other cognitive functions, but these changes are not significant enough to significantly affect daily activities. Key features of MCI include memory loss, language problems, impaired judgment, difficulty concentrating, and problems with spatial perception.

[0003] Cognitive training is a method of training aimed at improving an individual's thinking and perception abilities. It can enhance cognitive functions such as memory, attention, logical reasoning, language skills, and spatial perception. Cognitive training can be achieved through various activities and exercises to help individuals better process information, solve problems, make decisions, and improve brain function.

[0004] Compared to traditional cognitive training, mixed reality (MR) training can deliver superior training results. Using sensors, display devices, and computer algorithms, MR seamlessly integrates virtual content into the real world, providing an immersive interactive experience. Combining physical and virtual reality, MR allows users to simultaneously see and interact with both the real environment and virtual objects. MR can enhance spatial cognition, offering greater opportunities for improving spatial orientation for individuals with cognitive impairment.

[0005] To accurately assess the effectiveness of cognitive training, many studies have introduced brain-computer interface (BCI) technology. BCI is a system that directly connects the brain to external devices without relying on peripheral nerves and muscle tissue, allowing the brain to control computers or other electronic devices through electrical signals from the scalp. BCI enables communication between the brain and devices and assesses cognitive abilities, and therefore has enormous potential in fields such as medical rehabilitation and neuroscience research. Group BCI supports multi-user interaction, allowing multiple users to participate in tasks or training together through competition or collaboration. Therefore, group BCI adds elements of social interaction and teamwork, compensating for the lack of motivation and persistence in individual cognitive training.

[0006] Currently, BCI cognitive training tasks are mostly limited to a single scenario and can only improve a certain cognitive ability. Multi-scenario BCI, on the other hand, adds training scenarios for cognitive abilities and supports a variety of cognitive training tasks for users. The advantage of multi-scenario BCI is that it can promote the generalization of cognitive abilities and improve the user's adaptability in different situations. This method places the user in a variety of different scenarios, so that cognitive training is not limited to a single task or environment, thereby increasing the practical applicability and effectiveness of the training.

[0007] The mechanisms of cognitive impairment are complex, and single-modal data often struggles to characterize patterns. Multimodal data offers the advantage of more precise assessment of a user's cognitive function and diagnosis of their cognitive impairment. For example, electroencephalogram (EEG) can record the brain's electrical activity, electromyography (EMG) can monitor muscle fiber status, and eye movement signals (EMS) can track and record eye movements.

[0008] In the cognitive training process, the assessment of the cognitive ability of the subject is an important part. Traditional cognitive ability assessment methods mainly rely on questionnaires and EEG. However, this method has many limitations, such as: the use of a single physiological signal, the user is easily disturbed by the environment during the assessment process, and the accuracy of the assessment results is insufficient. Existing cognitive ability assessment methods can only be used in a specific single scenario. This type of cognitive ability assessment algorithm is usually trained in a specific environment and task, so it is difficult to show the same accuracy and effectiveness in different scenarios. Since it cannot cover all possible situations and tasks, these algorithms are prone to failure when facing new environments. Cognitive ability assessment in a single scenario relies on data from a specific scenario. These data often lack diversity and cannot reflect the complex and changing situations in the real world. The limitations of the data make it difficult for the algorithm to adapt to different application scenarios.

[0009] Second, existing cognitive training programs are typically designed for individuals. These programs lack the elements of social interaction and teamwork, which can lead to insufficient motivation and persistence. Research has shown that group training can enhance participants' motivation and persistence through social interaction and collaboration, leading to longer-lasting training effects. Group training can also provide emotional support and encouragement.

[0010] In the existing technology, there is a lack of a group cognitive training method based on BCI in multiple scenarios with high training efficiency, high generalization degree and strong applicability. Summary of the Invention

[0011] In order to solve the technical problems of single-person cognitive training and single training scenarios in the existing technology, the embodiment of the present invention provides a group multi-scenario cognitive training method and device based on MR and multimodal brain-computer interface. The technical solution is as follows:

[0012] On the one hand, a group multi-scenario cognitive training method based on MR and a multimodal brain-computer interface is provided. The method is implemented by a group multi-scenario cognitive device and includes:

[0013] S1. Conduct cognitive ability assessment based on the cognitive ability assessment scale to obtain cognitive ability assessment results;

[0014] S2. Based on the cognitive ability database, the preset group size, and the candidate group database, cognitive ability matching is performed according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training groups, and competitive training groups;

[0015] S3. Based on the cognitive task library, perform task matching according to the group cognitive ability vector to obtain matching cognitive tasks;

[0016] S4. In an MR scenario, performing group cognitive training based on the cooperative training group, the competitive training group, and the cognitive task, and performing signal acquisition to obtain multimodal physiological signals;

[0017] S5. Perform cognitive ability assessment using a cognitive ability assessment model based on the multimodal physiological signals to obtain a post-training cognitive ability assessment result.

[0018] On the other hand, a group multi-scenario cognitive training device based on MR and a multimodal brain-computer interface is provided. The device is applied to a group multi-scenario cognitive training method based on MR and a multimodal brain-computer interface. The device includes:

[0019] A pre-training cognitive ability assessment module is used to conduct cognitive ability assessment based on a cognitive ability assessment scale and obtain cognitive ability assessment results;

[0020] A group matching module is used to perform cognitive ability matching based on the cognitive ability database, the preset group size and the candidate group database according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training groups and competitive training groups;

[0021] A cognitive task matching module, configured to perform task matching based on the cognitive task library and the group cognitive ability vector to obtain matching cognitive tasks;

[0022] a signal acquisition module, configured to perform group cognitive training based on the cooperative training group, the competitive training group, and the cognitive task, and to acquire signals to obtain multimodal physiological signals;

[0023] The post-training ability assessment module is used to perform cognitive ability assessment based on the multimodal physiological signals through a cognitive ability assessment model to obtain a post-training cognitive ability assessment result.

[0024] On the other hand, a group multi-scenario cognitive device is provided, which includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned group multi-scenario cognitive training methods based on MR and multimodal brain-computer interface is implemented.

[0025] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned group multi-scenario cognitive training methods based on MR and multimodal brain-computer interface.

[0026] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0027] The present invention proposes a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, which can comprehensively and accurately reflect the physiological state of the subject by combining EEG signals, myoelectric signals and eye movement signals. The fusion of multimodal data can effectively improve the accuracy of cognitive ability assessment, overcome the limitations of single signal assessment, and provide more detailed and comprehensive assessment results. Through the group cognitive training model, interaction and collaboration between subjects are promoted, the diversity and fun of training are enhanced, and the enthusiasm of subjects to participate is stimulated. Team training can provide richer cognitive training tasks, improve the comprehensiveness of training, and overcome the problem of insufficient motivation caused by the lack of interaction and team goals in single-person training. The cross-scenario and cross-subject cognitive ability assessment algorithm can accurately assess the cognitive ability levels of different users in different cognitive training scenarios, and improve the applicability and generalization of the cognitive ability assessment algorithm model. The present invention is a group cognitive training method based on BCI with high training efficiency, high generalization degree and strong applicability in multiple scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 This is a flow chart of a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface provided by an embodiment of the present invention;

[0030] Figure 2This is a block diagram of a group multi-scenario cognitive training device based on MR and multimodal brain-computer interface provided by an embodiment of the present invention;

[0031] Figure 3 It is a structural diagram of a group multi-scenario cognitive device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0033] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0034] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0035] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0036] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0037] The embodiment of the present invention provides a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, which can be implemented by a group multi-scenario cognitive device, which can be a terminal or a server. Figure 1 The flowchart of the group multi-scenario cognitive training method based on MR and multimodal brain-computer interface is shown. The processing flow of the method may include the following steps:

[0038] S1. Conduct cognitive ability assessment based on the cognitive ability assessment scale to obtain cognitive ability assessment results.

[0039] In a feasible implementation, the present invention provides multi-modal brain-computer interface group multi-scenario cognitive training in an MR scenario, and the cognitive ability assessment scale used for new users involves multiple technical fields.

[0040] For existing users, the most recent cognitive ability assessment results after cognitive training stored in the user's system are retrieved. If a new user switches to the scale interface, the user uses the cognitive assessment scale to complete the cognitive ability assessment and obtain the cognitive ability assessment results. The cognitive ability assessment results include four cognitive aspects: "memory", "spatial cognition", "attention", and "execution". The assessment results for each cognitive aspect are (1-4). The cognitive ability assessment result is a one-dimensional vector of size 4, with the contents of the vector being (1-4).

[0041] Electronic cognitive ability assessment scales are displayed in the MR device, and users complete these electronic scale tests to assess their cognitive abilities. These include the Montreal Cognitive Assessment (MoCA), the International Cognitive Ability Resource (ICAD), the Mental Rotation Test (MRT), and the Cold Cognitive Control Test (CCCT).

[0042] S2. Based on the cognitive ability database, the preset group size and the candidate group database, cognitive ability matching is performed according to the cognitive ability assessment results to obtain the group cognitive ability vector, the cooperative training group cognitive ability vector and the competitive training group cognitive ability vector.

[0043] Optionally, based on the cognitive ability database, the preset group size, and the candidate group database, cognitive ability matching is performed according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training group cognitive ability vectors, and competitive training group cognitive ability vectors, including:

[0044] S21. Based on the cognitive ability database and the preset group size, according to the cognitive ability assessment results, a greedy strategy based on complementarity evaluation is used to perform cognitive ability matching to obtain a cooperative training group;

[0045] S22. Based on the cognitive ability database and the candidate group database, cognitive ability matching is performed according to the cooperative training group to obtain the group cognitive ability vector and the competitive training group.

[0046] Optionally, based on the cognitive ability database and the preset group size, according to the cognitive ability assessment results, a greedy strategy based on complementarity evaluation is used to perform cognitive ability matching to obtain a cooperative training group, including:

[0047] S211, constructing a current training group for the corresponding user based on the cognitive ability assessment result;

[0048] S212. Based on the current training group, calculate the average ability value according to the ability assessment results to obtain the group cognitive ability mean;

[0049] S213: Calling a cognitive ability database to obtain a set of candidate user cognitive ability assessment results;

[0050] S214: Calculate the difference degree based on the candidate user cognitive ability evaluation results and the group cognitive ability mean to obtain a candidate user difference degree set;

[0051] S215. Based on the complementarity evaluation, select the candidate user corresponding to the maximum value in the candidate user difference set; and add the candidate user to the current training group;

[0052] S216. Repeat steps S212-S215. When the number of group members reaches the preset number of group members, the current training group is determined as a cooperative training group.

[0053] In one feasible implementation, this process is based on "complementarity evaluation" and a "greedy algorithm." Input is a user's cognitive ability vector, such as [3, 2, 4, 1], representing each user's cognitive ability across four dimensions. Output is a group consisting of a preset number of people (e.g., 3).

[0054] Based on the current user's cognitive ability vector, such as [3, 2, 4, 1], calculate the current average ability value of the group. If there is only one member in the group [3, 2, 4, 1], the average vector is [3, 2, 4, 1]. Calculate the difference between each candidate user. For each candidate user (a user who has not yet joined the group), calculate the difference between its cognitive ability vector and the current average ability value of the group. The calculation process is as follows (1):

[0055] (1);

[0056] in, The candidate user is The ability value of each dimension, The current group is in The average ability value of each dimension. The greater the difference, the more complementary the user is to the current group.

[0057] From all candidate users, select the user with the greatest difference and add them to the group. Update the group's cognitive ability vector and recalculate the average value for each dimension within the group. Repeat these steps until the group reaches the desired size (e.g., 3 people).

[0058] Optionally, based on the cognitive ability database and the candidate group database, cognitive ability matching is performed according to the cooperative training group to obtain the group cognitive ability vector and the competitive training group, including:

[0059] S221. Based on the cognitive ability database, perform cognitive ability evaluation calculation according to the cooperative training group and ability assessment results to obtain a group cognitive ability vector;

[0060] S222: Call the candidate group database to obtain a candidate group set;

[0061] S223. Based on the cognitive ability database, similarity calculation is performed based on the candidate group set and the group cognitive ability vector to obtain a candidate group cognitive ability similarity set;

[0062] S224. Select the candidate group corresponding to the minimum value in the candidate group cognitive ability similarity set as the competitive training group.

[0063] In one feasible implementation, this process is based on "similarity evaluation using Euclidean distance." The input is a set of cognitive ability vectors for each group, such as [12, 10, 13, 8], representing the group's overall ability across various dimensions. The output is the competitive training group that most closely matches the current group's ability.

[0064] Calculate the cognitive ability vector of the current group. For a given group, sum the cognitive ability vectors of all its members. For each candidate competitive training group, calculate the Euclidean distance between them and the current group. The calculation process is as follows (2):

[0065] (2);

[0066] in, The current group is in The ability value of each dimension, Is the candidate competition training group in the first Ability value of each dimension.

[0067] Among all candidate groups, the group with the smallest similarity is selected, indicating that the ability level of this group is closest to that of the current group; the matched group serves as the competitive training group of the current group.

[0068] S3. Based on the cognitive task library, task matching is performed according to the group's cognitive ability vector to obtain matching cognitive tasks.

[0069] Optionally, based on the cognitive task library, task matching is performed according to the group cognitive ability vector to obtain matching cognitive tasks, including:

[0070] S31. Calling the cognitive task library to obtain the cognitive task difficulty level;

[0071] S32, calculating a matching score based on the group cognitive ability vector and the cognitive task difficulty level to obtain a matching score set;

[0072] S33: Select the cognitive task corresponding to the maximum matching score in the matching score set as the matching cognitive task.

[0073] In a feasible implementation, based on the cognitive ability assessment results, a cooperative training group and a competitive training group are matched among online users of cognitive tasks, and cognitive training tasks of appropriate difficulty are matched in a cognitive task library.

[0074] This method uses the "minimum difference algorithm based on Euclidean distance" to implement the problem. The input is the group's comprehensive cognitive ability vector, for example, [12, 10, 13, 8], and a preset cognitive task library. Each cognitive task has a difficulty vector, for example, [20, 10, 10, 10], which represents the difficulty of the cognitive task in various dimensions. The difficulty vector for cognitive task 1 is [10, 9, 12, 7], and that for cognitive task 2 is [14, 11, 13, 9].

[0075] Calculate the difference between the current group's cognitive ability vector and the difficulty vector of each cognitive task, use the Euclidean distance formula, and use the inverse of the difference to express the matching degree. The calculation process is as follows (3):

[0076] (3);

[0077] in, The group is in The ability value of each dimension, Is the cognitive task The difficulty value of each dimension.

[0078] Among all cognitive tasks, the cognitive task with the largest matching score was selected, indicating that the difficulty of the cognitive task was more suitable for the ability of the current group.

[0079] A total of 24 cognitive training tasks are selected to form a cognitive training task library, in which every 6 cognitive tasks are training cognitive tasks in four cognitive aspects of "memory", "spatial cognitive ability", "attention" and "execution" at different training levels.

[0080] S4. In the MR scenario, group cognitive training is conducted based on cooperative training groups, competitive training groups, and cognitive tasks, and signal acquisition is performed to obtain multimodal physiological signals.

[0081] In one feasible implementation, multiple physiological signals, including EEG, EMG, and eye movement signals, are collected using high-density electrode arrays, electrode patches, or surface EMG sensors, along with eye tracking equipment. These signals are then bandpass filtered to ensure accuracy and validity.

[0082] The present invention uses a standard 64-lead non-invasive EEG cap to collect EEG signals, uses an 8-channel discrete EMG signal collector to collect EMG signals, and uses an eye movement capturer to capture eye movement signals.

[0083] S5. Based on the multimodal physiological signals, cognitive ability assessment is performed using a cognitive ability assessment model to obtain post-training cognitive ability assessment results.

[0084] In a feasible implementation, the screen of the cognitive task of the present invention will be presented to the user through the MR device, and the player matching and cognitive task matching will be automatically completed by the system. Cognitive tasks are selected from the cognitive task library. The cognitive task library contains a variety of training tasks for different cognitive domains, aiming to exercise and improve the user's cognitive function in an interesting way. Each cognitive domain has corresponding training cognitive tasks containing different cognitive task scenarios in the cognitive task library. These cognitive tasks help users improve their cognitive abilities in specific areas through repeated practice and challenging tasks. Cognitive training cognitive tasks not only provide diverse tasks and cross-scenario training cognitive tasks, but also motivate users to make continuous progress, thereby effectively enhancing their cognitive level.

[0085] Optionally, based on the multimodal physiological signals, an ability assessment is performed using a cognitive ability assessment model to obtain a post-training ability assessment result, including:

[0086] S51. Preprocessing the multimodal physiological signal to obtain a processed physiological signal;

[0087] S52, performing feature extraction on the processed physiological signal to obtain multimodal physiological signal features;

[0088] S53, inputting the multimodal physiological signal features into the scene discrimination model to obtain a training scene;

[0089] S54. Selecting a cognitive ability assessment sub-model corresponding to the training scenario in the cognitive ability assessment model according to the training scenario;

[0090] S55. Input the multimodal physiological signal features into the cognitive ability assessment sub-model to obtain the post-training ability assessment results.

[0091] In a feasible implementation, the collected EEG is band-pass filtered with a filter range set to 0.5Hz to 40Hz to remove low-frequency drift and high-frequency noise, retaining effective EEG activity information. Electrode patches or surface electromyography sensors are used to collect electromyographic signals of the right arm. The collected EMG is band-pass filtered with a filter range set to 20Hz to 450Hz to remove low-frequency baseline drift and high-frequency noise, retaining effective signals generated by muscle contraction. Eye tracking equipment is used to collect eye movement signals, including horizontal and vertical eye movement data. The collected eye movement signals are band-pass filtered with a filter range set to 0.1Hz to 30Hz to remove low-frequency and high-frequency noise, retaining effective signals generated by eye movement.

[0092] Extract EEG, EMG, and eye movement signal features, such as mean, absolute mean, time-domain integrated EMG, median frequency, mean frequency, standard deviation, and skewness. Extract correlation features between the three physiological signals, such as the linear correlation index between the EEG and EMG mean values ​​and the time delay between the EEG and EMG peaks. Based on these signal features, generate a feature vector.

[0093] Among them, it is characterized in that the scene discrimination model is constructed based on the encoder structure of the Transformer model;

[0094] The cognitive ability assessment model includes n cognitive ability assessment sub-models; the cognitive ability assessment sub-models are constructed based on the encoder structure of the Transformer model; n is the total number of preset training scenarios.

[0095] In one feasible implementation, the scene discrimination model utilizes the encoder portion of the Transformer, which has powerful feature extraction and global information capture capabilities. This model is trained using a pre-existing historical information dataset. The model's input is a user's multimodal signal feature vector, and its output is a numerical label for the scene in which the user is located.

[0096] The cognitive ability assessment model uses the encoder component of the Transformer. This model is trained using pre-existing data. Its input is the user's multimodal signal feature vector in the scenario. The output is a one-dimensional cognitive ability vector of size 4 representing the user's post-training cognitive ability assessment.

[0097] The present invention proposes a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, which can comprehensively and accurately reflect the physiological state of the subject by combining EEG signals, myoelectric signals and eye movement signals. The fusion of multimodal data can effectively improve the accuracy of cognitive ability assessment, overcome the limitations of single signal assessment, and provide more detailed and comprehensive assessment results. Through the group cognitive training model, interaction and collaboration between subjects are promoted, the diversity and fun of training are enhanced, and the enthusiasm of subjects to participate is stimulated. Team training can provide richer cognitive training tasks, improve the comprehensiveness of training, and overcome the problem of insufficient motivation caused by the lack of interaction and team goals in single-person training. The cross-scenario and cross-subject cognitive ability assessment algorithm can accurately assess the cognitive ability levels of different users in different cognitive training scenarios, and improve the applicability and generalization of the cognitive ability assessment algorithm model. The present invention is a group cognitive training method based on BCI with high training efficiency, high generalization degree and strong applicability in multiple scenarios.

[0098] Figure 2 This is a block diagram of a group multi-scenario cognitive training device based on MR and multi-modal brain-computer interface according to an exemplary embodiment. The device is used for a group multi-scenario cognitive training method based on MR and multi-modal brain-computer interface. Figure 2 The device includes a pre-training ability assessment module 210, a group matching module 220, a cognitive task matching module 230, a signal acquisition module 240, and a post-training ability assessment module 250.

[0099] A pre-training ability assessment module 210 is used to perform a cognitive ability assessment based on a cognitive ability assessment scale to obtain a cognitive ability assessment result;

[0100] A group matching module 220 is configured to perform cognitive ability matching based on the cognitive ability database, the number of preset groups, and the candidate group database, and to obtain group cognitive ability vectors, cooperative training groups, and competitive training groups according to the cognitive ability assessment results;

[0101] A cognitive task matching module 230 is used to perform task matching based on the cognitive task library and the group cognitive ability vector to obtain matching cognitive tasks;

[0102] Signal acquisition module 240, for performing group cognitive training based on cooperative training groups, competitive training groups, and cognitive tasks, and performing signal acquisition to obtain multimodal physiological signals;

[0103] The post-training ability evaluation module 250 is used to perform cognitive ability evaluation based on the multimodal physiological signals through a cognitive ability evaluation model to obtain a post-training cognitive ability evaluation result.

[0104] Optionally, the group matching module 220 is further configured to:

[0105] S21. Based on the cognitive ability database and the preset group size, according to the ability assessment results, a greedy strategy based on complementarity evaluation is used to match cognitive abilities to obtain a cooperative training group;

[0106] S22. Based on the cognitive ability database and the candidate group database, cognitive ability matching is performed according to the cooperative training groups to obtain the competitive training groups.

[0107] Optionally, the group matching module 220 is further configured to:

[0108] S211, constructing a current training group for the corresponding user based on the cognitive ability assessment result;

[0109] S212. Calculate the average cognitive ability of the current training group according to the cognitive ability assessment results to obtain the group cognitive ability mean;

[0110] S213: Calling a cognitive ability database to obtain a set of candidate user cognitive ability assessment results;

[0111] S214: Calculate the difference degree based on the candidate user cognitive ability evaluation results and the group cognitive ability mean to obtain a candidate user difference degree set;

[0112] S215. Based on the complementarity evaluation, select the candidate user corresponding to the maximum value in the candidate user difference set; and add the candidate user to the current training group;

[0113] S216. Repeat steps S212-S215. When the number of group members reaches the preset number of group members, the current training group is determined as a cooperative training group.

[0114] Optionally, the group matching module 220 is further configured to:

[0115] S221. Based on the cognitive ability database, perform cognitive ability evaluation calculation according to the cooperative training group and cognitive ability assessment results to obtain a group cognitive ability vector;

[0116] S222: Call the candidate group database to obtain a candidate group set;

[0117] S223. Based on the cognitive ability database, similarity calculation is performed based on the candidate group set and the group cognitive ability vector to obtain a candidate group cognitive ability similarity set;

[0118] S224. Select the candidate group corresponding to the minimum value in the candidate group cognitive ability similarity set as the competitive training group.

[0119] Optionally, the cognitive task matching module 230 is further configured to:

[0120] S31. Calling the cognitive task library to obtain the cognitive task difficulty level;

[0121] S32, calculating a matching score based on the group cognitive ability vector and the cognitive task difficulty level to obtain a matching score set;

[0122] S33: Select the cognitive task corresponding to the maximum matching score in the matching score set as the matching cognitive task.

[0123] Optionally, the post-training capability assessment module 250 is further configured to:

[0124] S51. Preprocessing the multimodal physiological signal to obtain a processed physiological signal;

[0125] S52, performing feature extraction on the processed physiological signal to obtain multimodal physiological signal features;

[0126] S53, inputting the multimodal physiological signal features into the scene discrimination model to obtain a training scene;

[0127] S54. Selecting a cognitive ability assessment sub-model corresponding to the training scenario in the cognitive ability assessment model according to the training scenario;

[0128] S55. Input the multimodal physiological signal features into the cognitive ability assessment sub-model to obtain the post-training cognitive ability assessment results.

[0129] Among them, the scene discrimination model is built based on the encoder structure of the Transformer model;

[0130] The cognitive ability assessment model includes n cognitive ability assessment sub-models; the cognitive ability assessment sub-models are constructed based on the encoder structure of the Transformer model; n is the total number of preset training scenarios.

[0131] The present invention proposes a group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, which can comprehensively and accurately reflect the physiological state of the subject by combining EEG signals, myoelectric signals and eye movement signals. The fusion of multimodal data can effectively improve the accuracy of cognitive ability assessment, overcome the limitations of single signal assessment, and provide more detailed and comprehensive assessment results. Through the group cognitive training model, interaction and collaboration between subjects are promoted, the diversity and fun of training are enhanced, and the enthusiasm of subjects to participate is stimulated. Team training can provide richer cognitive training tasks, improve the comprehensiveness of training, and overcome the problem of insufficient motivation caused by the lack of interaction and team goals in single-person training. The cross-scenario and cross-subject cognitive ability assessment algorithm can accurately assess the cognitive ability levels of different users in different cognitive training scenarios, and improve the applicability and generalization of the cognitive ability assessment algorithm model. The present invention is a group cognitive training method based on BCI with high training efficiency, high generalization degree and strong applicability in multiple scenarios.

[0132] Figure 3 is a structural diagram of a group multi-scenario cognitive device provided by an embodiment of the present invention, such as Figure 3 As shown, the group multi-scenario cognitive device may include the above Figure 2 The group multi-scenario cognitive training device based on MR and multimodal brain-computer interface is shown. Optionally, the group multi-scenario cognitive device 310 may include a first processor 2001.

[0133] Optionally, the group multi-scenario recognition device 310 may further include a memory 2002 and a transceiver 2003 .

[0134] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0135] The following combination Figure 3 The components of the group multi-scenario recognition device 310 are described in detail:

[0136] The first processor 2001 is the control center of the group multi-scenario recognition device 310 and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), or application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0137] Optionally, the first processor 2001 can perform various functions of the group multi-scenario cognitive device 310 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0138] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in FIG.

[0139] In a specific implementation, as an embodiment, the group multi-scenario recognition device 310 may also include multiple processors, such as Figure 3 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0140] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0141] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and accessed through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0142] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0143] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 3 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0144] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0145] It should be noted that Figure 3 The structure of the group multi-scenario recognition device 310 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0146] In addition, the technical effects of the group multi-scenario cognitive device 310 can refer to the technical effects of the group multi-scenario cognitive training method based on MR and multimodal brain-computer interface described in the above method embodiment, and will not be repeated here.

[0147] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0148] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0149] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0150] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0151] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0152] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0153] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0154] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0155] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0156] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0157] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0158] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0159] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A group multi-scenario cognitive training method based on MR and multimodal brain-computer interface, characterized in that: The method comprises: S1. Conduct cognitive ability assessment based on the cognitive ability assessment scale to obtain cognitive ability assessment results; S2. Based on the cognitive ability database, the preset group size, and the candidate group database, cognitive ability matching is performed according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training groups, and competitive training groups; The method of performing cognitive ability matching based on the cognitive ability database, the preset group size, and the candidate group database according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training groups, and competitive training groups includes: S21. Based on the cognitive ability database and the preset group size, according to the cognitive ability assessment results, a greedy strategy based on complementarity evaluation is used to perform cognitive ability matching to obtain a cooperative training group; S22. Based on the cognitive ability database and the candidate group database, perform cognitive ability matching according to the cooperative training group to obtain a group cognitive ability vector and a competitive training group; S3. Based on the cognitive task library, perform task matching according to the group cognitive ability vector to obtain matching cognitive tasks; S4. In an MR scenario, performing group cognitive training based on the cooperative training group, the competitive training group, and the cognitive task, and performing signal acquisition to obtain multimodal physiological signals; S5. Perform cognitive ability assessment using a cognitive ability assessment model based on the multimodal physiological signals to obtain a post-training cognitive ability assessment result.

2. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 1 is characterized in that: The method of matching cognitive abilities based on the cognitive ability database and the preset group size according to the cognitive ability assessment results using a greedy strategy based on complementarity evaluation to obtain a cooperative training group includes: S211, constructing a current training group based on the corresponding users of the cognitive ability assessment results; S212: Calculate the average cognitive ability of the current training group according to the cognitive ability assessment results to obtain a group cognitive ability mean; S213: Calling a cognitive ability database to obtain a set of candidate user cognitive ability assessment results; S214: Calculate the difference based on the candidate user cognitive ability assessment results and the group cognitive ability mean to obtain a candidate user difference set; S215. Based on the complementarity evaluation, select a candidate user corresponding to the maximum value in the candidate user difference set; and add the candidate user to the current training group; S216. Repeat steps S212-S215. When the number of group members reaches the preset number of group members, the current training group is determined as a cooperative training group.

3. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 1 is characterized in that: The method of performing cognitive ability matching based on the cooperative training group based on the cognitive ability database and the candidate group database to obtain the group cognitive ability vector and the competitive training group includes: S221. Based on the cognitive ability database, perform cognitive ability evaluation calculation according to the cooperative training group and the ability assessment results to obtain a group cognitive ability vector; S222: Call the candidate group database to obtain a candidate group set; S223. Based on the cognitive ability database, similarity calculation is performed according to the candidate group set and the group cognitive ability vector to obtain a candidate group cognitive ability similarity set; S224. Select the candidate group corresponding to the minimum value in the candidate group cognitive ability similarity set as the competitive training group.

4. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 1 is characterized in that: The step of performing task matching based on the cognitive task library and the group cognitive ability vector to obtain matching cognitive tasks includes: S31. Calling the cognitive task library to obtain the cognitive task difficulty level; S32, calculating a matching score based on the group cognitive ability vector and the cognitive task difficulty level to obtain a matching score set; S33: Select the cognitive task corresponding to the maximum matching score in the matching score set as the matching cognitive task.

5. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 1 is characterized in that: The cognitive ability assessment is performed using a cognitive ability assessment model based on the multimodal physiological signals to obtain a post-training cognitive ability assessment result: S51, preprocessing the multimodal physiological signal to obtain a processed physiological signal; S52, performing feature extraction on the processed physiological signal to obtain multimodal physiological signal features; S53, inputting the multimodal physiological signal features into a scene discrimination model to obtain a training scene; S54. Selecting a cognitive ability assessment sub-model corresponding to the training scenario in the cognitive ability assessment model according to the training scenario; S55. Input the multimodal physiological signal features into the cognitive ability assessment sub-model to obtain a post-training cognitive ability assessment result.

6. The group multi-scenario cognitive training method based on MR and multimodal brain-computer interface according to claim 5 is characterized in that: The scene discrimination model is constructed based on the encoder structure of the Transformer model; The cognitive ability assessment model includes n cognitive ability assessment sub-models; the cognitive ability assessment sub-models are constructed based on the encoder structure of the Transformer model; and n is the total number of preset training scenarios.

7. A group multi-scenario cognitive training device based on MR and a multimodal brain-computer interface, wherein the group multi-scenario cognitive training device based on MR and a multimodal brain-computer interface is used to implement the group multi-scenario cognitive training method based on MR and a multimodal brain-computer interface as described in any one of claims 1 to 6, characterized in that: The device comprises: A pre-training ability assessment module is used to conduct cognitive ability assessment based on a cognitive ability assessment scale and obtain cognitive ability assessment results; A group matching module is used to perform cognitive ability matching based on the cognitive ability database, the preset group size and the candidate group database according to the cognitive ability assessment results to obtain group cognitive ability vectors, cooperative training groups and competitive training groups; A cognitive task matching module, configured to perform task matching based on the cognitive task library and the group cognitive ability vector to obtain matching cognitive tasks; a signal acquisition module, configured to perform group cognitive training based on the cooperative training group, the competitive training group, and the cognitive task, and to acquire signals to obtain multimodal physiological signals; The post-training ability assessment module is used to perform cognitive ability assessment based on the multimodal physiological signals through a cognitive ability assessment model to obtain a post-training cognitive ability assessment result.

8. A group multi-scenario cognitive device, characterized in that: The group multi-scenario cognitive device includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 6.

Images