Human cognitive ability adaptive training method and device based on multi-modal data, edge computing device and storage medium

Abstract

The present disclosure provides a user cognitive ability adaptive training method and device based on multi-modal data, an edge computing device and a storage medium, relating to the technical field of cognitive training. The method obtains multi-modal data of a user, obtains a cognitive assessment result of the user based on the multi-modal data, and obtains a matched cognitive ability training scheme based on the cognitive assessment result. In the process of the user's cognitive ability training, a loop adjustment process is executed: in the training process of executing the nth cognitive ability training task, real-time physiological feedback data and real-time training performance data of the user are obtained to adjust at least one of the difficulty and the task type of the n+1 cognitive ability training task; n is updated to n+1, and the adjustment process is continued until a preset training termination condition is met. By dynamically adjusting the cognitive ability training scheme, the adaptability of the cognitive ability training scheme is improved, so that the user's cognitive training can always be in the optimal intensity interval.

Description

Technical Field

[0001] This disclosure relates to the field of cognitive assessment and training technology, and in particular to an adaptive training method, apparatus, edge computing device and storage medium for human cognitive abilities based on multimodal data. Background Technology

[0002] With the development of society, more and more people are experiencing cognitive impairment. Therefore, in order to improve people's cognitive impairment, cognitive training methods are usually adopted to carry out targeted cognitive training such as attention and memory, so as to gradually improve and enhance cognitive abilities.

[0003] Currently, most traditional cognitive training methods employ fixed-difficulty training approaches, meaning they utilize cognitive ability training programs with a fixed level of difficulty. However, this type of cognitive ability training program has significant limitations; the training difficulty is often fixed, failing to consider individual differences and exhibiting poor adaptability. Summary of the Invention

[0004] The technical problem this disclosure aims to solve is: how to make the difficulty of cognitive ability training programs adaptable to users in order to meet their personalized training needs.

[0005] In a first aspect, to address the aforementioned technical problems, embodiments of this disclosure provide an adaptive training method for human cognitive abilities based on multimodal data, the method comprising: Acquire user multimodal data; The user's cognitive assessment result is determined based on the multimodal data; Based on the cognitive assessment results, a matching cognitive ability training program is obtained. The cognitive ability training program includes multiple cognitive ability training tasks with adjustable task attributes, including difficulty and task type. The user undergoes cognitive ability training according to the aforementioned cognitive ability training scheme, wherein: during the cognitive ability training process, the following adjustment procedure is executed cyclically: During the training process of the nth cognitive ability training task among the multiple cognitive ability training tasks, the user's real-time physiological feedback data and real-time training performance data are obtained, where n is a positive integer; based on the real-time physiological feedback data and the real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until the preset training termination condition is met.

[0006] In some embodiments, adjusting at least one of the difficulty and task type of the (n+1)th cognitive ability training task based on the real-time physiological feedback data and the real-time training performance data includes: Based on the real-time physiological feedback data, the degree of user attention distraction is determined; If the degree of attention distraction is greater than the first threshold, then the difficulty of the (n+1)th cognitive ability training task is reduced. If the degree of attention distraction is less than or equal to the first threshold, and the real-time training performance is less than the lower limit of the preset target range, then the task type of the (n+1)th cognitive ability training task is adjusted. If the degree of attention distraction is less than or equal to the first threshold, and the real-time training performance exceeds the lower limit of the preset target range, then the difficulty of the (n+1)th cognitive ability training task is increased.

[0007] In some embodiments, the real-time physiological feedback data includes: electroencephalogram (EEG) signals; determining the user's level of attention distraction based on the real-time physiological feedback data includes: The EEG data is processed using a built-in attention coefficient estimation model to generate the user's attention coefficient, which is used to characterize the degree of the user's attentional distraction.

[0008] In some embodiments, the method further includes: If the duration of the attention distraction exceeding the first threshold is greater than a preset duration, and the difficulty of the (n+1)th cognitive ability training task is reduced to a preset minimum difficulty, then the corresponding training control strategy is executed according to the user's training status. When the training state includes a fatigue level greater than a first fatigue level and less than or equal to a second fatigue level, a first training control strategy is executed to guide the user to rest and recover. When the training state includes abnormal emotional state and / or training willingness value is less than the first willingness value but greater than or equal to the second willingness value, a second training regulation strategy is executed to provide training assistance intervention. When the training state includes a fatigue level greater than the second fatigue level, and / or a state of severe emotional disorder, and / or a training willingness value less than the second willingness value, the cognitive ability training of the user shall be terminated. Wherein, the first fatigue level is less than the second fatigue level, and the first willingness value is greater than the second willingness value.

[0009] In some embodiments, when the degree of attention distraction is greater than a first threshold, the method further includes: Obtain the user's attention distraction type, which is either fatigue-related or distraction-related. If the type of attention distraction is fatigue-related, stop the cognitive training for the user and guide the user to relax and recover. When the attention distraction type is the interference type, adjust the task context of the nth cognitive training task.

[0010] In some embodiments, the cognitive assessment result includes: cognitive scores in multiple cognitive dimensions; and based on the cognitive assessment result, obtaining a matching cognitive ability training plan includes: Based on the cognitive scores of each of the aforementioned cognitive dimensions, the target cognitive dimension of cognitive impairment is determined; Obtain the degree of impairment value of the target cognitive dimension; Based on the degree of impairment of the target cognitive dimension, multiple cognitive ability training tasks with training difficulty matching the degree of impairment are obtained to obtain a preliminary solution; Based on the preliminary plan, a matching cognitive ability training plan is obtained.

[0011] In some embodiments, based on the preliminary scheme, a matching cognitive ability training scheme is obtained, including: The initial scheme is optimized using at least one of the following optimization strategies to obtain a matching cognitive ability training scheme, wherein the multiple optimization strategies include: Optimization Strategy 1: When there are multiple target cognitive dimensions, for each target cognitive dimension, based on the correlation between the target cognitive dimensions, optimize at least one of the number of cognitive ability training tasks corresponding to the target cognitive dimension and the duration of a single training session in the preliminary scheme. The correlation is the correlation between the degree of damage of the target cognitive dimension and the recovery speed of other target cognitive dimensions. Optimization Strategy 2: Based on the user's personalized difficulty progression path under the target cognitive dimension, optimize at least one of the following in the preliminary solution: the number of cognitive ability training tasks corresponding to the target cognitive dimension, the duration of a single training session, and the difficulty. Optimization Strategy 3: Based on the user's optimal training time window, optimize the training period of the cognitive ability training task corresponding to the target cognitive dimension in the preliminary solution; Optimization Strategy 4: Given that the user has undergone historical cognitive training, based on the cognitive recovery rate of the user's target cognitive dimension, optimize at least one of the difficulty and difficulty progression step size of the cognitive ability training task corresponding to the target cognitive dimension in the preliminary scheme.

[0012] In some embodiments, the cognitive assessment results are obtained based on multiple sets of multimodal data from the user within a preset time period; before optimizing the initial scheme using at least one of the following optimization strategies, the method further includes: Obtain the standard deviation of multiple initial scores for the target cognitive dimension, each initial score being obtained based on a set of the multimodal data; If the standard deviation of the multiple initial scores is greater than the standard deviation threshold, increase the training frequency of the cognitive ability training task corresponding to the target cognitive dimension in the preliminary scheme, or extend the single training duration of the cognitive ability training task corresponding to the target cognitive dimension.

[0013] In some embodiments, before obtaining multiple cognitive ability training tasks whose training difficulty matches the impairment value based on the impairment value of the target cognitive dimension, the method further includes: When the consistency of multiple data sets is higher than the consistency threshold, the norm parameters corresponding to the user under the target cognitive dimension are obtained. The norm parameters include the norm mean and the norm standard deviation. The multiple data sets are used to calculate the cognitive score of the target cognitive dimension. Based on the norm parameters and the cognitive score of the target cognitive dimension, the degree of impairment of the target cognitive dimension is determined.

[0014] In some embodiments, the multimodal data includes: subjective data and objective data; The subjective data is obtained based on the user's subjective evaluation results; the objective data includes one or more of the following: real-time physiological feedback data, eye movement data, and behavioral data.

[0015] In some embodiments, the method further includes: Obtain the target training mode of the user's preference, wherein the target training mode is a 2D mode or a virtual reality (VR) mode; The cognitive ability training scheme is loaded according to the target training mode for the user to perform cognitive training.

[0016] In some embodiments, the cognitive ability training task is constructed based on a psychomotor vigilance paradigm, a GO / NO-GO paradigm, or a combination of the psychomotor vigilance paradigm and the GO / NO-GO paradigm.

[0017] Secondly, embodiments of this disclosure provide an adaptive training device for human cognitive abilities based on multimodal data, the device comprising: The first acquisition module is used to acquire the user's multimodal data; A determination module is used to determine the user's cognitive assessment result based on the multimodal data; The second acquisition module is used to acquire a matching cognitive ability training scheme based on the cognitive assessment results. The cognitive ability training scheme includes multiple cognitive ability training tasks with adjustable task attributes, and the task attributes include: difficulty and task type. The training module is used to train the user's cognitive abilities according to the cognitive ability training scheme, wherein: during the user's cognitive ability training, the following adjustment process is executed cyclically: during the training of the nth cognitive ability training task among the plurality of cognitive ability training tasks, the user's real-time physiological feedback data and real-time training performance data are obtained, where n is a positive integer; based on the real-time physiological feedback data and the real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues to be executed until a preset training termination condition is met.

[0018] Thirdly, embodiments of this disclosure provide an edge computing device, including: a processor and a memory, wherein the memory stores a computer program that can run on the processor, and the computer program, when executed by the processor, implements the adaptive training method for human cognitive abilities based on multimodal data as described in any of the first aspects above.

[0019] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the adaptive training method for human cognitive abilities based on multimodal data as described in any of the first aspects above.

[0020] As can be seen from the above technical solution, this disclosure provides a method, apparatus, edge computing device, and storage medium for adaptive training of human cognitive abilities based on multimodal data. This method can acquire the user's multimodal data, obtain the user's cognitive assessment results based on the multimodal data, obtain a matching cognitive ability training scheme based on the cognitive assessment results, and train the user's cognitive abilities according to the cognitive ability training scheme. During the training process of the user's nth cognitive ability training task in the cognitive ability training scheme, real-time physiological feedback data and real-time training performance data of the user are acquired, and at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted based on the real-time physiological feedback data and real-time training performance data. Because the cognitive ability training scheme can be dynamically adjusted according to the user's real-time physiological feedback data and real-time training performance data during the cognitive ability training process, it can ensure that the adjusted cognitive ability training scheme is accurately adapted to the user's real-time training performance data and real-time physiological feedback data, improving the adaptability of the cognitive ability training scheme. This ensures that the user's cognitive training is always in the optimal intensity range, maximizing training benefits and thus meeting the user's training needs. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating an adaptive training method for human cognitive abilities based on multimodal data, as disclosed in an embodiment of this disclosure. Figure 2 This is a flowchart illustrating another adaptive training method for human cognitive abilities based on multimodal data disclosed in this embodiment. Figure 3 This is a schematic diagram of a cognitive score for a cognitive dimension disclosed in an embodiment of this disclosure; Figure 4 This is a schematic flowchart of a cognitive ability training scheme disclosed in an embodiment of this disclosure; Figure 5 This is a schematic diagram of adjusting the (n+1)th cognitive ability training task disclosed in an embodiment of this disclosure; Figure 6 This is a flowchart illustrating another adaptive training method for human cognitive abilities based on multimodal data disclosed in this embodiment. Figure 7 This is a schematic diagram of the structure of an adaptive training device for human cognitive ability based on multimodal data disclosed in this embodiment; Figure 8 This is a schematic diagram of the structure of an edge computing device disclosed in an embodiment of this disclosure. Detailed Implementation

[0023] The embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings and examples. The detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of this disclosure by way of example, but should not be used to limit the scope of this disclosure. This disclosure can be implemented in many different forms and is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

[0024] These embodiments are provided to make the disclosure thorough and complete, and to fully express the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specifically stated, the relative arrangement of components and steps, material composition, numerical expressions, and values ​​set forth in these embodiments should be interpreted as exemplary only and not as limiting.

[0025] All terms used in this disclosure have the same meaning as understood by one of ordinary skill in the art to which this disclosure pertains, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant art, and not as idealized or highly formalized, unless expressly defined herein.

[0026] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0027] The physiological data involved in this disclosure are various measurable data signals in the human body, including but not limited to electrocardiogram (ECG) signals, skin temperature (SKT) signals, photoplethysmogram (PPG) signals, electrodermal activity (EDA) signals, heart rate (HR) signals, electromyogram (EMG) signals, electroencephalogram (EEG) signals, and peripheral capillary oxygen saturation (SPO2) signals.

[0028] With the development of society, more and more people are experiencing cognitive impairment. Therefore, in order to improve people's cognitive impairment, cognitive training methods are usually adopted to carry out targeted cognitive training such as attention and memory, so as to gradually improve and enhance cognitive abilities.

[0029] Currently, most traditional cognitive training methods employ fixed-difficulty training approaches, meaning they utilize cognitive ability training programs with a fixed level of difficulty. However, this type of cognitive ability training program has significant limitations. The training difficulty is often fixed, failing to consider individual differences and exhibiting poor adaptability, thus failing to meet the training needs of individuals with cognitive impairments.

[0030] For example, if the difficulty is too high, users will frequently experience frustration and lose confidence during training, making it difficult to persist in the long term; if the difficulty is too low, it will not effectively stimulate cognitive functions and will be difficult to achieve the expected training results.

[0031] In view of this, embodiments of this disclosure provide a method, apparatus, edge computing device, and storage medium for adaptive training of human cognitive abilities based on multimodal data. The method can acquire a user's multimodal data, obtain the user's cognitive assessment results based on the multimodal data, obtain a matching cognitive ability training scheme based on the cognitive assessment results, and train the user's cognitive abilities according to the cognitive ability training scheme. During the user's cognitive ability training, the following adjustment process is executed cyclically: during the training of the nth cognitive ability training task among multiple cognitive ability training tasks, real-time physiological feedback data and real-time training performance data of the user are acquired; based on the real-time physiological feedback data and real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until a preset training termination condition is met. Because the cognitive ability training program can be dynamically adjusted based on the user's real-time physiological feedback data and real-time training performance data during the cognitive ability training process, it can ensure that the adjusted cognitive ability training program is accurately matched with the user's real-time training performance data and real-time physiological feedback data. This improves the adaptability of the cognitive ability training program, thereby ensuring that the user's cognitive training is always in the optimal intensity range, maximizing the training benefits to the greatest extent, and thus meeting the user's training needs.

[0032] The adaptive training method for human cognitive ability based on multimodal data provided in this disclosure will now be described in detail with reference to the accompanying drawings. Figure 1 This disclosure provides an adaptive training method for human cognitive abilities based on multimodal data. The method is applied to an edge computing device, such as a processor within the edge computing device. Optionally, the edge computing device can be an edge computing terminal. For details, see [link to relevant documentation]. Figure 1 The method includes: Step 101: Obtain the user's multimodal data.

[0033] Multimodal data can include at least subjective data, and may also include both subjective and objective data. Subjective data is based on the user's subjective assessment results, such as scales and question-and-answer assessments. Objective data includes one or more of the following: multimodal physiological data, eye-tracking data, and behavioral data.

[0034] Subjective data can reflect a user's own perception and feedback on their cognitive state, while objective data can accurately capture quantitative indicators such as physiological and behavioral aspects of the user's cognitive process.

[0035] Step 102: Obtain the user's cognitive assessment results based on the user's multimodal data.

[0036] The cognitive assessment results include: cognitive scores for each of the multiple cognitive dimensions, that is, the user's cognitive scores for each of the multiple cognitive dimensions. Different cognitive dimensions correspond to different cognitive functions.

[0037] Since multimodal data includes both subjective and objective data, integrating users' subjective and objective data for cognitive assessment can avoid distortion of assessment results caused by bias in a single data point, thereby ensuring that the determined cognitive assessment results have high reliability and objectivity, and can truly and objectively reflect the user's actual cognitive level.

[0038] Step 103: Based on the cognitive assessment results, obtain a matching cognitive ability training plan.

[0039] The cognitive ability training program may include: at least one training content, each corresponding to a target cognitive dimension that requires cognitive training. Each training content may include: multiple cognitive ability training tasks with adjustable task attributes. Task attributes include: difficulty and task type.

[0040] Step 104: Conduct cognitive ability training for the user according to the cognitive ability training plan. During the cognitive ability training process, the following adjustment process is executed cyclically: During the training of the nth cognitive ability training task, real-time physiological feedback data and real-time training performance data of the user are obtained, where n is a positive integer; Based on the real-time physiological feedback data and real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until the preset training termination condition is met.

[0041] Where n is an integer greater than or equal to 1 and less than or equal to N. N is the total number of cognitive ability training tasks. Training termination conditions include: training end or stop. Training stop occurs when cognitive training ends during the execution of cognitive training tasks, and any unexecuted cognitive training tasks are no longer executed.

[0042] In other words, starting from the first cognitive training task performed by the user, the edge computing device can adjust at least one of the difficulty and task type of the next cognitive training task based on the user's real-time physiological feedback data and real-time training performance data during the execution of each cognitive training task, until the training ends or stops. The real-time physiological feedback data includes at least EEG data.

[0043] In summary, this disclosure provides an adaptive training method for human cognitive abilities based on multimodal data. This method acquires a user's multimodal data, obtains the user's cognitive assessment results based on the multimodal data, acquires a matching cognitive ability training plan based on the cognitive assessment results, and trains the user's cognitive abilities according to the cognitive ability training plan. During the user's cognitive ability training, the following adjustment process is executed iteratively: during the training of the nth cognitive ability training task, real-time physiological feedback data and real-time training performance data of the user are acquired; based on the real-time physiological feedback data and real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until a preset training termination condition is met. Because the cognitive ability training program can be dynamically adjusted based on the user's real-time physiological feedback data and real-time training performance data during the cognitive ability training process, it can ensure that the adjusted cognitive ability training program is accurately matched with the user's real-time training performance data and real-time physiological feedback data. This improves the adaptability of the cognitive ability training program, thereby ensuring that the user's cognitive training is always in the optimal intensity range, maximizing the training benefits to the greatest extent, and thus meeting the user's training needs.

[0044] Figure 2 A flowchart illustrating another adaptive training method for human cognitive abilities based on multimodal data, provided as an embodiment of this disclosure. This method can be applied to edge computing devices. For details, see [link to relevant documentation]. Figure 2 The method may include: Step 201: Obtain the user's multimodal data.

[0045] During cognitive testing, edge computing devices can acquire multimodal data from users. This multimodal data can include both subjective and objective data. Subjective data is based on the user's subjective assessment results, which can take the form of questionnaires, scales, and paradigms. Questionnaires (or scales, paradigms) can be standard or custom-designed, allowing users to choose flexibly based on their usage scenario. For example, standard scales can include the Mini-Mental State Examination (MMSE) and the Hamilton Depression Rating Scale (HAMD).

[0046] Objective data includes one or more of the user's multimodal physiological data, eye-tracking data, and behavioral data. Multimodal physiological data may include electroencephalogram (EEG) data and heart rate data, etc. Eye-tracking data may include reaction time, fixation point, and fixation duration, etc. Behavioral data may include facial expressions, accuracy rate, and reaction time, etc. Facial expressions may include neutral, happy, or sad, etc.

[0047] Taking subjective data obtained through scale responses as an example, the process of acquiring subjective data is illustrated below: Edge computing devices have multiple built-in scales covering neurocognitive function assessment, mental symptom assessment, daily living skills and behavior assessment, and comprehensive clinical assessment. Each scale includes: identifiers (IDs) for each item, the question stem text, and an array of options. Of course, edge computing devices can also have built-in score mapping tables or rule functions for the scales. After the edge computing device is activated, it can read multiple scales and their corresponding score mapping tables or rule functions into memory at once, forming a unified question bank array for subsequent cognitive testing of users.

[0048] When conducting cognitive tests on users, edge computing devices can load a questionnaire and display the questions within it. Users can then answer the questions. After the user completes and submits their answers, the edge computing device can automatically output the user's subjective data based on the score mapping table or rule function corresponding to the questionnaire, and simultaneously clear the locally cached answer data to prevent duplicate submissions. The subjective data can include: total score, number of questions completed, and total time spent.

[0049] The question types include both regular and special questions. Regular questions can include multiple-choice questions, while special questions can include HAMD question 14, MMSE drawing / continuous subtraction of 7 questions, etc. The score mapping table for regular questions records the score corresponding to the "selected index" of the question.

[0050] Taking objective data, including EEG data, eye-tracking data, and facial expressions, as examples, the acquisition of objective data is illustrated below: Edge computing devices are equipped with EEG acquisition devices for collecting EEG data, eye-tracking acquisition devices for collecting eye-tracking data, and cameras for capturing facial images of the user. It should be noted that in VR cognitive assessment scenarios, edge computing devices are paired with VR headsets, and the eye-tracking acquisition devices can be integrated into the VR headsets.

[0051] Assuming a cognitive assessment is conducted in a VR cognitive evaluation scenario, before the cognitive assessment is carried out, the user can first complete the wearing and adjustment of the accompanying EEG acquisition device, as well as the wearing of the accompanying VR headset device that integrates eye-tracking acquisition devices (such as eye-tracking sensors), to ensure stable device connection.

[0052] During cognitive testing, edge computing devices load preset experimental paradigms to ensure the standardization and validity of the collected data. EEG acquisition devices, eye-tracking acquisition modules, and cameras work synchronously to collect high-precision EEG data, eye-tracking data, and facial images, respectively. The collected multimodal data is transmitted in real time to the edge computing devices for local processing.

[0053] Edge computing devices can receive and store eye-tracking data, and use built-in expression classification models to process captured facial images in real time, accurately outputting the user's facial expressions and simultaneously writing them into a behavior log. For EEG data, edge computing devices can monitor signal quality in real time to ensure the signal-to-noise ratio meets standards. Furthermore, edge computing devices can perform bandpass filtering on the received raw EEG data to remove power frequency interference and baseline drift before inputting it into a pre-trained deep learning model. This model automatically identifies and removes artifacts (such as EEG and EMG artifacts), achieving effective enhancement of EEG signals and intelligent completion of missing segments. Based on this, power spectral density of key frequency bands can be extracted according to the international 10-20 system electrode partitioning, focusing on calculating the relative energy ratio of alpha waves (8–13Hz) to beta waves (13–30Hz), and deriving quantitative indicators such as attention index and relaxation index. These indicators objectively reflect the user's brain activation state and cognitive load level under different cognitive tasks, providing crucial physiological evidence for subsequent cognitive assessment and adaptive training regulation. The deep learning model can be a CNN-BiLSTM hybrid network, etc.

[0054] Optionally, the EEG acquisition device can be a high-density 64-lead wireless EEG cap. This wireless EEG cap uses a standard lead system covering the entire scalp, enabling high spatiotemporal resolution acquisition of electrical activity in the cerebral cortex.

[0055] When collecting eye-tracking data, the edge computing device needs to load an eye-tracking experimental paradigm. The eye-tracking experimental paradigms provided in this disclosure can include: a basic eye-tracking paradigm, a cognitive and reading paradigm, and a visual attention and salience paradigm. The basic eye-tracking paradigm includes: a random test point paradigm and a free-viewing paradigm. In the random test point paradigm, the screen sequentially illuminates target points at fixed locations, and the user needs to gaze at the target points to complete the calibration. In the free-viewing paradigm, the screen displays a scene map, at which time the gaze sequence and duration of the object are recorded.

[0056] Cognitive and reading paradigms include: the moving window paradigm, the boundary paradigm, and the disappearing text paradigm. In the moving window paradigm, only the "visual window" centered on the current gaze point is retained, while the rest of the text is masked; the window moves in real-time with eye saccades. In the boundary paradigm, keywords are replaced instantaneously before the eye saccades cross the boundary, used to measure the preview effect. In the disappearing text paradigm, the entire line of text disappears when the gaze point remains fixed, and the number of bounces during rereading is recorded.

[0057] Visual attention and saliency paradigms include: the visual-contextual paradigm, the Landolt-C paradigm, and the visual crowding paradigm. In the visual-contextual paradigm, a target object is embedded within a natural scene, and the gaze time of the user's first glance at the object is recorded. In the Landolt-C paradigm, rings with different opening orientations are randomly presented, and the user identifies and indicates the opening direction; the system records the accuracy and average saccade latency. In the visual crowding paradigm, the spacing between the target and the distracting grid is progressively reduced to quantify the spatial resolution limit.

[0058] Step 202: Obtain the user's cognitive assessment results based on the user's multimodal data.

[0059] Cognitive assessment results include cognitive scores for each of multiple cognitive dimensions, that is, the user's cognitive scores for each of these dimensions. Different cognitive dimensions correspond to different cognitive functions. For example, these multiple cognitive dimensions include at least: executive function, memory, attention, thinking speed, and language ability.

[0060] Optionally, edge computing devices can obtain a user's cognitive assessment results based on a set of multimodal data. This set of multimodal data can be the multimodal data obtained after the user's most recent cognitive assessment.

[0061] Alternatively, edge computing devices can obtain a user's cognitive assessment results based on multiple sets of multimodal data. These multiple sets of multimodal data are obtained after the user performs multiple cognitive assessments within a preset time period. The end time of the preset time period can be the time when the multiple sets of multimodal data are obtained. In other words, edge computing devices can obtain a user's cognitive assessment results based on the most recent multiple sets of multimodal data obtained after the user has performed multiple cognitive assessments recently.

[0062] The duration of the preset time period can be less than a time threshold, and the duration of the preset time period can be pre-stored by the edge computing device, for example, it can be 5 days, 6 days, or 7 days. Setting the duration of the preset time period to a shorter length can ensure that the obtained multimodal data can reflect the user's current cognitive status to a greater extent.

[0063] Obtaining cognitive assessment results from multiple sets of multimodal data from users ensures high reliability and a more accurate representation of their actual cognitive state. Consequently, training programs based on these assessments are more targeted and effective in training users.

[0064] Here's a simplified explanation of how edge computing devices acquire user cognitive assessment results based on multiple sets of multimodal data: The edge computing device can obtain an initial score for each cognitive dimension of the user based on each set of multimodal data, resulting in multiple initial scores for each cognitive dimension. These initial scores correspond one-to-one with the multiple sets of multimodal data. Subsequently, for each cognitive dimension, the edge computing device can perform time-weighted fusion of these multiple initial scores to obtain the cognitive score for that dimension.

[0065] The time weight of each initial score in the time-weighted fusion process is inversely proportional to the interval between the test time corresponding to that initial score and the acquisition time of multiple sets of multimodal data. That is, the shorter the interval, the closer the corresponding test time, and the higher the time weight of the initial score. The test time corresponding to the initial score refers to the moment when the user begins the cognitive assessment to obtain the multimodal data used to calculate that initial score.

[0066] Specifically, for each cognitive dimension, assuming the total number of initial scores is... I , I The first initial score i Initial score s i Time weight w i for Where e is the natural constant, λ The attenuation coefficient is... t now In order to obtain the time, t i For the first i Initial score s i The corresponding test time. Then, the cognitive score for this cognitive dimension. S It can satisfy: .

[0067] Regarding the implementation method of edge computing devices obtaining the cognitive assessment results of users based on a single set of multimodal data, the edge computing devices can directly determine the initial scores of each cognitive dimension obtained based on this single set of multimodal data as the user's cognitive score in the corresponding cognitive dimension.

[0068] The following is an example illustrating how edge computing devices obtain initial scores for users across various cognitive dimensions based on multimodal data: For each cognitive dimension, edge computing devices can adopt an adaptive weighted fusion strategy to determine the target weights of various data (such as subjective data, EEG data, eye-tracking data, and behavioral data) in multimodal data, and then perform weighted fusion of these various data based on their target weights to obtain the user's initial score for that cognitive dimension.

[0069] An adaptive weighted fusion strategy is used to dynamically assign weights to various data, which ensures a more accurate initial score compared to using fixed weights.

[0070] The following is an illustrative example of the process of determining the target weights of various data in multimodal data using an adaptive weighted fusion strategy: The edge computing device can acquire the initial weights of various data, as well as the signal quality of the data during the data acquisition process. Then, for each data, the edge computing device can dynamically adjust the initial weight of the data based on the signal quality to obtain the target weight. Specifically, if the signal quality of the data is lower than a quality threshold, the initial weight is reduced to obtain the target weight, or the target weight of the data is determined to be 0. If the signal quality of the data is higher than or equal to the quality threshold, the initial weight is maintained as the target weight.

[0071] The signal quality of each data source can be output from the acquisition device (i.e., the EEG acquisition device, eye-tracking acquisition module, and camera mentioned above) to the edge computing device. The initial weights of various data can be pre-stored by the edge computing device. For example, they can be determined based on the correlation between a large amount of historical cognitive scores and various data.

[0072] Optionally, for scenarios with high computational efficiency requirements, edge computing devices can pre-deploy a pre-trained lightweight fusion model (such as a multimodal feature fusion network based on an attention mechanism) locally. In this way, the edge computing device can directly output the initial scores of each cognitive dimension through the model, reducing real-time computation overhead.

[0073] Optionally, after receiving the user's cognitive assessment results, the edge computing device can also display these results for the user to view. For example, assuming multiple cognitive dimensions include executive function, memory, attention, thinking speed, and language ability, then see... Figure 3 Edge computing devices can display users' cognitive assessment results in the form of a five-dimensional assessment radar chart, so that users can intuitively view their cognitive scores in each cognitive dimension.

[0074] In this embodiment of the application, before obtaining the user's cognitive assessment results based on multimodal data, the edge computing device can also perform preprocessing on various data in the user's multimodal data. This preprocessing includes at least timestamp alignment, normalization, and sliding window processing of physiological data.

[0075] It is understandable that each cognitive dimension includes multiple sub-dimensions. For example, the executive function dimension can include sub-dimensions such as task switching and inhibitory control. Memory can include sub-dimensions such as working memory and short-term memory. Attention dimension can include sub-dimensions such as sustained attention and selective attention. Thinking speed dimension can include sub-dimensions such as information processing efficiency. Language ability dimension can include sub-dimensions such as semantic comprehension and fluency of expression.

[0076] Step 203: Based on the user's cognitive assessment results, obtain a matching cognitive ability training plan.

[0077] The following is combined with Figure 4 The implementation process of step 203 is illustrated by an example, which may include: Step 2031: Based on the cognitive scores of each cognitive dimension, determine the target cognitive dimension of cognitive impairment.

[0078] Edge computing devices can identify target cognitive dimensions that are cognitively impaired, i.e., weak cognitive dimensions, based on the cognitive scores of each cognitive dimension across multiple cognitive dimensions. For example, edge computing devices may have preset score thresholds. For each cognitive dimension, if the edge computing device determines that the cognitive score of that dimension is less than the score threshold, then that cognitive dimension can be designated as a target cognitive dimension that is impaired. There can be one or more such target cognitive dimensions.

[0079] Step 2032: Obtain the degree of damage to the target cognitive dimension.

[0080] The damage level value refers to the degree of damage to the target cognitive dimension.

[0081] In this embodiment, when the consistency of multiple data sets exceeds a consistency threshold, the edge computing device can obtain the normative parameters corresponding to the user in the target cognitive dimension. These normative parameters include the normative mean and the normative standard deviation, and these multiple data sets participate in calculating the cognitive score of the target cognitive dimension. Subsequently, the edge computing device can determine the degree of impairment in the target cognitive dimension based on these normative parameters and the cognitive score of the target cognitive dimension.

[0082] The process by which edge computing devices acquire the consistency of various data sources involved in calculating cognitive dimensions of a target may include: First, the correlation coefficient between each pair of data points is obtained, resulting in multiple correlation coefficients. Specifically, for each pair of data points, the edge computing device can use a correlation coefficient calculation algorithm to process the two data points and obtain the correlation coefficient between them. Optionally, the correlation coefficient calculation algorithm can be the Pearson correlation coefficient calculation algorithm, and correspondingly, the correlation coefficient can be the Pearson correlation coefficient.

[0083] Subsequently, the consistency of various data types is obtained based on multiple correlation coefficients. This consistency score characterizes the agreement among the different data types, and a higher consistency score indicates stronger agreement. The consistency score is proportional to the mean of the multiple correlation coefficients. For example, the consistency score is the mean of these correlation coefficients.

[0084] Understandably, after obtaining the consistency of various data sets, the edge computing device can compare the consistency with a consistency threshold. If the edge computing device determines that the consistency is less than or equal to the consistency threshold, it can determine that the reliability of the various data sets used to evaluate the target cognitive dimension is insufficient, and the cognitive score is likely incorrect. Therefore, the cognitive assessment of that target cognitive dimension can be re-executed. The consistency threshold can be pre-stored by the edge computing device, for example, it could be 0.6.

[0085] In principle, the various data used to calculate the target cognitive dimension should have a high degree of consistency. However, factors such as environmental interference, the location and accuracy of the data acquisition equipment can lead to lower matching degrees and lower consistency among the data. When consistency is low, the determined cognitive score will deviate significantly from the actual cognitive score, or even be completely distorted. Therefore, directly using the cognitive score of the target cognitive dimension to obtain the degree of impairment, and then determining a cognitive ability training scheme based on this degree of impairment, will result in a low effectiveness of the determined cognitive ability training scheme, failing to provide effective training for the user.

[0086] Therefore, in this embodiment, the degree of impairment of the target cognitive dimension is obtained only after the consistency of multiple data sets exceeds a consistency threshold, in order to determine a cognitive ability training scheme. That is, a cognitive ability training scheme is determined only when the cognitive score is confirmed to reflect the user's actual cognitive state. This ensures that the determined cognitive ability training scheme is highly effective and allows for targeted training of the user.

[0087] In this embodiment, the edge computing device can obtain a standard score based on norm parameters and the cognitive score of the target cognitive dimension, and then obtain a damage level value for the target cognitive dimension based on the standard score. This damage level value can be represented by a damage grade or a damage score. The standard score can be a Z-score or a T-score. The Z-score satisfies: . X The user's cognitive score is represented by μ, which is the norm mean. σ The standard deviation is the norm.

[0088] Assuming the degree of damage is represented by a damage level, with a standard score of Z-score, then edge computing devices can determine the damage level based on the range of the Z-score. For example, if Z > -0.5, the target's cognitive dimension is slightly damaged, and the damage level is determined to be level 0. If -1.5 ≤ Z < -0.5, the target's cognitive dimension is mildly damaged, and the damage level is determined to be level 1. If -2.5 ≤ Z < -1.5, the target's cognitive dimension is moderately damaged, and the damage level is determined to be level 2. If Z < -2.5, the target's cognitive dimension is severely damaged, and the damage level is determined to be level 3.

[0089] In this embodiment, the edge computing device may have a built-in norm database, which is constructed using a multi-dimensional hierarchical technique. For example, the database may be grouped according to age groups (e.g., every 5 or 10 years as an interval), years of education (e.g., ≤9 years, 10-12 years, ≥13 years), and / or cognitive impairment type (e.g., mild cognitive impairment MCI, Alzheimer's disease AD, post-stroke cognitive impairment PSCI, etc.), and store the cognitive mean and cognitive standard deviation of each cognitive dimension for the corresponding population in each group.

[0090] Edge computing devices can acquire users' basic attributes (also known as demographic characteristics) and, based on these attributes, automatically match corresponding groups in a norm database. The mean and / or standard deviation of the target cognitive dimension for the population in that group are then used as the norm parameters for the user within that target cognitive dimension. These basic attributes may include age, years of education, and type of cognitive impairment.

[0091] Step 2033: Based on the degree of impairment of the target cognitive dimension, obtain multiple cognitive ability training tasks whose training difficulty matches the degree of impairment, and obtain a preliminary solution.

[0092] Edge computing devices can determine the training difficulty matching the impairment level based on the target cognitive dimension's impairment value and the mapping relationship between the impairment value and difficulty. Based on this matched training difficulty, multiple matching cognitive ability training tasks can be obtained. The training difficulty recorded in this mapping relationship can be negatively correlated with the impairment value. The preliminary scheme includes: these multiple cognitive ability training tasks. The task attributes of these multiple cognitive ability training tasks are adjustable. The task attributes include difficulty and task type.

[0093] Understandably, when there are multiple target cognitive dimensions, a preliminary solution could include: multiple training contents, each matched with a impairment value for one target cognitive dimension. Each training content includes multiple cognitive ability training tasks with adjustable matching task attributes. Training content matched with the impairment value of the target cognitive dimension means that the task type of the cognitive ability training tasks matches the target cognitive dimension, and the difficulty also matches the impairment value.

[0094] Step 2034: Based on the preliminary plan, obtain a matching cognitive ability training plan.

[0095] In this embodiment, the edge computing device can directly determine the preliminary solution as a matching cognitive ability training solution. Alternatively, the edge computing device can optimize the preliminary solution to obtain a matching cognitive ability training solution.

[0096] In this embodiment, the edge computing device may employ at least one of the following optimization strategies to optimize the initial scheme to obtain a matching cognitive ability training scheme. The optimization strategies include: Optimization Strategy 1: When there are multiple target cognitive dimensions, dynamically adjust the training resource allocation ratio for each dimension based on the correlations between them. Specifically, for each target cognitive dimension, the edge computing device can optimize at least one of the following in the initial scheme: the number of cognitive ability training tasks corresponding to that dimension and the duration of each training session. The correlation for that target cognitive dimension is the relationship between the degree of impairment of that dimension and the recovery speed of other target cognitive dimensions. The other target cognitive dimensions are those other than the target cognitive dimension mentioned above.

[0097] When the degree of damage to the target cognitive dimension is negatively correlated with the recovery speed of other target cognitive dimensions (i.e., the more severe the damage, the slower the recovery speed), and the degree of correlation is greater than the preset degree value, the edge computing device can increase the number of cognitive ability training tasks corresponding to the target cognitive dimension and / or the duration of a single training session in the preliminary scheme. When the degree of damage to the target cognitive dimension is positively correlated with or not significantly correlated with the recovery speed of other target cognitive dimensions, the edge computing device can maintain the number of cognitive ability training tasks corresponding to the target cognitive dimension and the duration of each training session unchanged in the initial scheme, that is, maintain the initial allocation ratio.

[0098] Under this optimization strategy, for each target cognitive dimension, the edge computing device can calculate the correlation coefficient between the historical damage level of that target cognitive dimension and the historical recovery speed of other target cognitive dimensions, resulting in multiple correlation coefficients. When the edge computing device determines that the absolute value of at least one of the multiple correlation coefficients is greater than a preset threshold, it can determine that the correlation between the damage level of that target cognitive dimension and the recovery speed of other target cognitive dimensions is greater than a preset threshold. That is, the damage level of that target cognitive dimension has a significant impact on the recovery speed of other target cognitive dimensions. Whether the correlation is negative or positive depends on the sign of the correlation coefficient; a negative sign indicates a negative correlation, and a positive sign indicates a positive correlation.

[0099] Optimization Strategy Two: Based on the user's personalized difficulty progression path within the target cognitive dimension, optimize at least one of the following in the initial solution: the number of cognitive ability training tasks, the duration of a single training session, and the difficulty level. Specifically, different users experience different levels of cognitive training effectiveness. Based on this, the edge computing device can predict the amount of training required for a user to reach the expected level (e.g., proficiency) at the initial training difficulty (i.e., training difficulty matching the impairment level), and predict the user's performance at a higher level of training difficulty. Based on this training amount and predicted performance, a difficulty progression path is generated to transition from the initial training difficulty to the next higher level. The training amount can depend on the number of cognitive ability training tasks, the training duration, and the difficulty of the cognitive ability training tasks. Subsequently, the edge computing device can optimize at least one of the following in the initial solution: the number of cognitive ability training tasks, the training duration, and the difficulty level, based on the training amount required to progress from the initial training difficulty to the next higher level in the difficulty progression path.

[0100] In this way, it can be ensured that the overall training difficulty of the optimized initial solution is always within the user's "zone of proximal development" (i.e., the difficulty range with the greatest expected benefit), thus ensuring that the optimized initial solution is more targeted and better suited to the user.

[0101] For example, assuming the initial training difficulty level is L1 and the next level up is L2, the user's difficulty progression path is from L1 to L2. This progression path includes the advancement conditions and the amount of training required to advance from L1 to L2. Assuming the training amount depends on the number of cognitive ability training tasks, and the number of cognitive ability training tasks required to advance from L1 to L2 under the target cognitive dimension is 18, while the initial plan has 15 cognitive ability training tasks corresponding to the target cognitive dimension, then the number of cognitive ability training tasks in the initial plan can be increased from 15 to 18.

[0102] Under this optimization strategy, the edge computing device fits a personalized learning curve model for the user based on the user's historical training data, and predicts the amount of training required for the user to reach the expected level at the initial training difficulty, as well as the predicted training performance at a higher level of training difficulty, based on the learning curve model.

[0103] Optimization Strategy 3: Based on the user's optimal training time window, optimize the training period for the cognitive ability training task corresponding to the target cognitive dimension in the initial solution. Specifically, edge computing devices can obtain the correlation between the user's historical training performance and training period to identify the user's optimal training time window, and optimize the training time window of the cognitive ability training task in the initial solution based on this optimal training time window. For example, update the training time window in the initial solution to this optimal training time window.

[0104] Optimization Strategy 4: Given the user's historical cognitive training, based on the user's cognitive recovery rate (i.e., cognitive recovery rate trend) for the target cognitive dimension, optimize at least one of the difficulty and difficulty progression step size of the cognitive ability training task corresponding to the target cognitive dimension in the initial plan. Specifically, the edge computing device can acquire the user's cognitive recovery rate. When the cognitive recovery rate is greater than a preset first rate threshold, the edge computing device can increase the difficulty progression step size or increase the difficulty of the cognitive ability training task to increase the proportion of higher-difficulty tasks (i.e., tasks with difficulty greater than the difficulty threshold). When the cognitive recovery trend is less than or equal to a preset second rate threshold, the difficulty of the cognitive ability training task corresponding to the target cognitive dimension in the initial plan is reduced, or a rest task is inserted into the initial plan, or auxiliary prompts are added, etc. When the cognitive recovery trend is greater than the second rate threshold but less than the first rate threshold, the initial plan remains unchanged. The difficulty progression step size refers to the difficulty level span between two adjacent difficulty advancements.

[0105] Under this optimization strategy, edge computing devices can obtain the difference Δ between historical cognitive scores before and after the most recent training. current The historical cognitive score before training can be considered as the historical cognitive score after the previous training. Subsequently, the edge computing device can use this difference Δ current This allows us to obtain the user's cognitive recovery trend. This cognitive recovery trend can be proportional to the difference Δ. current The quotient of the effective training interval, which can be this value. The effective training interval is the time difference between the most recent cognitive assessment and the user's previous cognitive assessment. The target duration can be expressed in days.

[0106] Optionally, the edge computing device maintains a sequence of users' historical cognitive assessment results, which records the historical cognitive scores before and after each training session. It can be understood that the historical cognitive score before each training session is the same as the historical cognitive score after the previous training session.

[0107] Edge computing devices can acquire front k The difference in historical cognitive scores before and after each training iteration is used to obtain the difference sequence {Δ1,Δ2,...,Δ...}. k Subsequently, the edge computing device can calculate the preceding... k The average of the differences and standard deviation σ history And based on the mean and standard deviation, determine the most recent difference Δ. current Is there any abnormal fluctuation? If |Δ current |>2 σ history If the difference in historical cognitive scores before and after the most recent training session is abnormal, then Δ can be used to determine the abnormal fluctuation. current After correction, the corrected difference Δ is obtained. corrected Then, the edge computing device can base its analysis on the corrected difference Δ. corrected To obtain the user's cognitive recovery trend. T It can satisfy Where Δt is the effective training interval duration.

[0108] Optionally, edge computing devices can employ a Bayesian correction method for Δ. current Make corrections, and the corrected Δ corrected It can satisfy: Δ corrected = α Δ current +(1 α ) .in, α The reliability coefficient represents the reliability of the multimodal physiological data collected during the most recent training and testing phase; it is positively correlated with the data quality of the multimodal physiological data collected during the most recent training and testing phase. Furthermore, this reliability coefficient can be pre-stored on an edge computing device.

[0109] In some optional embodiments, before optimizing the initial scheme using at least one of the aforementioned optimization strategies, the edge computing device may also obtain the standard deviations of multiple initial scores for the target cognitive dimension. Each initial score is obtained based on a set of multimodal data. If the standard deviations of multiple initial scores are greater than a standard deviation threshold, the edge computing device may increase the training frequency of the cognitive ability training task corresponding to the target cognitive dimension in the initial scheme, or extend the single training duration of the cognitive ability training task corresponding to the target cognitive dimension. This allows for stability correction of the initial scheme. The standard deviation threshold may be pre-stored by the edge computing device.

[0110] It should be understood that the stability of a user's cognitive function in the target cognition dimension may be poor, and their control ability may be weak. In this case, edge computing devices can first perform stability correction on the initial solution, and then optimize the initial solution after stability correction.

[0111] If the edge computing device determines that the standard deviation is greater than the standard deviation threshold, meaning that the fluctuations of multiple initial scores are large, then it can be determined that the user's cognitive function in that target cognitive dimension is unstable, i.e., the user's target cognitive dimension is a state-unstable dimension. Subsequently, the edge computing device can correct the stability of the initial scheme by increasing the training frequency of the cognitive ability training tasks corresponding to the target cognitive dimension in the initial scheme, or by extending the single training duration of the cognitive ability training tasks corresponding to the target cognitive dimension.

[0112] If the edge computing device determines that the standard deviation is less than or equal to the standard deviation threshold, that is, the fluctuation of multiple initial scores is small, then it is determined that the user's cognitive function in this target cognitive dimension is stable, and there is no need to make stability corrections to the initial scheme.

[0113] By introducing a stability correction mechanism based on standard deviation, we can specifically strengthen weak areas: when a user's cognitive function in a target cognitive dimension exhibits significant fluctuations and weak neural regulation ability, increasing training frequency or extending the duration of each session can provide saturated stimulation to that dimension, helping the user quickly overcome the "learning plateau" and establish stable cognitive strategies. The learning plateau refers to the stage where a user's progress slows significantly or even stagnates after acquiring certain knowledge or skills.

[0114] On the other hand, it can optimize the allocation of training resources: the stability correction mechanism avoids a "one-size-fits-all" training mode. For dimensions with stable states, edge computing devices can maintain a normal training volume to avoid excessive user fatigue; for unstable dimensions, edge computing devices automatically allocate resources. This dynamic adjustment method maximizes training efficiency while ensuring training effectiveness.

[0115] On the other hand, it can improve the robustness of the cognitive ability training program: by correcting the unstable initial program, it ensures that the final output cognitive ability training program has good convergence, preventing the training difficulty from fluctuating due to fluctuations in the user's state, thereby ensuring the continuity and scientific nature of cognitive rehabilitation training.

[0116] In the embodiments of this application, each of the multiple cognitive ability training tasks is constructed based on a psychomotor vigilance paradigm, a GO / NO-GO paradigm, or a combination of psychomotor vigilance paradigm and GO / NO-GO paradigm.

[0117] The psychomotor vigilance paradigm requires users to react quickly when stimuli appear randomly, without inhibition, memory, or categorization judgment. It only measures maintenance attention, alertness, and reaction speed, with key indicators including reaction time and the number of missed stimuli. The testing procedure for the psychomotor vigilance paradigm may include: "Attentional alertness" reminding users to remain alert and prepare to react to the impending stimulus; after the stimulus appears, users need to press a key as quickly as possible.

[0118] The GO / NO-GO paradigm requires users to react quickly to a "Go" stimulus and inhibit their response to a "No-Go" stimulus. It primarily measures response inhibition, impulse control, and executive function. Key indicators include accidental No-Go button presses (inhibition failure), missed Go button presses, and Go reaction time. The GO / NO-GO paradigm trial procedure may include: randomly presenting two graphics / pictures; the user sees the target stimulus (Go) button and resists pressing the inhibitory stimulus (No-Go). For example, presenting two circular stimuli of different colors, the left circular stimulus being the Go stimulus and the right circular stimulus being the No-Go stimulus. The user is required to respond to the left circular stimulus and inhibit their response to the right circular stimulus.

[0119] Optionally, cognitive ability training tasks built upon the psychomotor alert paradigm and / or the GO / NO-GO paradigm can be game tasks. For example, game tasks built upon the psychomotor alert paradigm could include: a classic no-prop version of whack-a-mole (no bombs, no interfering obstacles), or a spot-tracking game. In the spot-tracking game, numbers / light dots randomly appear on the screen, requiring the user to click them immediately. Game tasks built upon the GO / NO-GO paradigm could include: a fruit and vegetable cutting challenge, or a traffic light reaction game. In the fruit and vegetable cutting challenge, the user is required to cut the fruit upon seeing it, and is prohibited from cutting the fruit upon seeing a bomb.

[0120] A game task constructed by integrating the psychomotor alert paradigm and the GO / NO-GO paradigm could be a "whack-a-mole" game with bombs. In this game, the "whack-a-mole" action itself incorporates the basic mechanism of "target appearance-response-inhibition." The mole (i.e., the Go stimulus) represents the target stimulus requiring a response, while the mine (i.e., the No-Go stimulus) represents the interfering stimulus requiring inhibition of the response. The distribution of holes provides spatial positioning, and the random appearance of moles or mines at the holes perfectly integrates the dual cognitive requirements of "continuous attention + rapid response" in psychomotor alert tasks and "stimulus differentiation + response inhibition" in Go / No-Go tasks, achieving simultaneous assessment of multi-dimensional abilities within a single task.

[0121] Step 204: Obtain the target training mode that users prefer.

[0122] The target training mode is either 2D mode or VR mode.

[0123] Optionally, the edge computing device can display a first selection control for 2D mode and a second selection control for VR mode. The user can touch either the first or second selection control. In response to the user's selection of any of these controls, the edge computing device can determine the training mode corresponding to that control as the target training mode.

[0124] The method provided in this disclosure supports the selection of 2D mode or VR mode. The same training logic can be applied to multiple modes, allowing users to make their own choices based on their experience, thus improving the flexibility of training.

[0125] Step 205: Load the cognitive ability training program according to the target training mode for users to conduct cognitive training.

[0126] When the target training mode is 2D, the edge computing device, after loading the cognitive ability training scheme according to the target training mode, can drive its screen to display the cognitive ability training task. When the target training mode is VR, the edge computing device, after loading the cognitive ability training scheme according to the target training mode, can drive the matching VR headset to display the cognitive ability training task, thereby guiding the user to perform the cognitive ability training task.

[0127] Optionally, when the cognitive ability training scheme includes training content for multiple target cognitive dimensions, the edge computing device can drive the screen or VR display device to present the training content for multiple target cognitive dimensions in descending order of training priority.

[0128] The training priority of each target cognitive dimension can be positively correlated with the degree of impairment of that target cognitive dimension. In other words, cognitive dimensions with higher impairment can be trained first, followed by cognitive dimensions with lower impairment.

[0129] Step 206: During the user's cognitive training, the following adjustment process is executed cyclically: During the training of the nth cognitive ability training task among multiple cognitive ability training tasks, the user's real-time physiological feedback data and real-time training performance data are obtained, where n is a positive integer; based on the real-time physiological feedback data and real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until the preset training termination condition is met.

[0130] Where n is an integer greater than or equal to 1 and less than or equal to N. N is the total number of cognitive ability training tasks. That is, starting from the user performing the first cognitive ability training task (i.e., starting with n=1), the edge computing device can acquire the user's real-time physiological feedback data and real-time training performance data. Based on this data, it determines whether to adjust the difficulty and / or task type of the second cognitive ability training task, and adjusts it if necessary. Subsequently, the edge computing device updates n to 2 and, based on the same data, adjusts the difficulty and / or task type of the third cognitive ability training task, and performs the adjustment if necessary. Then, the edge computing device can update n to 3 and continue the above process. This cycle continues until training ends or stops.

[0131] For example, the difficulty of a cognitive ability training task can be represented by a level. Adjusting the difficulty of the next cognitive ability training task means increasing or decreasing the difficulty of the next cognitive ability training task by at least one level. Adjusting the task type of the next cognitive ability training task means changing the stimulus pattern and / or interaction logic of the next cognitive ability training task. The stimulus pattern is used to define the material type, presentation sequence, frequency of occurrence, and display duration of visual stimuli in the cognitive ability task. The interaction logic is used to define the correctness judgment rules corresponding to user input operations.

[0132] The real-time physiological feedback data may include at least EEG data, such as EEG data and eye movement data. The real-time training performance data may include at least one of accuracy and reaction time. Reaction time is the time it takes for the user to respond from receiving a stimulus.

[0133] Assuming real-time physiological feedback data includes EEG and eye-tracking data, during training, the edge computing device can simultaneously drive the EEG and eye-tracking acquisition devices to work collaboratively, synchronously acquiring the user's EEG and eye-tracking data during training. Furthermore, the edge computing device can drive a companion behavior camera to capture and analyze images of the user's behavior during training, obtaining the user's reaction time. Simultaneously, during training, the edge computing device can compare the user's responses with the standard responses, thereby automatically calculating the accuracy rate.

[0134] The following is an illustrative example of how an edge computing device adjusts at least one of the difficulty and task type of the (n+1)th cognitive ability training task based on real-time physiological feedback data and real-time training performance data: Edge computing devices can determine a user's level of attention distraction based on real-time physiological feedback data. Taking real-time physiological feedback data, including electroencephalogram (EEG) data, as an example, the edge computing device can process the EEG data using a built-in attention coefficient estimation model to generate the user's attention coefficient. This attention coefficient characterizes the user's level of attention distraction. The magnitude of the attention coefficient can be between 0 and 1.

[0135] Typically, when a user's attention is scattered, their training performance is likely to be poor. Conversely, when a user's attention is focused, their training performance may be good or bad. Therefore, if a user's attention is scattered beyond a first threshold, it indicates that the user's current cognitive load is overloaded. In this case, regardless of real-time training performance, the difficulty of the (n+1)th cognitive ability training task can be reduced. The first threshold can be an empirical value pre-stored by the edge computing device.

[0136] If the user's attention span is less than or equal to a first threshold, and the real-time training performance data is below the lower limit of a preset target range (i.e., real-time training is below expectations), it indicates that the user's current cognitive processing strategy does not match the task requirements. In this case, the edge computing device can adjust the task type of the (n+1)th cognitive ability training task. This preset target range is pre-stored by the edge computing device. At this point, the difficulty of the (n+1)th cognitive ability training task can remain unchanged. This allows for a change in the user's cognitive pathway, assisting the user in establishing an effective cognitive strategy. The user's cognitive strategy refers to the methods and techniques used by the user to process information.

[0137] Specifically, when a user's attention is relatively focused (i.e., low degree of distraction) but their training performance remains low, the possibility of performance decline due to lack of focus is ruled out. In this case, if the training difficulty matches the user's ability, it indicates a mismatch between the user's current cognitive strategy and the task requirements, preventing the user from effectively utilizing cognitive resources to complete the task. Therefore, while keeping the difficulty of the (n+1)th cognitive ability training task unchanged, the task type of the (n+1)th cognitive ability training task can be changed.

[0138] For example, edge computing devices can replace the current cognitive training task with a cognitive training task of a different stimulus pattern or interaction logic under the target cognitive dimension. By changing the input characteristics of the task, it helps users break out of their original ineffective mindset and try to establish effective cognitive strategies that match the new task format, thereby improving training performance.

[0139] If the user's attention span is less than or equal to the first threshold, and the real-time training performance data is within the preset target range (i.e., the real-time training performance meets expectations), it indicates that the user is in the optimal training state. In this case, the edge computing device maintains the difficulty of the (n+1)th cognitive ability training task unchanged to maintain the optimal challenge intensity. At this time, the task type of the (n+1)th cognitive ability training task can also remain unchanged.

[0140] If the user's attention is less than or equal to the first threshold and the real-time training performance data is greater than the upper limit of the preset target range (i.e., the real-time training performance is higher than expected), it indicates that the user has surplus cognitive resources. In this case, the edge computing device can significantly increase the difficulty of the (n+1)th cognitive ability training task to make full use of the cognitive plasticity window.

[0141] Optionally, within the user's cognitive plasticity window, edge computing devices can automatically push tasks with progressively increasing difficulty to maximize training gains. The cognitive plasticity window refers to the time period during which the user's attention is highly focused and their real-time training performance data is excellent (i.e., exceeding the lower limit of a preset target range).

[0142] Taking the accuracy rate as an example of real-time training performance data, the preset target interval can include: the first sub-interval corresponding to the accuracy rate. When the accuracy rate is less than the lower limit of the first sub-interval, the real-time training performance data is less than the lower limit of the preset target interval; when the accuracy rate is within the first sub-interval, the real-time training performance data is within the preset target interval; when the accuracy rate is greater than the upper limit of the first sub-interval, the real-time training performance data is greater than the upper limit of the preset target interval.

[0143] Similarly, when the real-time training performance data is reaction time, the preset target interval can include: the second sub-interval corresponding to the reaction time. When the reaction time is less than the lower limit of the second sub-interval, the real-time training performance data is less than the lower limit of the preset target interval; when the reaction time is within the second sub-interval, the real-time training performance data is within the preset target interval; when the reaction time is greater than the upper limit of the second sub-interval, the real-time training performance data is greater than the upper limit of the preset target interval.

[0144] When real-time training performance data includes at least accuracy and reaction time, the preset target interval can include at least a first sub-interval and a second sub-interval. In this case, when all data points (such as accuracy and reaction time) are located at the lower limit of their respective sub-intervals, the real-time training performance data is less than the lower limit of the preset target interval; when all data points are located within their respective sub-intervals, the real-time training performance data is within the preset target interval; and when all data points are located at the upper limit of their respective sub-intervals, the real-time training performance data is greater than the upper limit of the preset target interval.

[0145] Except for the situations described above, where all data points simultaneously fall on the same side of their corresponding sub-intervals (i.e., below the lower limit, within the sub-interval, or above the upper limit), the real-time training performance data can be considered abnormal. In this case, the edge computing device can pause training and handle errors. This disclosure does not limit this action; the specific settings for abnormal training performance data can be configured according to actual circumstances.

[0146] As described above, edge computing devices can combine user multimodal data and real-time training performance data to adjust either the difficulty or task type of the (n+1)th cognitive ability training task. Specifically, this can be achieved as follows: Figure 5 As shown: Step 2071: Detect whether the degree of attentional distraction is greater than the first threshold.

[0147] If the degree of distraction is greater than the first threshold, proceed to step 2072. If the degree of distraction is less than or equal to the first threshold, proceed to step 2073.

[0148] Step 2072: Reduce the difficulty of the (n+1)th cognitive ability training task.

[0149] Step 2073: Detect whether the real-time training performance data is within the preset target range.

[0150] If the real-time training performance data is less than the lower limit of the preset target interval, proceed to step 2074. If the real-time training performance data is within the preset target interval, proceed to step 2075. If the real-time training performance data is greater than the upper limit of the preset target interval, proceed to step 2076.

[0151] Step 2074: Adjust the task type of the (n+1)th cognitive ability training task.

[0152] Step 2075: Keep the difficulty of the (n+1)th cognitive ability training task unchanged.

[0153] Step 2076: Increase the difficulty of the (n+1)th cognitive ability training task.

[0154] In this embodiment, when the degree of attention distraction exceeds a first threshold, an attention recovery mechanism of the edge computing device can be triggered. Under this attention recovery mechanism, the edge computing device can obtain the user's attention distraction type, which is either fatigue-related or interference-related. Fatigue-related attention distraction refers to attention distraction caused by the user's own fatigue and exhibits a continuous characteristic. Interference-related attention distraction refers to attention distraction caused by external interference and exhibits a sudden characteristic.

[0155] When attention is distracted due to fatigue, edge computing devices can pause cognitive training and guide users through relaxation exercises to help them regain focus. For example, the device could display or play a break message such as, "You have been training for a long time; you can take a short break to relax." Alternatively, the device could directly insert a break task page into the training interface.

[0156] When attention is distracted by interference, edge computing devices can adjust the task context of the nth cognitive training task to reduce distractions. Adjusting the task context can include changing the task's visual background, increasing or decreasing the number of distracting elements in the image, and adjusting the background audio volume. For example, the edge computing device can change the task's visual background to a simpler one, reduce the number of distracting elements, and lower the background audio volume.

[0157] Optionally, the edge computing device acquires the decay slope of the user's attention curve and predicts the user's remaining effective training time based on the decay slope. If the edge device determines that the predicted effective training time is insufficient for the user to complete the current task, it can automatically split the task into units or save the training progress to avoid wasted training time.

[0158] Optionally, when a user's level of attention distraction is consistently less than or equal to a first threshold and greater than a second threshold, if the duration of this distraction exceeds a duration threshold, it indicates that the user is in a critical state of attention fluctuation (i.e., attention has decreased but has not yet completely dispersed). The edge computing device can temporarily switch the (n+1)th cognitive ability training task to an embedded attention training task. This embedded attention training task integrates the attention training objective (such as capturing specific visual stimuli) into the current cognitive training scenario while maintaining the difficulty level of the target cognitive dimension. The duration threshold is pre-stored by the edge computing device. The second threshold is less than the first threshold, and this second threshold can be an empirical value pre-stored by the edge computing device.

[0159] During the embedded attention training task, the edge computing device monitors the user's attention state, achieving integrated evaluation and training. This implicit intervention method can guide the user to refocus by changing the task interaction mechanism without interrupting the user's flow experience.

[0160] Understandably, if the user's attention is less than or equal to the first threshold and the duration of the second threshold is less than or equal to the duration threshold, the edge computing device can continue to detect whether the duration is greater than the duration threshold.

[0161] In this embodiment, for different cognitive ability training tasks, the edge computing device can employ a multi-parameter linkage adjustment mechanism to adjust the difficulty of the cognitive ability training task. Adjustable parameters may include time pressure parameters, cognitive load parameters, interference suppression parameters, and feedback delay parameters. The time pressure parameter is used to adjust the task time limit or stimulus presentation duration; the cognitive load parameter is used to adjust the number of information units presented simultaneously; the interference suppression parameter is used to adjust the number and salience of irrelevant stimuli; and the feedback delay parameter is used to adjust the time interval for result feedback.

[0162] For example, in a whack-a-mole game, edge computing devices can adjust the difficulty using time pressure parameters and cognitive compliance parameters. Specifically, the difficulty can be increased by shortening the mole's dwell time and increasing the maximum number of moles on the screen at the same time; the difficulty can be decreased by extending the mole's dwell time and / or reducing the maximum number of moles.

[0163] In addition, edge computing devices can employ an adaptive step size mechanism to adjust the difficulty. The initial adjustment can be one level; if adjustments are made continuously in the same direction, the step size increases; if the adjustment direction is reversed, the step size is reset to avoid difficulty oscillations.

[0164] In this embodiment, if the user's attention span exceeds a first threshold for a duration longer than a preset duration, and the difficulty of the (n+1)th cognitive ability training task has been reduced to its minimum, the edge computing device can pause the user's cognitive ability training and acquire the user's training status. When the training status includes a fatigue level greater than a first fatigue level and less than or equal to a second fatigue level, the edge computing device can execute a first training control strategy to guide the user to rest and recover. When the training status includes an abnormal emotional state and / or a training willingness value less than a first willingness value but greater than or equal to a second willingness value, the edge computing device can execute a second training control strategy to provide training assistance intervention. When the training status includes a fatigue level greater than a second fatigue level, and / or a state of severe emotional disorder, and / or a training willingness value less than a second willingness value, the edge computing device can terminate the user's cognitive ability training.

[0165] In this system, the first fatigue level is lower than the second fatigue level, and the first willingness value is higher than the second willingness value. The first fatigue level, the second fatigue level, the first willingness value, and the second willingness value can all be pre-stored by the edge computing device. The preset duration can also be pre-stored by the edge computing device, such as the training duration for three consecutive cognitive ability training tasks.

[0166] If the duration of the user's attention being consistently above the first threshold exceeds the preset duration, and the difficulty of the (n+1)th cognitive ability training task has been reduced to the minimum, it indicates that the user's training effect is poor or compliance is reduced.

[0167] The training state includes a fatigue level greater than a first fatigue level but less than or equal to a second fatigue level, indicating that the user is experiencing physiological fatigue. The edge computing device can guide the user to take short physiological and psychological breaks to alleviate fatigue by executing a first training control strategy. Afterwards, the edge computing device can re-observe the user's fatigue level and attempt to resume cognitive ability training once the fatigue level has subsided. The first training control strategy may include: displaying and / or broadcasting rest messages, and / or inserting a rest interface.

[0168] Abnormal emotional state can refer to emotions such as anxiety and depression. A training willingness score less than the first willingness score but greater than or equal to the second willingness score indicates a training motivation barrier for the user. The edge computing device adjusts the user's training state by executing a second training regulation strategy. This second training regulation strategy involves loading auxiliary interventions. These interventions can specifically include music-assisted measures and drawing / conversation-assisted measures. Music-assisted measures refer to automatically matching and playing specific frequency bands of audio (such as soothing music) based on the user's emotional state. Drawing / conversation-assisted measures refer to displaying a drawing board for immersive drawing or activating auxiliary conversation functions to alleviate negative emotions and increase participation willingness.

[0169] Training status is categorized into two levels: a fatigue level greater than the second level indicates extreme fatigue, while a training willingness level less than the second level indicates a complete loss of training willingness. In this case, the edge computing device can directly terminate the cognitive ability training.

[0170] Optionally, after the edge computing device finishes cognitive training, it can mark the cognitive training program as having a risk of causing user discomfort, so as to optimize the training parameters in the future. For example, if the user's fatigue level is high, the edge computing device can mark the cognitive training program of this cognitive training as likely to cause user fatigue, so as to preset shorter task units and more frequent rest nodes when generating subsequent cognitive training programs.

[0171] It is understandable that if the user's training status does not improve (i.e., compliance continues to decrease) after the edge computing device executes the first and second training control strategies mentioned above, the current cognitive ability training can also be terminated.

[0172] In this embodiment, after the edge computing device pauses cognitive training for the user, it can save the current training progress for resumption. Furthermore, the edge computing device can push a self-assessment questionnaire to obtain the user's self-assessment results. These results can include fatigue level, emotional state, and training willingness, and can be used to characterize the user's training status.

[0173] Optionally, the self-assessment results may include a score for negative emotions (such as anxiety, depression, and anger) (e.g., the highest score). The edge computing device can determine that the user's emotional state is abnormal when this score is greater than a first score but less than a second score; and determine that the user is in a state of severe emotional disorder when the emotional score is greater than the second score. The second score is greater than the first score, and both the first and second scores can be pre-stored by the edge computing device.

[0174] It is understandable that, when there are multiple target cognitive dimensions, the edge computing device can execute the above steps 206 and 207 when the user performs cognitive training on any target cognitive dimension.

[0175] As described in steps 201 to 207 above, the approach of the method provided in this embodiment is as follows: after the edge computing device is started, if... Figure 6 As shown, the evaluation phase begins first. During this phase, edge computing devices load questionnaires, scales, and paradigms to obtain subjective data from users, and can also load preset experimental paradigms to collect objective data from users.

[0176] Subsequently, the training scheme generation phase begins. In this phase, the edge computing device can obtain the user's cognitive assessment results based on subjective and objective data, identify the damaged target cognitive dimensions based on the cognitive assessment results, and obtain a cognitive training scheme.

[0177] Next, the cognitive training phase begins. During this phase, the edge computing device selects a training mode—either 2D or VR—and loads a cognitive ability training program according to the selected mode for the user to train with. Throughout the cognitive training process, an adjustment process is continuously executed to refine the cognitive ability training program until training concludes or stops.

[0178] In summary, the adaptive training method for human cognitive abilities based on multimodal data provided in this disclosure can acquire a user's multimodal data, obtain the user's cognitive assessment results based on the multimodal data, obtain a matching cognitive ability training plan based on the cognitive assessment results, and train the user's cognitive abilities according to the cognitive ability training plan. During the user's cognitive ability training, the following adjustment process is executed iteratively: during the training of the nth cognitive ability training task, real-time physiological feedback data and real-time training performance data of the user are acquired; based on the real-time physiological feedback data and real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until a preset training termination condition is met. Because the cognitive ability training program can be dynamically adjusted based on the user's real-time physiological feedback data and real-time training performance data during the cognitive ability training process, it can ensure that the adjusted cognitive ability training program is accurately matched with the user's real-time training performance data and real-time physiological feedback data. This improves the adaptability of the cognitive ability training program, thereby ensuring that the user's cognitive training is always in the optimal intensity range, maximizing the training benefits to the greatest extent, and thus meeting the user's training needs.

[0179] Figure 4 An adaptive training device for human cognitive abilities based on multimodal data is provided as an embodiment of the disclosed invention. See also Figure 4 The device 300 includes: The first acquisition module 301 is used to acquire the user's multimodal data.

[0180] The determination module 302 is used to determine the user's cognitive assessment result based on the multimodal data.

[0181] The second acquisition module 303 is used to acquire a matching cognitive ability training scheme based on the cognitive assessment results. The cognitive ability training scheme includes multiple cognitive ability training tasks with adjustable task attributes, and the task attributes include: difficulty and task type. Training module 304 is used to train the user's cognitive abilities according to the cognitive ability training scheme, wherein: during the user's cognitive ability training, the following adjustment process is executed cyclically: during the training of the nth cognitive ability training task among multiple cognitive ability training tasks, real-time physiological feedback data and real-time training performance data of the user are obtained, where n is a positive integer; based on the real-time physiological feedback data and real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues to be executed until the preset training termination condition is met.

[0182] In some embodiments, the training module 304 may be used for: Determine the degree of user distraction based on real-time physiological feedback data; If the degree of attention distraction is greater than the first threshold, then reduce the difficulty of the (n+1)th cognitive ability training task; If the degree of attention distraction is less than or equal to the first threshold and the real-time training performance is less than the lower limit of the preset target range, then the task type of the (n+1)th cognitive ability training task is adjusted. If the degree of attention distraction is less than or equal to the first threshold, and the real-time training performance is greater than the upper limit of the preset target range, then the difficulty of the (n+1)th cognitive ability training task is increased.

[0183] In some embodiments, the real-time physiological feedback data includes: electroencephalogram (EEG) signals; the process by which the training module 304 determines the user's level of attentional distraction based on the real-time physiological feedback data may include: The built-in attention coefficient estimation model processes EEG data to generate the user's attention coefficient, which is used to characterize the degree of the user's attentional distraction.

[0184] In some embodiments, the device may further include: The training control module is used to stop the cognitive ability training of the user and obtain the user's training status if the duration of the attention distraction is greater than the first threshold and the difficulty of the (n+1)th cognitive ability training task is reduced to the preset minimum difficulty. When the training state includes a fatigue level greater than the first fatigue level but less than the second fatigue level, the first training regulation strategy is executed to guide the user to rest and recover. When the training state includes abnormal emotional state and / or the training willingness value is less than the first willingness value but greater than or equal to the second willingness value, the second training regulation strategy is implemented to provide training assistance intervention. The cognitive ability training for the user shall be terminated when the training state includes fatigue level greater than the second fatigue level, and / or, being in a state of severe emotional disorder, and / or, training willingness value less than the second willingness value. Among them, the first fatigue level is less than the second fatigue level, and the first willingness value is greater than the second willingness value.

[0185] In some embodiments, the device may further include: The attention recovery module is used to obtain the user's attention distraction type, which is either fatigued or distracted. If the user's attention deficit is of the fatigue type, stop the cognitive training and guide the user to relax and recover. When the type of attention distraction is interference, adjust the task context of the nth cognitive training task.

[0186] In some embodiments, the cognitive assessment result includes: cognitive scores for each of multiple cognitive dimensions. The second acquisition module 302 can be used to: Based on the cognitive scores of each cognitive dimension, the target cognitive dimension of cognitive impairment is determined; Obtain the degree of impairment in the target's cognitive dimensions; Based on the degree of impairment in the target cognitive dimension, multiple cognitive ability training tasks that match the training difficulty with the degree of impairment are obtained, resulting in a preliminary solution. Based on the preliminary plan, a matching cognitive ability training plan will be obtained.

[0187] In some embodiments, the second acquisition module 302 may be used for: The initial plan is optimized using at least one of the following optimization strategies to obtain a matching cognitive ability training plan. The optimization strategies include: Optimization Strategy 1: When there are multiple target cognitive dimensions, for each target cognitive dimension, based on the correlation between the target cognitive dimensions, optimize at least one of the number of cognitive ability training tasks corresponding to the target cognitive dimension and the duration of a single training session in the preliminary plan. The correlation is the relationship between the degree of damage to the target cognitive dimension and the recovery speed of other target cognitive dimensions. Optimization Strategy 2: Based on the user's personalized difficulty progression path under the target cognition dimension, optimize at least one of the following in the initial plan: the number of cognitive ability training tasks corresponding to the target cognition dimension, the duration of a single training session, and the difficulty. Optimization Strategy 3: Based on the user's optimal training time window, optimize the training period of the cognitive ability training task corresponding to the target cognitive dimension in the initial plan; Optimization Strategy 4: Given that the user has undergone historical cognitive training, optimize at least one of the difficulty and difficulty progression step size of the cognitive ability training task corresponding to the target cognitive dimension in the initial plan based on the user's cognitive recovery rate of the target cognitive dimension.

[0188] In some embodiments, the cognitive assessment results are obtained based on multiple multimodal data sets of the user within a preset time period; the device may further include: The correction module is used to obtain the standard deviation of multiple initial scores of the target cognitive dimension before optimizing the initial scheme using at least one of the following optimization strategies, each initial score being obtained based on a multimodal dataset; If the standard deviation of multiple initial scores is greater than the standard deviation threshold, increase the training frequency of the cognitive ability training task corresponding to the target cognitive dimension in the preliminary plan, or extend the single training duration of the cognitive ability training task corresponding to the target cognitive dimension.

[0189] In some embodiments, the device may further include: The third acquisition module is used to acquire the norm parameters corresponding to the user under the target cognitive dimension when the consistency of multiple data is higher than the consistency threshold. The norm parameters include the norm mean and the norm standard deviation. These multiple data participate in the calculation of the cognitive score of the target cognitive dimension. Based on the norm parameters and the cognitive scores of the target cognitive dimension, the degree of impairment of the target cognitive dimension is determined.

[0190] In some embodiments, cognitive ability training tasks are constructed based on a psychomotor vigilance paradigm, a GO / NO-GO paradigm, or a combination of psychomotor vigilance and GO / NO-GO paradigms.

[0191] In some embodiments, multimodal data includes: subjective data and objective data; Subjective data is based on the user's subjective evaluation results; objective data includes one or more of the following: real-time physiological feedback data, eye-tracking data, and behavioral data.

[0192] In some embodiments, the device may further include: The loading module is used to obtain the user's preferred target training mode, which can be either 2D mode or virtual reality (VR) mode; and loads a cognitive ability training program according to the target training mode for the user to conduct cognitive training.

[0193] In summary, this disclosure provides an adaptive training device for human cognitive abilities based on multimodal data. This device can acquire a user's multimodal data, obtain the user's cognitive assessment results based on the multimodal data, obtain a matching cognitive ability training plan based on the cognitive assessment results, and train the user's cognitive abilities according to the cognitive ability training plan. During the user's cognitive ability training, the following adjustment process is executed cyclically: during the training of the nth cognitive ability training task, real-time physiological feedback data and real-time training performance data of the user are acquired; based on the real-time physiological feedback data and real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until a preset training termination condition is met. Because the cognitive ability training program can be dynamically adjusted based on the user's real-time physiological feedback data and real-time training performance data during the cognitive ability training process, it can ensure that the adjusted cognitive ability training program is accurately matched with the user's real-time training performance data and real-time physiological feedback data. This improves the adaptability of the cognitive ability training program, thereby ensuring that the user's cognitive training is always in the optimal intensity range, maximizing the training benefits to the greatest extent, and thus meeting the user's training needs.

[0194] This disclosure also provides an edge computing device, see [link to relevant documentation]. Figure 5 The edge computing device 400 includes a processor 401 and a memory 402. The memory 402 stores a computer program that can run on the processor 401. When executed by the processor 401, the computer program implements the adaptive training method for human cognitive abilities based on multimodal data provided in the above-described method embodiments. For example, implementing... Figure 1 or Figure 2 The method shown.

[0195] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the adaptive training method for human cognitive abilities based on multimodal data provided in the above-described method embodiments. For example, implementing... Figure 1 or Figure 2 The method shown.

[0196] This disclosure also provides a computer program product that, when executed by a processor, implements the adaptive training method for human cognitive abilities based on multimodal data provided in the above-described method embodiments. For example, it implements... Figure 1 or Figure 2 The method shown.

[0197] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0198] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. In particular, as long as there is no structural conflict, the technical features mentioned in the various embodiments can be combined in any manner.

Claims

1. An adaptive training method for human cognitive abilities based on multimodal data, characterized in that, The method includes: Acquire user multimodal data; The user's cognitive assessment result is determined based on the multimodal data; Based on the cognitive assessment results, a matching cognitive ability training program is obtained. The cognitive ability training program includes multiple cognitive ability training tasks with adjustable task attributes, including difficulty and task type. The user undergoes cognitive ability training according to the aforementioned cognitive ability training scheme, wherein: during the cognitive ability training process, the following adjustment procedure is executed cyclically: During the training process of the nth cognitive ability training task among the multiple cognitive ability training tasks, the user's real-time physiological feedback data and real-time training performance data are obtained, where n is a positive integer; based on the real-time physiological feedback data and the real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until the preset training termination condition is met.

2. The method according to claim 1, characterized in that, Based on the real-time physiological feedback data and the real-time training performance data, adjust at least one of the difficulty and task type of the (n+1)th cognitive ability training task, including: Based on the real-time physiological feedback data, the degree of user attention distraction is determined; If the degree of attention distraction is greater than the first threshold, then the difficulty of the (n+1)th cognitive ability training task is reduced. If the degree of attention distraction is less than or equal to the first threshold, and the real-time training performance data is less than the lower limit of the preset target range, then the task type of the (n+1)th cognitive ability training task is adjusted. If the degree of attention distraction is less than or equal to the first threshold, and the real-time training performance data is greater than the upper limit of the preset target interval, then the difficulty of the (n+1)th cognitive ability training task is increased.

3. The method according to claim 2, characterized in that, The real-time physiological feedback data includes: electroencephalogram (EEG) signals; based on the real-time physiological feedback data, determining the user's level of attentional distraction includes: The EEG data is processed using a built-in attention coefficient estimation model to generate the user's attention coefficient, which is used to characterize the degree of the user's attentional distraction.

4. The method according to claim 2, characterized in that, The method further includes: If the duration of the attention distraction exceeding the first threshold is greater than a preset duration, and the difficulty of the (n+1)th cognitive ability training task is reduced to a preset minimum difficulty, then the cognitive ability training of the user is stopped, and the user's training status is obtained. When the training state includes a fatigue level greater than a first fatigue level and less than or equal to a second fatigue level, a first training control strategy is executed to guide the user to rest and recover. When the training state includes abnormal emotional state and / or training willingness value is less than the first willingness value but greater than or equal to the second willingness value, a second training regulation strategy is executed to provide training assistance intervention. When the training state includes a fatigue level greater than the second fatigue level, and / or a state of severe emotional disorder, and / or a training willingness value less than the second willingness value, the cognitive ability training of the user shall be terminated. Wherein, the first fatigue level is less than the second fatigue level, and the first willingness value is greater than the second willingness value.

5. The method according to claim 2, characterized in that, If the degree of attention distraction exceeds a first threshold, the method further includes: Obtain the user's attention distraction type, which is either fatigue-related or distraction-related. If the type of attention distraction is fatigue-related, stop the cognitive training for the user and guide the user to relax and recover. When the attention distraction type is the interference type, adjust the task context of the nth cognitive training task.

6. The method according to any one of claims 1 to 5, characterized in that, The cognitive assessment results include: cognitive scores corresponding to each of multiple cognitive dimensions; based on the cognitive assessment results, a matching cognitive ability training plan is obtained, including: Based on the cognitive scores of each of the aforementioned cognitive dimensions, the target cognitive dimension of cognitive impairment is determined; Obtain the degree of impairment value of the target cognitive dimension; Based on the degree of impairment of the target cognitive dimension, multiple cognitive ability training tasks with training difficulty matching the degree of impairment are obtained, and a preliminary plan is generated. Based on the preliminary plan, a matching cognitive ability training plan is obtained.

7. The method according to claim 6, characterized in that, Based on the preliminary scheme, a matching cognitive ability training scheme is obtained, including: The initial scheme is optimized using at least one of the following optimization strategies to obtain a matching cognitive ability training scheme, wherein the multiple optimization strategies include: Optimization Strategy 1: When there are multiple target cognitive dimensions, for each target cognitive dimension, based on the correlation between the target cognitive dimensions, optimize at least one of the number of cognitive ability training tasks corresponding to the target cognitive dimension and the duration of a single training session in the preliminary scheme. The correlation is the correlation between the degree of damage of the target cognitive dimension and the recovery speed of other target cognitive dimensions. Optimization Strategy 2: Based on the user's personalized difficulty progression path under the target cognitive dimension, optimize at least one of the following in the preliminary solution: the number of cognitive ability training tasks corresponding to the target cognitive dimension, the duration of a single training session, and the difficulty. Optimization Strategy 3: Based on the user's optimal training time window, optimize the training period of the cognitive ability training task corresponding to the target cognitive dimension in the preliminary solution; Optimization Strategy 4: Given that the user has undergone historical cognitive training, based on the cognitive recovery rate of the user's target cognitive dimension, optimize at least one of the difficulty and difficulty progression step size of the cognitive ability training task corresponding to the target cognitive dimension in the preliminary scheme.

8. The method according to claim 7, characterized in that, The cognitive assessment results are obtained based on multiple sets of multimodal data from the user within a preset time period; before optimizing the preliminary solution using at least one of the following optimization strategies, the method further includes: Obtain the standard deviation of multiple initial scores for the target cognitive dimension, wherein each initial score is obtained based on a set of the multimodal data; If the standard deviation of the multiple initial scores is greater than the standard deviation threshold, increase the training frequency of the cognitive ability training task corresponding to the target cognitive dimension in the preliminary scheme, or extend the single training duration of the cognitive ability training task corresponding to the target cognitive dimension.

9. The method according to claim 6, characterized in that, Before obtaining multiple cognitive ability training tasks whose training difficulty matches the impairment value based on the impairment value of the target cognitive dimension, the method further includes: When the consistency of multiple data sets is higher than the consistency threshold, the norm parameters corresponding to the user under the target cognitive dimension are obtained. The norm parameters include the norm mean and the norm standard deviation. The multiple data sets are used to calculate the cognitive score of the target cognitive dimension. Based on the norm parameters and the cognitive score of the target cognitive dimension, the degree of impairment of the target cognitive dimension is determined.

10. The method according to any one of claims 1 to 5, characterized in that, The cognitive ability training task is constructed based on the psychomotor vigilance paradigm, the GO / NO-GO paradigm, or a combination of the psychomotor vigilance paradigm and the GO / NO-GO paradigm.

11. An adaptive training device for human cognitive abilities based on multimodal data, characterized in that, The device includes: The first acquisition module is used to acquire the user's multimodal data; A determination module is used to determine the user's cognitive assessment result based on the multimodal data; The second acquisition module is used to acquire a matching cognitive ability training scheme based on the cognitive assessment results. The cognitive ability training scheme includes multiple cognitive ability training tasks with adjustable task attributes, and the task attributes include: difficulty and task type. The training module is used to train the user's cognitive abilities according to the cognitive ability training scheme, wherein: during the user's cognitive ability training, the following adjustment process is executed cyclically: During the training process of the nth cognitive ability training task among the multiple cognitive ability training tasks, the user's real-time physiological feedback data and real-time training performance data are obtained, where n is a positive integer; based on the real-time physiological feedback data and the real-time training performance data, at least one of the difficulty and task type of the (n+1)th cognitive ability training task is adjusted; n is updated to n+1, and the adjustment process continues until the preset training termination condition is met.

12. An edge computing device, characterized in that, include: A processor and a memory, the memory storing a computer program that can run on the processor, the computer program which, when executed by the processor, implements the method as claimed in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, A computer program is stored on the readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 10.

Images