Method and system for cognitive training based on eye tracking control technology

Abstract

This application provides a method and system for cognitive training based on eye-tracking control technology, relating to the field of eye-tracking. The method includes: acquiring test data from subjects corresponding to various eye-tracking test tasks; wherein the various eye-tracking test tasks include eye-tracking test tasks corresponding to each cognitive ability to be evaluated; for each cognitive ability, determining the index values ​​of various ability assessment indicators corresponding to that cognitive ability based on the subject's eye-tracking test data related to that cognitive ability, thereby determining the ability score corresponding to that cognitive ability; and determining a matching target cognitive training game based on the subject's ability scores for various cognitive abilities. This method assesses cognitive abilities based on the subject's eye-tracking data and selects a matching game to train cognitive abilities based on the assessment results, enhancing the fun of cognitive training and significantly improving the training effect.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and may relate to the field of eye-tracking technology. Specifically, this application relates to a method and system for cognitive training based on eye-tracking control technology. Background Technology

[0002] With an aging population, the number of patients with neurodegenerative diseases such as Alzheimer's disease is increasing year by year. These diseases mainly impair cognitive function, manifesting as mild cognitive impairment (MCI) in their early stages. MCI is a neurodegenerative disease in which cognitive abilities (such as memory, attention, executive function, orientation, etc.) decline rapidly. Without intervention, nearly 10% of MCI patients will progress to dementia each year, a much higher proportion than the general elderly population. Therefore, the diagnosis and intervention of MCI are particularly important.

[0003] Numerous scientific research findings have demonstrated a close physiological connection between the eyes and the brain. Eye movement is a fine motor activity that involves the precise control of multiple brain regions and neural circuits. By observing eye movements, one can assess the level of cognitive function in the brain. Furthermore, by controlling eye movements, one can inversely train the relevant brain regions and neural circuits involved, thereby improving brain physiology and function and enhancing cognitive function.

[0004] Therefore, there is an urgent need for a method based on eye-tracking control technology to enhance cognitive training for intervention in neurodegenerative diseases such as MCI. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for cognitive training based on eye-tracking control technology that can effectively improve the training effect of cognitive ability. To achieve this purpose, the technical solution provided by this application is as follows:

[0006] On one hand, embodiments of this application provide a method for cognitive training based on eye-tracking control technology, characterized in that the method includes:

[0007] Acquire test data of subjects corresponding to multiple eye movement test tasks; wherein, the multiple eye movement test tasks include eye movement test tasks corresponding to each of the at least one cognitive ability to be evaluated, and the test data of the eye movement test tasks corresponding to each cognitive ability include eye movement test data related to at least one ability assessment index of that cognitive ability;

[0008] For each cognitive ability, based on the eye movement test data of the subjects related to that cognitive ability, the index values ​​of various ability assessment indicators corresponding to that cognitive ability are determined;

[0009] For each cognitive ability, the subject's ability score corresponding to that cognitive ability is determined based on the index values ​​of various ability assessment indicators corresponding to that cognitive ability.

[0010] Based on the subject's ability scores corresponding to various cognitive abilities, a target cognitive training game matching the subject is determined from a variety of candidate cognitive training games.

[0011] Optionally, each of the candidate cognitive training games has a base weight corresponding to various cognitive abilities; a candidate cognitive training game corresponds to a base weight of a cognitive ability, which represents the degree to which the candidate cognitive training game improves that cognitive ability, and the larger the base weight, the greater the improvement;

[0012] The step of determining the target cognitive ability training game that matches the subject from a variety of candidate cognitive training games includes:

[0013] For each candidate cognitive training game, based on the subject's ability score for each cognitive ability, the base weight of the candidate cognitive training game for each cognitive ability is adjusted to obtain the adjusted weight of the candidate cognitive training game for each cognitive ability; wherein, the higher the ability score of a candidate cognitive training game for a cognitive ability, the smaller the weight adjustment amount of the candidate cognitive training game for that cognitive ability.

[0014] For each candidate cognitive training game, the game matching score of that candidate cognitive training game is obtained by integrating the adjusted weights of various cognitive abilities corresponding to that candidate cognitive training game.

[0015] The candidate cognitive training game with the highest matching score is selected as the target cognitive training game.

[0016] Optionally, for each candidate cognitive training game, the step of adjusting the base weight of the candidate cognitive training game for each cognitive ability based on the subject's ability score for each cognitive ability, to obtain the adjusted weight of the candidate cognitive training game for that cognitive ability, includes:

[0017] For each cognitive ability, the degree of impairment of the subject corresponding to that cognitive ability is determined based on the subject's ability score for that cognitive ability, wherein the subject's ability score for a cognitive ability is negatively correlated with the degree of impairment;

[0018] For each cognitive ability, the weight adjustment amount of the candidate cognitive training game corresponding to that cognitive ability is determined based on the degree of impairment of the subject corresponding to that cognitive ability and the basic weight of the candidate cognitive training game corresponding to that cognitive ability.

[0019] For each cognitive ability, the adjusted weight of the candidate cognitive training game for that cognitive ability is determined based on the basic weight of the candidate cognitive training game for that cognitive ability and the weight adjustment amount of the candidate cognitive training game for that cognitive ability.

[0020] Optionally, for each cognitive ability, determining the subject's ability score corresponding to that cognitive ability based on the index values ​​of various ability assessment indicators corresponding to that cognitive ability includes:

[0021] Obtain the indicator weights of each ability assessment indicator for this type of cognitive ability;

[0022] Based on the weights of each indicator, the indicator values ​​of various ability assessment indicators corresponding to the cognitive ability of the test subjects are weighted and fused to obtain the ability score of the test subjects corresponding to the cognitive ability.

[0023] Optionally, the method further includes:

[0024] Determine the game difficulty-related parameters for the subjects;

[0025] Based on the game difficulty-related parameters of the test subjects and the correspondence between the game difficulty-related parameters and each game difficulty level, the target game difficulty level of the test subjects corresponding to the target cognitive training game is determined.

[0026] The game difficulty-related parameters of the test subjects include at least one of the following:

[0027] The user attribute information of the subjects includes the user's age or education level;

[0028] The participants' ability scores corresponded to various cognitive abilities;

[0029] The subjects are assigned index values ​​corresponding to at least one cognitive ability assessment index.

[0030] Optionally, the method further includes:

[0031] The eye movement data of the subject in the target cognitive training game is obtained. The eye movement data includes at least one of the following: the percentage of fixation time of each area of ​​interest in the game interface, the number of eye saccades when fixating each area of ​​interest, and the change in pupil diameter.

[0032] When the eye movement data of the subject meets the first condition, the target cognitive training game is re-determined based on the subject's ability scores corresponding to various cognitive abilities.

[0033] The first condition includes at least one of the following:

[0034] At least one area of ​​interest has a gaze duration percentage less than the first threshold;

[0035] At least one area of ​​interest has fewer eye saccades than the second threshold;

[0036] The pupil diameter change in at least one region of interest is less than the third threshold.

[0037] Optionally, the method further includes:

[0038] Acquire eye movement data of the subjects while they observe the game interface of the target cognitive training game;

[0039] Based on the eye movement data of the subjects, determine the index values ​​of the subjects under at least one preset eye movement observation index, wherein the at least one eye movement observation index includes at least one of fixation time, saccade rate and pupil diameter change;

[0040] Based on the index values ​​of the subjects under the at least one eye-tracking observation index, the cognitive state of the subjects is determined by a state assessment model.

[0041] Optionally, the method further includes:

[0042] Acquire the game performance data of the subjects; wherein the game performance data includes at least one of game completion time or game accuracy rate;

[0043] Based on the game performance data and the preset game difficulty assessment strategy, the game difficulty of the target cognitive training game for the subjects is assessed, and the difficulty assessment result is obtained.

[0044] Based on the difficulty assessment results, the game difficulty level of the target cognition training game is adjusted.

[0045] Optionally, the at least one cognitive ability includes at least one of memory, attention, executive function, orientation, motor control, and abstract logical reasoning.

[0046] Among them, at least one of the following ability assessment indicators for memory is: memory capacity, memory accuracy, or memory placement error;

[0047] At least one indicator of attentional ability assessment includes at least one of the following: saccade latency, saccade velocity, or probability of abnormal interruption of saccade.

[0048] At least one performance indicator for executive function includes: the incidence of reflex saccades or the average tracking error;

[0049] At least one of the following assessment indicators of orientation ability is: saccade accuracy or landing point error;

[0050] At least one of the following ability assessment indicators for abstract logical reasoning ability is: saccade accuracy or classification accuracy.

[0051] At least one performance indicator for motor control includes at least one of the following: tracking average error, landing error, saccade accuracy, or saccade loss rate.

[0052] On the other hand, embodiments of this application also provide a device for cognitive training based on eye-tracking control technology, the device comprising:

[0053] An eye-tracking data acquisition module is used to acquire test data of subjects corresponding to multiple eye-tracking test tasks; wherein, the multiple eye-tracking test tasks include eye-tracking test tasks corresponding to each of the at least one cognitive ability to be evaluated, and the test data of the eye-tracking test tasks corresponding to each cognitive ability includes eye-tracking test data related to at least one ability evaluation index of that cognitive ability.

[0054] The ability assessment index determination module is used to determine the index values ​​of various ability assessment indicators corresponding to each cognitive ability based on the eye movement test data of the subject related to that cognitive ability.

[0055] The cognitive ability score determination module is used to determine the ability score of the subject for each cognitive ability based on the index values ​​of various ability assessment indicators corresponding to that cognitive ability.

[0056] The game selection module is used to determine the target cognitive training game that matches the subject from a variety of candidate cognitive training games based on the subject's ability scores corresponding to various cognitive abilities.

[0057] On the other hand, embodiments of this application also provide a system for cognitive training based on eye-tracking control technology. The system includes an eye-tracking device, a memory, and a processor. The eye-tracking device is used to collect test data of the subject corresponding to various eye-tracking test tasks. The memory stores a computer program, and the processor executes the computer program to implement the method provided in any optional embodiment of this application.

[0058] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in any optional embodiment of this application.

[0059] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the methods provided in any optional embodiment of this application.

[0060] The beneficial effects of the technical solution provided in this application are as follows:

[0061] The method for cognitive training based on eye-tracking control technology provided in this application acquires test data from subjects corresponding to multiple eye-tracking test tasks. These multiple eye-tracking test tasks include eye-tracking test tasks corresponding to each cognitive ability to be evaluated. For each cognitive ability, based on the subject's eye-tracking test data related to that cognitive ability, the method determines the index values ​​of various ability assessment indicators corresponding to that cognitive ability, thereby determining the subject's ability score for that cognitive ability. Based on the subject's ability scores for various cognitive abilities, the method determines a target cognitive training game that matches the subject. This method assesses cognitive abilities based on the subject's eye-tracking data and selects a matching game to train cognitive abilities based on the assessment results, enhancing the fun and effectiveness of training and significantly improving the training effect of cognitive training. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0063] Figure 1 A schematic flowchart illustrating a cognitive training method based on eye-tracking control technology applicable to embodiments of this application;

[0064] Figure 2 A schematic diagram illustrating a cognitive ability assessment and cognitive training method provided in an embodiment of this application;

[0065] Figure 3This is a schematic diagram of the structure of a cognitive training system provided in an embodiment of this application;

[0066] Figure 4 A schematic diagram of the structure of a training device for cognitive training based on eye-tracking control technology provided in an embodiment of this application;

[0067] Figure 5 This is a schematic diagram of the structure of a training system for cognitive training based on eye-tracking control technology, provided in an embodiment of this application. Detailed Implementation

[0068] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0069] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.” When describing multiple (two or more) items, if the relationship between the multiple items is not explicitly defined, the multiple items can refer to one, several or all of the multiple items. For example, the description of "parameter A includes A1, A2, A3" can be implemented as parameter A includes A1 or A2 or A3, or it can be implemented as parameter A includes at least two of the three items A1, A2 and A3.

[0070] It should be noted that, in the optional embodiments of this application, the data related to object information (such as user eye data) requires the permission or consent of the object when the embodiments of this application are applied to specific products or technologies. Furthermore, the collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if the embodiments of this application involve data related to the object, it must be obtained with the object's authorization and consent, the authorization and consent of relevant departments, and in accordance with the relevant laws, regulations, and standards of the country and region. If the embodiments involve personal information, the acquisition of all personal information requires the individual's consent. If sensitive information is involved, the separate consent of the information subject is required, and the embodiments must also be implemented with the object's authorization and consent.

[0071] First, some terms used in the embodiments of this application will be explained and described:

[0072] Fixation: keeping the eyes fixed on a single point and maintaining a gaze.

[0073] Sagging: The eye movement that involves saccades from one fixation point to the next. During saccades, visual input is suppressed, and things are actually not clearly visible.

[0074] Microsaccades are involuntary, small-amplitude saccades that occur during fixation, accompanied by nystagmus and eye drift. When microsaccades disappear, human visual perception becomes blurred and eventually disappears within a very short time due to adaptation by the visual nervous system. Simultaneously, microsaccades play a crucial role in precisely controlling fixation position during high-precision visual processing.

[0075] Positive saccade (task): An initial fixation point is displayed at the initial position, and it is detected whether the subject keeps looking at the initial fixation point. After a random delay, the initial fixation point is stopped, and a target point is displayed at a first predetermined position different from the initial position. The subject needs to look at the target point quickly and accurately. The saccade parameters such as the subject's saccade amplitude, saccade velocity, saccade latency, saccade accuracy, and saccade direction can be detected.

[0076] Reverse saccade (task): An initial fixation point is displayed at the initial position, and it is detected whether the subject keeps fixating on the initial fixation point. After a random delay, the initial fixation point is stopped from being displayed, and a target point is displayed at a first predetermined position different from the initial position. The subject needs to quickly look at the reverse position (mirror position) of the target point. The saccade parameters such as the amplitude of the reverse saccade, the speed of the reverse saccade, the latency of the reverse saccade, the error saccade rate, and the deviation of the saccade trajectory can be detected.

[0077] Memory saccades (task): The initial fixation point is displayed at the initial position, and it is detected whether the subject keeps looking at the initial fixation point. While the initial fixation point is displayed, a target point is briefly flashed at a first predetermined position different from the initial position. After the subject is detected to have looked at the initial fixation point for a preset time, the display of the initial fixation point is stopped. The subject needs to look at the first predetermined position in memory. The saccade parameters such as the subject's saccade amplitude, saccade speed, saccade accuracy, saccade frequency, and fixation duration can be detected.

[0078] Two-step saccade (task): An initial fixation point is displayed at the initial position, and it is detected whether the subject keeps fixating on the initial fixation point. A target point is briefly flashed at a first predetermined position different from the initial position, and then a target point is briefly flashed at a second predetermined position. After the subject is detected to have been fixating on the initial fixation point for a preset time, the display of the initial fixation point is stopped. The subject needs to perform two consecutive saccades, looking at the first and second predetermined positions in memory in turn. The saccade parameters such as the subject's saccade amplitude, saccade speed, saccade latency, saccade trajectory deviation, and number of saccades can be detected.

[0079] Smooth tracking (task): This task is used to track continuously moving visual objects of interest in real time. For example, a target point is displayed on a screen according to a predetermined motion trajectory, and the subject needs to follow the target point's movement with their eyes as much as possible. It can detect the subject's tracking error, tracking gain (eye movement speed / target point movement speed), eye movement speed, acceleration and other eye movement parameters.

[0080] N-back working memory task: A working memory test that requires participants to compare whether the current stimulus is the same as the stimulus presented in the nth previous trial.

[0081] The GO-NO-GO saccade task involves two different types of visual stimuli: "GO" stimuli and "NO-GO" stimuli. These stimuli are easily distinguishable in terms of appearance, attributes, or presentation, such as different colored dots, different shaped graphics, or symbols in different positions. Participants are required to respond differently depending on the type of stimulus presented. When a "GO" stimulus appears, participants must quickly saccade, shifting their gaze to the designated target location. When a "NO-GO" stimulus appears, participants must suppress their saccade impulse, maintaining focus on the current location without saccadeing.

[0082] Principal Component Analysis (PCA) is a commonly used data analysis technique for dimensionality reduction and removing redundant information from data while preserving its main characteristics. It transforms a set of potentially correlated variables into a set of linearly uncorrelated variables through orthogonal transformation; these new variables are called principal components.

[0083] The technical solutions of this application and their effects are described below through several embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0084] The present application provides a method for cognitive training based on eye-tracking control technology, which can be executed by any electronic device, such as a server or terminal (such as an eye-tracking device).

[0085] Figure 1 This is a flowchart illustrating a method for cognitive training based on eye-tracking control technology provided in an embodiment of this application. The method may include the following steps S110-S140, wherein:

[0086] Step S110: Obtain test data of subjects corresponding to multiple eye movement test tasks, wherein the multiple eye movement test tasks include eye movement test tasks corresponding to each of the at least one cognitive ability to be evaluated, and the test data of the eye movement test tasks corresponding to each cognitive ability include eye movement test data related to at least one ability assessment index of that cognitive ability.

[0087] Eye movement is a fine motor skill that involves the precise control of multiple brain regions and neural circuits. Therefore, the level of cognitive function can be assessed by observing eye movements. In this embodiment, the at least one cognitive ability to be evaluated includes one or more cognitive domains such as attention, memory, executive function, orientation, motor control, and abstract logical reasoning. The ability assessment indicators for each cognitive ability are constructed based on a large amount of clinical and experimental data. The ability assessment indicators for different cognitive abilities are not entirely the same. The eye movement test task for each cognitive ability includes eye movement test tasks related to the various ability assessment indicators of that cognitive ability.

[0088] For example, the eye-tracking test tasks corresponding to each cognitive ability, and the corresponding ability assessment indicators, can be shown in Table 1 below:

[0089]

[0090] It is understandable that, for each cognitive ability, when collecting relevant eye-tracking test data based on eye-tracking test tasks related to each ability assessment index of that cognitive ability, the collection of index values ​​for each ability assessment index can be considered as one eye-tracking test task. Alternatively, when some ability assessment indicators correspond to the same eye-tracking test task, eye-tracking test data related to each ability assessment index within that portion of the ability assessment indicators can be collected based on that eye-tracking test task. Taking saccade latency and saccade velocity in attention ability assessment indicators as examples, forward saccade tasks can be used to collect eye-tracking test parameters related to saccade latency and saccade velocity of subjects. In practice, these parameters can be obtained simultaneously through the same forward saccade task; alternatively, the collection of saccade latency can be set as a separate eye-tracking test task, and the collection of saccade velocity can be set as another eye-tracking test task, thus achieving the collection of eye-tracking test parameters through different eye-tracking test tasks. This application does not restrict the specific division of eye-tracking test tasks and can flexibly set them according to actual research needs.

[0091] In the embodiments of this application, such as Figure 2 As shown, participants can perform various eye-tracking tasks using an eye-tracking device, and the test data generated during each task is collected. These tasks include one or more of the following: forward saccades, reverse saccades, memory saccades, bistep saccades, smooth tracking, working memory capacity, GO-NO-GO saccades, and object sorting. The test data for each task includes eye movement trajectory data such as saccade count, saccade amplitude, fixation point position, saccade velocity, saccade acceleration, fixation duration, pupil diameter, saccade latency, spatial error / landing point error, etc.

[0092] Before performing eye-tracking tests using an eye-tracking device, eye-tracking calibration can be performed first. Specifically, the eye-tracking device emits near-infrared light and shines it into the subject's eyes. N calibration points are displayed sequentially on the screen for a preset duration (e.g., 2 seconds). The subject looks at the calibration points displayed on the screen in turn. The camera of the eye-tracking device captures the eye image of the subject when observing each calibration point. Based on the relative relationship between the corneal reflection spot and the pupil position in the eye image when observing each calibration point, the least squares method is used to fit the actual position of the calibration point with the gaze point position collected by the eye-tracking device to establish a position mapping model. The calibration parameters of the eye-tracking device are adjusted accordingly to ensure that the error between the collected eye movement data and the actual eye movement position is controlled within a preset range (e.g., 0.15° visual angle), thus ensuring the accuracy of the subsequently collected test data.

[0093] Step S120: For each cognitive ability, based on the eye movement test data of the subject related to that cognitive ability, determine the index values ​​of various ability assessment indicators corresponding to that cognitive ability.

[0094] Optionally, since subjects may blink or move during eye-tracking tests, leading to inaccurate data, the acquired test data can be cleaned before cognitive ability assessment to remove noise and outliers caused by blinking, slight head movements, etc. Based on the cleaned eye-tracking test data related to the cognitive ability, the indicator values ​​for various ability assessment indicators corresponding to that cognitive ability are determined. This application does not limit the type of denoising algorithm used; for example, a Kalman filter algorithm can be used for data cleaning.

[0095] Step S130: For each cognitive ability, determine the subject's ability score corresponding to that cognitive ability based on the index values ​​of various ability assessment indicators corresponding to that cognitive ability.

[0096] Each cognitive ability corresponds to at least one ability assessment indicator (also known as eye movement behavior feature). Since different ability assessment indicators reflect the performance of different brain functions, the importance of different ability assessment indicators varies. When assessing each cognitive ability, the indicator weight of each ability assessment indicator for that cognitive ability can be obtained. Based on the indicator weights, the indicator values ​​of various ability assessment indicators for the subject corresponding to that cognitive ability are weighted and fused to obtain the subject's ability score for that cognitive ability.

[0097] For example, the ability assessment indicators corresponding to inhibitory function include the incidence of reflex saccades and the average target tracking error. The weight of the reflex saccade incidence rate is 0.6, and the weight of the average target tracking error is 0.4. Therefore, the subject's ability score corresponding to inhibitory function is 0.6. The incidence of reflex saccades was +0.4%. Average target tracking error.

[0098] Step S140: Based on the subject's ability scores corresponding to various cognitive abilities, determine the target cognitive training game that matches the subject from a variety of candidate cognitive training games.

[0099] Optionally, the game resource library includes a variety of candidate cognitive training games. These games can train various cognitive abilities by guiding participants to observe the eye movements generated by the game visuals. For example... Figure 2As shown, the game screen of the target cognitive training game is displayed on the game terminal. Subjects observe the target object in the game screen and perform corresponding game tasks to train various cognitive abilities. This application embodiment does not limit the types of games included in the game resource library, as long as they can be used to improve at least one cognitive ability.

[0100] As one implementation method, the game resource library can include a variety of cognitive training games for training different cognitive abilities, such as: attention concentration training games (such as focus target tracking games, attention switching challenge games, etc.), memory training games (such as memory matching games, memory sequence reconstruction games, etc.), and executive function training games (such as reverse eye movement control games, task planning simulation games, information sorting and sequencing games, etc.).

[0101] After obtaining the participants' ability scores for various cognitive abilities, for each cognitive ability, the impairment of that cognitive ability can be determined based on the participant's score and the corresponding benchmark score. The benchmark score for each cognitive ability can be the average or lowest score of the normal population for that cognitive ability. If the participant's score for that cognitive ability is lower than the benchmark score, the participant's cognitive ability is considered impaired; otherwise, the participant's cognitive ability is considered normal. Impaired cognitive abilities are those in which the participant performs below the level of the normal population. Based on the participants' impaired cognitive abilities, target cognitive training games are determined to match the participants. For example, if a participant's attention is impaired and significantly lower than average, the target cognitive training games would include various attention concentration training games to improve the participant's attention.

[0102] In this embodiment of the application, cognitive assessment results of the subjects can be generated based on their ability scores corresponding to various cognitive abilities and their impaired cognitive abilities, so that the subjects can understand their own cognitive status.

[0103] Optionally, each candidate cognitive training game in the game resource library can be used for the collaborative training of multiple cognitive abilities. Each candidate cognitive training game has a base weight (default training weight) corresponding to various cognitive abilities. The base weight of a candidate cognitive training game for a specific cognitive ability represents the degree to which the candidate cognitive training game improves that cognitive ability; the larger the base weight, the greater the improvement. For example, game A has a base weight of 0.7 for attention, 0.2 for memory, and 0.1 for executive function. This means that game A is mainly used to improve the user's attention, has some effect on improving memory, but has limited effect on improving executive function.

[0104] When selecting target cognitive training games that match the participants, for each candidate cognitive training game, the basic weights of the candidate cognitive training game corresponding to each cognitive ability can be adjusted based on the participants' ability scores for each cognitive ability, resulting in adjusted weights for each cognitive ability. Specifically, the higher the ability score of a candidate cognitive training game for a particular cognitive ability, the smaller the weight adjustment for that cognitive ability. The adjusted weights of the candidate cognitive training game for each cognitive ability are then combined to obtain the game matching score for that candidate cognitive training game. The candidate cognitive training game with the highest game matching score is selected as the target cognitive training game.

[0105] Optionally, after determining the game matching scores of various candidate cognitive training games for the subjects, for each candidate cognitive training game, if the game matching score of the candidate cognitive training game is greater than the preset game benchmark score, then the candidate cognitive training game is regarded as a game to be recommended. Based on the game matching scores of each game to be recommended for the subjects, the recommendation order of each game to be recommended is determined, and the games are recommended to the subjects in the order of recommendation.

[0106] Optionally, for each candidate cognitive training game, when adjusting the base weights of the candidate cognitive training game corresponding to various cognitive abilities based on the participants' ability scores for each cognitive ability, the following steps can be taken: First, determine the degree of impairment (also known as the degree of deficit) of the participants for each cognitive ability based on their ability scores. The lower the participant's ability score for a particular cognitive ability, the higher the degree of impairment. Then, determine the weight adjustment amount for the candidate cognitive training game corresponding to that cognitive ability based on the participant's degree of impairment and the base weights of the candidate cognitive training game for that cognitive ability. Finally, determine the adjusted weights of the candidate cognitive training game corresponding to that cognitive ability based on the base weights of the candidate cognitive training game for that cognitive ability and the weight adjustment amount.

[0107] Optionally, to enhance the training of severely impaired cognitive abilities in participants, their weight in game matching can be increased to emphasize their importance. After determining the degree of impairment for each cognitive ability, this impairment can be amplified. Based on the amplified impairment, the base weights of candidate cognitive training games for each cognitive ability are adjusted, resulting in adjusted weights for each cognitive ability. Specifically, adjusting the base weights based on the amplified impairment ensures that the adjusted weights of severely impaired cognitive abilities are higher, facilitating matching with target cognitive training games that offer greater improvement to those severely impaired abilities during game matching.

[0108] Assume the game resource library contains m types of games, and the cognitive domains to be evaluated are n. The basic weight matrix for each game corresponding to each cognitive domain is as follows: For each element in the basic weight matrix , where represents the basic weight of the j-th cognitive domain corresponding to the i-th game.

[0109] After determining the subjects' ability scores for each cognitive domain, the degree of their deficit in each cognitive domain can be calculated using the following formula (1):

[0110] (1)

[0111] Where j represents the j-th cognitive domain, This represents the subject's ability score corresponding to the j-th cognitive domain. This indicates the degree of deficit of the subject corresponding to the j-th cognitive domain.

[0112] Subsequently, based on the subjects' deficit levels in each cognitive domain and the basic weight matrices of each game corresponding to each cognitive domain, the adjusted weight matrices for each game corresponding to each cognitive domain were determined. Among them, the weight matrix Each element in Let represent the adjusted weight of the j-th cognitive domain corresponding to the i-th game. The importance of the degree of deficiency is reinforced through squaring, resulting in the weight matrix. Each element in It can be represented as:

[0113] (2)

[0114] Then, for each game, the sum of the weights adjusted for each cognitive domain can be used as the recommendation score for that game (i.e., the game-matching score between the game and the participants), which can be expressed as:

[0115] (3)

[0116] in, Let represent the recommended score for the i-th game. This represents the adjusted weight of the j-th cognitive domain corresponding to the i-th game.

[0117] In summary, the recommended score for each game can be expressed as:

[0118] (4)

[0119] Where m represents the total number of games, n represents the total number of cognitive domains, i represents the i-th game, and j represents the j-th cognitive domain. Let represent the recommended score for the i-th game. This represents the basic weight of the j-th cognitive domain corresponding to the i-th game. This represents the subject's ability score (vector) corresponding to the j-th cognitive domain.

[0120] For example, suppose the game resource library includes two games, Game A and Game B. The cognitive abilities to be evaluated include three types: attention, memory, and executive ability. Game A has a base weight of 0.7 for attention, 0.2 for memory, and 0.1 for executive ability; Game B has a base weight of 0.3 for attention, 0.6 for memory, and 0.1 for executive ability, as shown in Table 2 below.

[0121]

[0122] Assuming the participant's attention score is 25, memory score is 50, and executive function score is 85, then the participant's deficit level for attention is 1 - 25 / 100 = 0.75, for memory deficit level is 1 - 50 / 100 = 0.5, and for executive function deficit level is 1 - 85 / 100 = 0.15, as shown in Table 3 below:

[0123]

[0124] Based on the degree of deficit of the participants in various cognitive abilities, the basic weights of each game corresponding to each cognitive ability were adjusted to obtain the adjusted weights of each game corresponding to each cognitive ability, as shown in Table 4 below:

[0125]

[0126] Based on the adjusted weights of each game corresponding to the participants' various cognitive abilities, the recommended scores (game matching scores) for each game were determined:

[0127] The recommended score for game A is 1.09375 + 0.25 + 0.10225 = 1.446

[0128] Recommended score for game B = 0.46875 + 0.75 + 0.10225 = 1.321

[0129] Since game A received the highest recommendation score, it was recommended as the game that best matched the participants.

[0130] Optionally, each candidate cognitive training game in the game resource library can be set to a fixed difficulty level by default, or each candidate cognitive training game can also have a difficulty level setting function, so as to make adaptive adjustments based on the cognitive ability assessment results or changes in individual ability levels of the test subjects.

[0131] Specifically, before using target cognitive training games to train cognitive abilities, the game difficulty-related parameters of the subjects can be determined. Based on the game difficulty-related parameters of the subjects and the correspondence between the game difficulty-related parameters and each game difficulty level, the target game difficulty level corresponding to the subjects can be determined. The game difficulty level of the target cognitive training game can then be adjusted to the target game difficulty level. The target cognitive training game with the adjusted difficulty level can then be used to train the subjects' cognitive abilities.

[0132] Among them, the game difficulty-related parameters for the test subjects included at least one of the following:

[0133] The user attribute information of the test subjects; among which, the user attribute information includes the user's age or education level, wherein the higher the user's age, the lower the corresponding game difficulty level; the higher the user's education level, the higher the corresponding game difficulty level;

[0134] The participants were assigned ability scores corresponding to various cognitive abilities. The lower the participant's ability score, the lower the difficulty level of the game they were matched with.

[0135] The subject's index value corresponding to at least one cognitive ability assessment indicator.

[0136] In this embodiment, the difficulty level of different games can be adjusted by changing the movement speed of the target object, the display size of the target object, and the number of interfering objects. Taking a target tracking game as an example, if the subject corresponds to a low difficulty level, the movement speed of the tracked target object is 0.5v, and the size of the target object is set to 1.5S; if the subject corresponds to a medium difficulty level, the movement speed of the tracked target object is 0.7v, and the size of the target object is set to S; if the subject corresponds to a high difficulty level, the movement speed of the tracked target object is v, and the size of the target object is set to 0.8S.

[0137] It is understandable that participants' cognitive abilities will improve in various aspects as they continuously train through cognitive training games. In order to recommend the most suitable games to users, participants' cognitive abilities can be reassessed periodically, and games can be recommended to participants based on the latest cognitive ability assessment results.

[0138] based on Figure 1 The cognitive training method described here assesses various cognitive abilities based on eye-tracking test data from subjects across multiple eye-tracking tasks. It then selects matching cognitive training games based on the subjects' scores for each cognitive ability, providing targeted improvement. This significantly enhances the fun and challenge of cognitive training, increases subject participation, and substantially improves the effectiveness and efficiency of cognitive rehabilitation training. It fills the gap in existing technologies for personalized cognitive rehabilitation training driven by eye-tracking data, meets the rehabilitation needs of a large number of patients with cognitive impairment, and promotes innovative development in the field of cognitive rehabilitation.

[0139] Furthermore, cognitive training games can set up different scenarios and tasks, covering collaborative training in multiple cognitive domains (such as attention, memory, executive function, etc.), which helps to comprehensively improve users' cognitive functions.

[0140] During game training, participants' interest in the game can be assessed based on their performance. If a participant's interest in the current game is low, they can be switched to another game to ensure the effectiveness of the training.

[0141] Optionally, eye movement data of the subjects during the target cognitive training game is obtained. The eye movement data during the game includes at least one of the following: the percentage of fixation time of each area of ​​interest in at least one area of ​​interest in the game interface, the number of saccades when fixating on different areas of interest, and changes in pupil diameter. When the eye movement data of the subjects meets the first condition, the target cognitive training game is re-determined based on the subjects' ability scores corresponding to various cognitive abilities.

[0142] The first condition includes at least one of the following:

[0143] At least one area of ​​interest has a gaze duration percentage less than the first threshold;

[0144] At least one area of ​​interest has fewer eye saccades than the second threshold;

[0145] The pupil diameter change in at least one region of interest is less than the third threshold.

[0146] The game interface can be divided into multiple Areas of Interest (AOIs). Based on eye movement data during gameplay, the fixation duration for each AOI, the pupil diameter, and the number of saccades generated while fixating on different AOIs are determined. The percentage of time each participant spends fixating on each AOI is determined based on the fixation duration and the duration of game training. Longer fixation duration, more saccades, and greater changes in pupil diameter (e.g., an increase or decrease exceeding 10%) indicate greater interest in that AOI. When dividing AOIs, game characters, mission objectives, and reward areas can each be considered as a separate AOI.

[0147] When the participant's eye movement data meets the first condition, it indicates that the participant has low interest in the current target cognitive training game, and other games to match the participant should be re-selected. For example, if the participant's fixation time in a certain AOI accounts for more than 30% of the total game time, and the number of saccades in that AOI is greater than 15 and the pupil diameter increases by more than 10%, then the participant is considered interested in the content of that AOI. If the participant's fixation time in multiple AOIs on the game interface accounts for less than 5% of the total game time, and the saccade frequency is low, then the participant is considered uninterested in the content of that AOI, and other types of games can be switched, such as switching from an action game to a puzzle game.

[0148] In this embodiment, the game difficulty level can be adjusted based on the subject's performance in the game. Specifically, after the subject completes the target cognitive training game, the subject's game performance data is obtained; wherein, the game performance data includes at least one of game completion time or game accuracy rate; based on the subject's game performance data and a preset game difficulty assessment strategy, the game difficulty of the target cognitive training game for the subject is assessed to obtain a difficulty assessment result; and the game difficulty level of the target cognitive training game is adjusted according to the difficulty assessment result.

[0149] The game difficulty reference data includes the game completion reference time and the game reference accuracy rate. The game completion reference time can be calculated by statistically analyzing the average game completion time of a normal population, and the game reference accuracy rate can be calculated by statistically analyzing the average game accuracy rate of a normal population.

[0150] The preset game difficulty assessment strategy may include at least one of the following:

[0151] Strategy 1:

[0152] Determine the average completion time of the game for the participants;

[0153] Determine the first completion time difference between the average game completion time and the reference game completion time;

[0154] Determine the percentage of the first completion time difference relative to the first time reference completion time of the game;

[0155] If the percentage of time spent in the first instance exceeds the preset percentage, the game difficulty level will be lowered.

[0156] Strategy Two:

[0157] Determine the average accuracy rate of the participants in the game;

[0158] If the game's average accuracy rate is lower than the game's reference accuracy rate, lower the game's difficulty level.

[0159] Strategy 3:

[0160] Determine the second completion time difference between the game completion time and the game completion reference time;

[0161] Determine the proportion of the second completion time difference to the second time reference completion time;

[0162] If the number of games in which the second time segment exceeds the preset time segment reaches the preset number, the game difficulty level will be lowered.

[0163] Strategy Four:

[0164] Determine the accuracy rate of the test subjects in the game;

[0165] If the number of times the game's accuracy rate falls below the game's reference accuracy rate reaches a preset number, the game difficulty level will be lowered.

[0166] For example, if a participant's game completion time is more than 30% longer than the reference completion time for three consecutive games, and the accuracy rate is less than 70%, then the game difficulty level is determined to be too high. The game difficulty level can be lowered by reducing the target movement speed or increasing the target size. If the participant's game completion time is 30% shorter than the reference completion time for three consecutive games, and the accuracy rate is higher than 90%, then the game difficulty level is determined to be too low. The game difficulty level can be increased by increasing the target movement speed, decreasing the target size, or adding distractors.

[0167] In this embodiment, the cognitive state (focus, distraction, thinking) of the subjects can also be detected in real time based on their eye movement data during game training. Specifically, eye movement data of the subjects observing the game interface of the target cognitive training game is acquired; based on the subjects' eye movement data, the index values ​​of the subjects under at least one preset eye movement observation index are determined, wherein the at least one eye movement observation index includes at least one of fixation time, scan rate, and pupil diameter change; based on the index values ​​of the subjects under at least one eye movement observation index, the cognitive state of the subjects is determined through a state assessment model. As an optional approach, the state assessment model can adopt a Hidden Markov Model (HMM).

[0168] Specifically, the steps for determining cognitive states using a state assessment model include:

[0169] Obtain a preset state transition probability matrix and observation probability matrix; wherein, the state transition probability matrix includes the probability of state transition between cognitive states, and the observation probability matrix includes the probability of different combinations of eye movement observation index values ​​observed in each cognitive state;

[0170] Based on the subject’s index value under at least one eye-tracking observation index in the current time period, the state probability distribution matrix corresponding to different cognitive states in the previous time period, the state transition probability matrix, and the observation probability matrix, determine the subject’s state probability distribution matrix corresponding to different cognitive states in the current time period.

[0171] Based on the probability distribution matrix of different cognitive states corresponding to the current time period, the cognitive state of the subject at the current time period is determined.

[0172] The subjects' multiple selectable cognitive states include focused, distracted, and thinking. The state transition probability matrix and observation probability matrix can be obtained by collecting a large amount of users' state data at different time periods and conducting statistical analysis.

[0173] When the test subject is detected to be distracted a lot during the game, it indicates that the subject has low interest in the current game, and other cognitive training games can be switched to display.

[0174] Optionally, the cognitive ability assessment results may also include eye movement behavior pattern detection results, which may be obtained through the following methods:

[0175] Identify multiple eye-movement behavior indicators related to eye-movement behavior pattern detection;

[0176] Based on the test data of the subjects corresponding to various eye movement test tasks, determine the eye movement test data of the subjects related to multiple eye movement behavior indicators;

[0177] Based on eye movement test data related to various eye movement behavior indicators, the index values ​​of various eye movement behavior indicators were determined;

[0178] The eye movement behavior pattern detection results are determined based on the index values ​​of various eye movement behavior indicators and the reference index values ​​of each eye movement behavior indicator.

[0179] Among them, eye movement behavior pattern detection includes the detection of various behavior patterns such as gaze stability detection and attentional flexibility detection. The eye movement behavior indicators related to each behavior pattern are not exactly the same. For example, the eye movement behavior indicators related to gaze stability detection include gaze duration, and the eye movement behavior indicators related to attentional flexibility detection include the fluency of the saccade path, the accuracy of the saccade path, and so on.

[0180] For gaze stability testing, the gaze duration of subjects at different points in each eye movement test task can be determined from the test data of subjects corresponding to various eye movement test tasks. Based on the gaze duration of subjects at different points in each eye movement test task and the gaze duration of normal people under the same task, the gaze stability of subjects can be judged. The greater the difference between the gaze duration distribution range of subjects and the gaze duration distribution range of normal people, the worse the gaze stability of subjects.

[0181] For assessing attentional flexibility, eye movement data such as the number of saccades, saccade acceleration, fixation point, and target observation point can be obtained from test data corresponding to various eye-tracking tasks. Based on the number of saccades and saccade acceleration, the smoothness of the saccade path in each task can be determined; a smaller number of saccades and a smaller standard deviation of saccade acceleration indicate a smoother saccade path. Based on the fixation point and target observation point, the accuracy of the saccade path in each task can be determined; a smaller positional deviation between the fixation point and the target observation point indicates higher accuracy. Finally, the smoothness and accuracy of the saccade path in each eye-tracking task can be used to assess the subject's attentional flexibility.

[0182] It should be noted that the above-mentioned eye-tracking test data can be used directly as the index value of the eye-tracking behavior indicator, or it can be used as the index value of the eye-tracking behavior indicator after processing. This application embodiment does not limit this. For example, when the eye-tracking behavior indicator is the fluency of the saccade path, the eye-tracking test data related to the fluency of the saccade path includes the number of saccades and the saccade acceleration. The fluency of the saccade path can be determined based on the number of saccades and the saccade acceleration. When the eye-tracking behavior indicator is the fixation duration, the fixation duration value in the eye-tracking test data can be directly read as the fixation duration index value.

[0183] In this embodiment, subjects can continuously engage in cognitive training through a target cognitive training game. Once the training duration reaches a preset time, the training effect can be evaluated to determine whether the subject's cognitive abilities have improved. Specifically, the training effect can be evaluated using any of the following methods:

[0184] Method 1: Re-examine the test data of the subjects under various eye-tracking test tasks, and determine the index values ​​of various ability assessment indicators corresponding to various cognitive abilities based on the eye-tracking test data of the subjects. Determine the ability scores of the subjects corresponding to various cognitive abilities based on the index values ​​of various ability assessment indicators corresponding to various cognitive abilities. Evaluate the training effect of the subjects in various cognitive abilities based on the ability scores of each cognitive ability after training and the ability scores of each cognitive ability before training.

[0185] Method 2: Obtain eye movement data and game performance data of the participants during game training. Eye movement data includes the number of eye saccades, saccade amplitude, fixation point position, and fixation duration when the participants observe the game screen. Game performance data includes the participants' game completion time, task accuracy, number of errors, etc. Based on the participants' game performance data, extract key feature data, and analyze the key feature data before and after training to evaluate the training effect on the participants in various cognitive abilities.

[0186] Optionally, when extracting key feature data from game performance data, principal component analysis (PCA) can be used to extract key feature data to reduce data dimensionality.

[0187] Optionally, when analyzing key feature data before and after training, a Support Vector Machine (SVM) algorithm can be used to classify the key feature data of the subjects throughout the training process, obtaining key feature data before and after training, and then performing comparative analysis. For example, key feature data includes the index values ​​corresponding to energy assessment indicators for each cognitive ability. By comparing the index values ​​of each cognitive ability's energy assessment indicators before and after training, the improvement of the subjects in each energy assessment indicator before and after training can be obtained.

[0188] Optionally, the training evaluation results may also include a comparative analysis of the cognitive abilities of the subjects and normal people of the same age, as well as suggestions for subsequent training. For example, for subjects whose attention has improved significantly but whose memory has improved slowly, the proportion of memory training games should be increased in subsequent training, and the difficulty of the games should be appropriately increased.

[0189] Figure 3 This is a schematic diagram of the structure of a cognitive training system provided in an embodiment of this application. The cognitive training system includes an eye-tracking data acquisition module, a data processing and analysis module, a game control and training module, and an evaluation and training model module. The eye-tracking data acquisition module can perform one or more eye-tracking paradigm tasks (such as forward saccades, reverse saccades, memory saccades, bistep saccades, smooth tracking, working memory capacity eye-tracking tasks, exploratory eye-tracking tasks, etc.) through an eye-tracking device (such as an eye tracker). It can also acquire eye-tracking images of subjects performing eye-tracking paradigm tasks, and perform pupil search and localization, eye-tracking trajectory analysis, etc. on the acquired eye-tracking images to obtain test data of subjects corresponding to various eye-tracking paradigm tasks (such as saccade count, fixation duration, saccade velocity, saccade acceleration, pupil position, etc.).

[0190] Subsequently, the test data of the participants corresponding to various eye-tracking paradigm tasks are sent to the data processing and analysis module. This module performs preprocessing such as cleaning and noise reduction on the test data, and extracts the index values ​​(eye-tracking parameters) of multiple ability assessment indicators corresponding to each cognitive ability. Based on the extracted index values ​​and the established cognitive domain assessment model, the participants' ability scores for each cognitive domain are determined. The established cognitive domain assessment model includes multiple ability assessment indicators for each cognitive domain, as well as the weights of each indicator.

[0191] The game control and training module can determine a matching target cognitive training game from the game resource library based on the test subjects' ability scores for various cognitive domains. It then determines the target game difficulty level for each test subject based on game difficulty-related parameters (such as age and cognitive ability scores), thus generating a training game plan for the test subject. The target cognitive training game at the target difficulty level is displayed on the game terminal. During the test subjects' cognitive training game activities, eye movement data (eye movement control and interaction data, including fixation parameters, saccade parameters, smooth tracking, etc.) and game performance data (game completion time, task completion rate, game accuracy, etc.) are recorded.

[0192] The evaluation and training model module can be continuously optimized based on big data analysis and machine learning algorithms. It reassesses participants' levels in various cognitive domains based on eye-tracking data, game performance data, and personal information (age, gender, education level, etc.) during game training, compares changes in cognitive domain levels before and after training, and generates a training evaluation report. Furthermore, the evaluation and training model module can adjust the game difficulty level based on participants' game performance data.

[0193] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to specific embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of this invention and does not strictly limit the scope of protection specifically claimed by this invention.

[0194] Example 1:

[0195] Patient A: Male, 65 years old, suffering from mild cognitive impairment.

[0196] First, eye-tracking calibration was performed on the subjects using an eye tracker. After calibration, initial eye-tracking data collection began, including collecting eye-tracking data under various eye-tracking tasks such as forward, backward, memory, bistep, smooth tracking, and N-back.

[0197] After collecting eye movement data from patient A under various eye movement tasks, the indicator values ​​corresponding to various cognitive abilities (attention, memory, executive function, etc.) were determined. Based on the indicator values ​​of each cognitive ability, patient A's attention level was found to be low or impaired. It was initially determined that patient A had problems with attention concentration and attention switching ability. Combining patient A's age, gender, and eye movement data, it was recommended to start with a simple two-dimensional attention concentration training game, such as the "Target Fixation Stabilization Game", with the initial difficulty set at a low level.

[0198] During game training, patient A controlled the survival status of the game character by maintaining gaze at a target point for a certain period of time and switched items in the game scene by scanning the prompts at the edge of the screen. As the game training progressed, it was found that the trainee's gaze time gradually increased and became more stable, and the saccade path became smoother. Therefore, based on the eye movement data during real-time training, the game difficulty was increased, such as shortening the time interval between the appearance of the target point and increasing the number and complexity of the prompts.

[0199] Once the preset training cycle is reached, a comprehensive assessment of patient A's cognitive abilities is conducted again. If it is found that the trainee has significantly improved in terms of attention concentration and switching ability, the subsequent training plan can be adjusted based on the training assessment results. For example, some memory training game elements can be added to the subsequent training to further improve the overall cognitive function.

[0200] Example 2:

[0201] Patient B: Female, 30 years old, experiencing cognitive decline following brain injury.

[0202] First, an eye tracker was used to calibrate the subjects' eye movements. After calibration, eye movement data of patient B under various eye movement tasks were collected.

[0203] Analysis of patient B's eye movement data revealed that patient B had low scores and significant impairment in executive functions, such as planning and organization. Based on patient B's ability scores across various cognitive domains and the base weights of candidate cognitive training games in the game resource library corresponding to each cognitive domain, game matching scores for patient B were determined. The game with the highest matching score was "3D Fishing," and the initial game difficulty level was set to medium for patient B.

[0204] In the game, Patient B needs to control their eye movements to catch various fish. During the catching process, they need to continuously track and watch the swimming fish with their eyes. Only after tracking the fish steadily for a short period of time can they successfully catch it and get a bonus. When some non-fish items appear (oil drums, plastic bottles, etc.), they cannot be caught and must be avoided, otherwise points will be deducted.

[0205] During training, eye movement parameters such as decision-making time, fixation accuracy, number of fish caught, and number of errors in the game are analyzed based on patient B's eye movement trajectory to continuously optimize the training program. For example, when it is found that the trainee is struggling to catch fast-moving fish, the speed of the fish is reduced, thereby lowering the game's difficulty level. Alternatively, more guidance information can be provided in the game interface or some task steps can be simplified. As the trainee's ability improves, the complexity of the tasks can be gradually restored and new challenges can be added.

[0206] After a period of training, Patient B showed significant improvement in the cognitive domains related to executive function, and was able to better cope with planning and organizing tasks in daily life.

[0207] Based on and Figure 1 Using the same principle as the method for cognitive training based on eye-tracking control technology, this application provides a device for cognitive training based on eye-tracking control technology, such as... Figure 4 As shown, the training device 200 may include: an eye-tracking data acquisition module 210, an ability assessment index determination module 220, a cognitive ability score determination module 230, and a game selection module 240, wherein:

[0208] The eye-tracking data acquisition module 210 is used to acquire test data of subjects corresponding to multiple eye-tracking test tasks; wherein, the multiple eye-tracking test tasks include eye-tracking test tasks corresponding to each of the at least one cognitive ability to be evaluated, and the test data of the eye-tracking test tasks corresponding to each cognitive ability includes eye-tracking test data related to at least one ability evaluation index of that cognitive ability.

[0209] The ability assessment index determination module 220 is used to determine the index values ​​of various ability assessment indicators corresponding to each cognitive ability based on the eye movement test data of the subject related to that cognitive ability.

[0210] The cognitive ability score determination module 230 is used to determine the ability score of the subject for each cognitive ability based on the index values ​​of various ability assessment indicators corresponding to that cognitive ability.

[0211] The game selection module 240 is used to determine a target cognitive training game that matches the subject from a variety of candidate cognitive training games based on the subject's ability scores corresponding to various cognitive abilities.

[0212] Optionally, each of the candidate cognitive training games has a base weight corresponding to various cognitive abilities; a candidate cognitive training game corresponds to a base weight of a cognitive ability, which represents the degree to which the candidate cognitive training game improves that cognitive ability, and the larger the base weight, the greater the improvement;

[0213] The game selection module 240 can be used for:

[0214] For each candidate cognitive training game, based on the subject's ability score for each cognitive ability, the base weight of the candidate cognitive training game for each cognitive ability is adjusted to obtain the adjusted weight of the candidate cognitive training game for each cognitive ability; wherein, the higher the ability score of a candidate cognitive training game for a cognitive ability, the smaller the weight adjustment amount of the candidate cognitive training game for that cognitive ability.

[0215] For each candidate cognitive training game, the game matching score of that candidate cognitive training game is obtained by integrating the adjusted weights of various cognitive abilities corresponding to that candidate cognitive training game.

[0216] The candidate cognitive training game with the highest matching score is selected as the target cognitive training game.

[0217] Optionally, the game selection module 240 can be used for:

[0218] For each cognitive ability, the degree of impairment of the subject corresponding to that cognitive ability is determined based on the subject's ability score for that cognitive ability, wherein the subject's ability score for a cognitive ability is negatively correlated with the degree of impairment;

[0219] For each cognitive ability, the weight adjustment amount of the candidate cognitive training game corresponding to that cognitive ability is determined based on the degree of impairment of the subject corresponding to that cognitive ability and the basic weight of the candidate cognitive training game corresponding to that cognitive ability.

[0220] For each cognitive ability, the adjusted weight of the candidate cognitive training game for that cognitive ability is determined based on the basic weight of the candidate cognitive training game for that cognitive ability and the weight adjustment amount of the candidate cognitive training game for that cognitive ability.

[0221] Optionally, the cognitive ability score determination module 230 can be used for:

[0222] Obtain the indicator weights of each ability assessment indicator for this type of cognitive ability;

[0223] Based on the weights of each indicator, the indicator values ​​of various ability assessment indicators corresponding to the cognitive ability of the test subjects are weighted and fused to obtain the ability score of the test subjects corresponding to the cognitive ability.

[0224] Optionally, the game selection module can also be used for:

[0225] Determine the game difficulty-related parameters for the subjects;

[0226] Based on the game difficulty-related parameters of the test subjects and the correspondence between the game difficulty-related parameters and each game difficulty level, the target game difficulty level of the test subjects corresponding to the target cognitive training game is determined.

[0227] The game difficulty-related parameters of the test subjects include at least one of the following:

[0228] The user attribute information of the subjects includes the user's age or education level;

[0229] The participants' ability scores corresponded to various cognitive abilities;

[0230] The subjects are assigned index values ​​corresponding to at least one cognitive ability assessment index.

[0231] Optionally, the training device further includes a game feedback module, which can be used for:

[0232] The eye movement data of the subject in the target cognitive training game is obtained. The eye movement data includes at least one of the following: the percentage of fixation time of each area of ​​interest in the game interface, the number of eye saccades when fixating each area of ​​interest, and the change in pupil diameter.

[0233] When the eye movement data of the subject meets the first condition, the target cognitive training game is re-determined based on the subject's ability scores corresponding to various cognitive abilities.

[0234] The first condition includes at least one of the following:

[0235] At least one area of ​​interest has a gaze duration percentage less than the first threshold;

[0236] At least one area of ​​interest has fewer eye saccades than the second threshold;

[0237] The pupil diameter change in at least one region of interest is less than the third threshold.

[0238] Optionally, the game feedback module can be used for:

[0239] Acquire eye movement data of the subjects while they observe the game interface of the target cognitive training game;

[0240] Based on the eye movement data of the subjects, determine the index values ​​of the subjects under at least one preset eye movement observation index, wherein the at least one eye movement observation index includes at least one of fixation time, saccade rate and pupil diameter change;

[0241] Based on the index values ​​of the subjects under the at least one eye-tracking observation index, the cognitive state of the subjects is determined by a state assessment model.

[0242] Optionally, the game feedback module can be used for:

[0243] Acquire the game performance data of the subjects; wherein the game performance data includes at least one of game completion time or game accuracy rate;

[0244] Based on the game performance data and the preset game difficulty assessment strategy, the game difficulty of the target cognitive training game for the subjects is assessed, and the difficulty assessment result is obtained.

[0245] Based on the difficulty assessment results, the game difficulty level of the target cognition training game is adjusted.

[0246] Optionally, the at least one cognitive ability includes at least one of memory, attention, executive function, orientation, motor control, and abstract logical reasoning.

[0247] Among them, at least one of the following ability assessment indicators for memory is: memory capacity, memory accuracy, or memory placement error;

[0248] At least one indicator of attentional ability assessment includes at least one of the following: saccade latency, saccade velocity, or probability of abnormal interruption of saccade.

[0249] At least one performance indicator for executive function includes: the incidence of reflex saccades or the average tracking error;

[0250] At least one of the following assessment indicators of orientation ability is: saccade accuracy or landing point error;

[0251] At least one of the following ability assessment indicators for abstract logical reasoning ability is: saccade accuracy or classification accuracy.

[0252] At least one performance indicator for motor control includes at least one of the following: tracking average error, landing error, saccade accuracy, or saccade loss rate.

[0253] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0254] This application provides a training system for cognitive training based on eye-tracking control technology, such as... Figure 5 As shown, the training system 300 includes an eye-tracking acquisition device 310, a memory 320, and a processor 330. The eye-tracking acquisition device 310 is used to collect test data of subjects corresponding to various eye-tracking test tasks. The memory 320 stores a computer program. When the processor 330 executes the computer program stored in the memory 320, it can implement the method in any optional embodiment of this application.

[0255] The memory 320 can be used to store operating systems and applications, etc. The applications can include computer programs that implement the methods shown in the embodiments of the present invention when invoked by the processor 330, and can also include programs for implementing other functions or services. The memory 320 can be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and computer programs, or it can be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0256] Processor 330 is connected to memory 320 via a bus and performs corresponding functions by calling application programs stored in memory 320. Processor 330 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present invention disclosure. Processor 330 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0257] Optionally, in the solution provided by the embodiments of the present invention, the memory 320 can be used to store a computer program that executes the solution of the present invention, and the processor 330 runs the computer program to implement the operation of the method or apparatus provided by the embodiments of the present invention.

[0258] Based on the same principle as the method provided in the embodiments of this application, the embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the corresponding content of the aforementioned method embodiments.

[0259] This application also provides a computer program product, which includes a computer program that, when executed by a processor, can implement the corresponding content of the aforementioned method embodiments.

[0260] It should be noted that the terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0261] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0262] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. An electronic device for cognitive training based on eye-tracking control technology, characterized in that, The electronic device includes a processor, the processor being configured to: Acquire test data of subjects corresponding to multiple eye movement test tasks; wherein, the multiple eye movement test tasks include eye movement test tasks corresponding to each of the at least one cognitive ability to be evaluated, and the test data of the eye movement test tasks corresponding to each cognitive ability include eye movement test data related to at least one ability assessment index of that cognitive ability; For each cognitive ability, based on the eye movement test data of the subjects related to that cognitive ability, the index values ​​of various ability assessment indicators corresponding to that cognitive ability are determined; For each cognitive ability, the subject's ability score corresponding to that cognitive ability is determined based on the index values ​​of various ability assessment indicators corresponding to that cognitive ability. For each candidate cognitive training game, based on the subject's ability score for each cognitive ability, the base weight of the candidate cognitive training game for each cognitive ability is adjusted to obtain the adjusted weight of the candidate cognitive training game for each cognitive ability; wherein, the higher the ability score of a candidate cognitive training game for a cognitive ability, the smaller the weight adjustment amount of the candidate cognitive training game for that cognitive ability. For each candidate cognitive training game, the game matching score of that candidate cognitive training game is obtained by integrating the adjusted weights of various cognitive abilities corresponding to that candidate cognitive training game. The candidate cognitive training game with the highest matching score is selected as the target cognitive training game.

2. The electronic device according to claim 1, characterized in that, Each of the candidate cognitive training games has a base weight corresponding to various cognitive abilities; a candidate cognitive training game corresponds to a base weight of a cognitive ability, which represents the degree to which the candidate cognitive training game improves that cognitive ability, and the larger the base weight, the greater the improvement.

3. The electronic device according to claim 1, characterized in that, For each candidate cognitive training game, the step of adjusting the base weight of the candidate cognitive training game for each cognitive ability based on the subject's ability score for each cognitive ability, to obtain the adjusted weight of the candidate cognitive training game for that cognitive ability, includes: For each cognitive ability, the degree of impairment of the subject corresponding to that cognitive ability is determined based on the subject's ability score for that cognitive ability, wherein the subject's ability score for a cognitive ability is negatively correlated with the degree of impairment; For each cognitive ability, the weight adjustment amount of the candidate cognitive training game corresponding to that cognitive ability is determined based on the degree of impairment of the subject corresponding to that cognitive ability and the basic weight of the candidate cognitive training game corresponding to that cognitive ability. For each cognitive ability, the adjusted weight of the candidate cognitive training game for that cognitive ability is determined based on the basic weight of the candidate cognitive training game for that cognitive ability and the weight adjustment amount of the candidate cognitive training game for that cognitive ability.

4. The electronic device according to claim 1, characterized in that, For each cognitive ability, the step of determining the subject's ability score corresponding to that cognitive ability based on the index values ​​of various ability assessment indicators corresponding to that cognitive ability includes: Obtain the indicator weights of each ability assessment indicator for this type of cognitive ability; Based on the weights of each indicator, the indicator values ​​of various ability assessment indicators corresponding to the cognitive ability of the test subjects are weighted and fused to obtain the ability score of the test subjects corresponding to the cognitive ability.

5. The electronic device according to any one of claims 1 to 4, characterized in that, The processor is also used for: Determine the game difficulty-related parameters for the subjects; Based on the game difficulty-related parameters of the test subjects and the correspondence between the game difficulty-related parameters and each game difficulty level, the target game difficulty level of the test subjects corresponding to the target cognitive training game is determined. The game difficulty-related parameters of the test subjects include at least one of the following: The user attribute information of the subjects includes the user's age or education level; The participants' ability scores corresponded to various cognitive abilities; The subjects are assigned index values ​​corresponding to at least one cognitive ability assessment index.

6. The electronic device according to any one of claims 1 to 4, characterized in that, The processor is also used for: The eye movement data of the subject in the target cognitive training game is obtained. The eye movement data includes at least one of the following: the percentage of fixation time of each area of ​​interest in the game interface, the number of eye saccades when fixating each area of ​​interest, and the change in pupil diameter. When the eye movement data of the subject meets the first condition, the target cognitive training game is re-determined based on the subject's ability scores corresponding to various cognitive abilities. The first condition includes at least one of the following: At least one area of ​​interest has a gaze duration percentage less than the first threshold; At least one area of ​​interest has fewer eye saccades than the second threshold; The pupil diameter change in at least one region of interest is less than the third threshold.

7. The electronic device according to any one of claims 1 to 4, characterized in that, The processor is also used for: Acquire eye movement data of the subjects while they observe the game interface of the target cognitive training game; Based on the eye movement data of the subjects, determine the index values ​​of the subjects under at least one preset eye movement observation index, wherein the at least one eye movement observation index includes at least one of fixation time, saccade rate and pupil diameter change; Based on the index values ​​of the subjects under the at least one eye-tracking observation index, the cognitive state of the subjects is determined by a state assessment model; the cognitive state includes at least one of focus, distraction, and thinking. When the number of times the subject is detected to be in a distracted state reaches a preset value, a new target cognitive training game is determined to match the subject.

8. The electronic device according to claim 5, characterized in that, The processor is also used for: Acquire the game performance data of the subjects; wherein the game performance data includes at least one of game completion time or game accuracy rate; Based on the game performance data and the preset game difficulty assessment strategy, the game difficulty of the target cognitive training game for the subjects is assessed, and the difficulty assessment result is obtained. Based on the difficulty assessment results, the game difficulty level of the target cognition training game is adjusted.

9. The electronic device according to claim 1, characterized in that, The at least one cognitive ability includes at least one of memory, attention, executive function, orientation, motor control, and abstract logical reasoning. Among them, at least one of the following ability assessment indicators for memory is: memory capacity, memory accuracy, or memory placement error; At least one indicator of attentional ability assessment includes at least one of the following: saccade latency, saccade velocity, or probability of abnormal interruption of saccade. At least one performance indicator for executive function includes: the incidence of reflex saccades or the average tracking error; At least one of the following assessment indicators of orientation ability is: saccade accuracy or landing point error; At least one of the following ability assessment indicators for abstract logical reasoning ability is: saccade accuracy or classification accuracy. At least one performance indicator for motor control includes at least one of the following: tracking average error, landing error, saccade accuracy, or saccade loss rate.

10. A device for cognitive training based on eye-tracking control technology, characterized in that, The device includes: An eye-tracking data acquisition module is used to acquire test data of subjects corresponding to multiple eye-tracking test tasks; wherein, the multiple eye-tracking test tasks include eye-tracking test tasks corresponding to each of the at least one cognitive ability to be evaluated, and the test data of the eye-tracking test tasks corresponding to each cognitive ability includes eye-tracking test data related to at least one ability evaluation index of that cognitive ability. The ability assessment index determination module is used to determine the index values ​​of various ability assessment indicators corresponding to each cognitive ability based on the eye movement test data of the subject related to that cognitive ability. The cognitive ability score determination module is used to determine the ability score of the subject for each cognitive ability based on the index values ​​of various ability assessment indicators corresponding to that cognitive ability. The game selection module is used to adjust the base weights of each candidate cognitive training game for each cognitive ability based on the subject's ability score for each cognitive ability, thereby obtaining the adjusted weights of the candidate cognitive training game for each cognitive ability. Specifically, the higher the ability score of a candidate cognitive training game for a particular cognitive ability, the smaller the weight adjustment for that cognitive ability. The adjusted weights of the candidate cognitive training game for each cognitive ability are then combined to obtain the game matching score for that candidate cognitive training game. The candidate cognitive training game with the highest game matching score is selected as the target cognitive training game.

11. A system for cognitive training based on eye-tracking control technology, characterized in that, The system includes an eye-tracking acquisition device and an electronic device according to any one of claims 1 to 9, wherein the eye-tracking acquisition device is used to acquire test data of the subject corresponding to multiple eye-tracking test tasks.

Images