Method and system for cognitive training based on eye movement tracking control technology

Abstract

The embodiment of the invention provides a cognitive training method and system based on an eye movement tracking control technology, and relates to the field of eye movement tracking, and the method comprises the steps: obtaining test data, corresponding to a plurality of eye movement test tasks, of a tested person; wherein the various eye movement test tasks comprise eye movement test tasks corresponding to each cognitive ability to be evaluated, aiming at each cognitive ability, according to eye movement test data, related to the cognitive ability, of the testee, index values of various ability evaluation indexes, corresponding to the cognitive ability, of the testee are determined; and determining the ability score corresponding to the cognitive ability, and determining a matched target cognitive training game according to the ability scores corresponding to various cognitive abilities of the testee. According to the method, the cognitive competence is evaluated based on the eye movement data of the testee, and the matched game is selected according to the evaluation result to train the cognitive competence, so that the interestingness of cognitive training is enhanced, and the training effect of cognitive training is remarkably improved.

Description

Technical Field

[0001] This application belongs to the field of computer technology and may relate to the field of eye movement tracking technology. Specifically, this application relates to a method and system for cognitive training based on eye movement tracking control technology. Background Art

[0002] With the aging of the population, the number of patients with neurodegenerative diseases such as dementia and Alzheimer's disease has been increasing year by year. These diseases mainly damage people's cognitive functions and manifest as mild cognitive impairment (MCI) in their early stages. MCI is a neurodegenerative disease in which cognitive abilities (such as memory, attention, executive function, orientation, etc.) decline rapidly. Without intervention, nearly 10% of MCI patients will progress to dementia every year, with a much higher proportion than the normal elderly population. Therefore, the diagnosis and intervention of MCI are particularly important.

[0003] Numerous research results have shown that there is a close physiological connection between the eyes and the brain. Eye movement is a fine movement precisely controlled by multiple brain regions and neural circuits in the brain. By observing eye movement, the level of brain cognitive function can be investigated, and at the same time, through eye movement control, the relevant brain regions and neural circuits involved can be trained in reverse to improve the brain's physiology and function and enhance the level of cognitive function.

[0004] Therefore, there is an urgent need for a method based on eye movement tracking control technology to improve cognitive training for intervening in neurodegenerative diseases such as MCI. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a method and system for cognitive training based on eye movement tracking control technology that can effectively improve the training effect of cognitive ability. To achieve this purpose, the technical solutions provided by the embodiments of this application are as follows: On the one hand, the embodiments of this application provide a method for cognitive training based on eye movement tracking control technology, characterized in that the method includes: Obtaining test data of the subject corresponding to multiple eye movement test tasks; wherein, the multiple eye movement test tasks include eye movement test tasks corresponding to each cognitive ability in at least one cognitive ability to be evaluated, and the test data of the eye movement test tasks corresponding to each cognitive ability includes eye movement test data related to at least one ability evaluation index of this cognitive ability; For each cognitive ability, determining the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability according to the eye movement test data of the subject related to this cognitive ability; For each cognitive ability, determine the ability score of the subject corresponding to this cognitive ability according to the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability; According to the ability scores of the subject corresponding to various cognitive abilities, determine a target cognitive training game that matches the subject from a variety of candidate cognitive training games.

[0006] Optionally, each of the candidate cognitive training games has a basic weight corresponding to various cognitive abilities; the basic weight of a candidate cognitive training game corresponding to a cognitive ability represents the degree of improvement of this candidate cognitive training game for this cognitive ability, and the greater the basic weight, the greater the improvement; The determining of the target cognitive ability training game that matches the subject from a variety of candidate cognitive training games includes: For each candidate cognitive training game, according to the ability score of the subject corresponding to each cognitive ability, adjust the basic weight of this candidate cognitive training game corresponding to each cognitive ability to obtain the adjusted weight of this candidate cognitive training game corresponding to each cognitive ability; among them, the higher the ability score of a candidate cognitive training game corresponding to a cognitive ability, the smaller the weight adjustment amount of this candidate cognitive training game corresponding to this cognitive ability; For each candidate cognitive training game, fuse the adjusted weights of this candidate cognitive training game corresponding to various cognitive abilities to obtain the game matching score of this candidate cognitive training game; Take the candidate cognitive training game with the highest game matching score as the target cognitive training game.

[0007] Optionally, for each candidate cognitive training game, the adjusting of the basic weight of this candidate cognitive training game corresponding to each cognitive ability according to the ability score of the subject corresponding to each cognitive ability to obtain the adjusted weight of this candidate cognitive training game corresponding to this cognitive ability includes: For each cognitive ability, determine the degree of impairment of the subject corresponding to this cognitive ability according to the ability score of the subject corresponding to this cognitive ability, where the ability score of the subject corresponding to a cognitive ability is negatively correlated with the degree of impairment; For each cognitive ability, determine the weight adjustment amount of this candidate cognitive training game corresponding to this cognitive ability according to the degree of impairment of the subject corresponding to this cognitive ability and the basic weight of this candidate cognitive training game corresponding to this cognitive ability; For each cognitive ability, based on the basic weight of the candidate cognitive training game corresponding to this cognitive ability and the weight adjustment amount of the candidate cognitive training game corresponding to this cognitive ability, determine the adjusted weight of the candidate cognitive training game corresponding to this cognitive ability.

[0008] Optionally, for each cognitive ability, determining the ability score of the subject corresponding to this cognitive ability according to the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability includes: Obtain the index weight of each ability evaluation index of this cognitive ability; According to each of the index weights, perform weighted fusion on the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability to obtain the ability score of the subject corresponding to this cognitive ability.

[0009] Optionally, the method further includes: Determine the game difficulty-related parameters of the subject; According to the game difficulty-related parameters of the subject and the corresponding relationship between the game difficulty-related parameters and each game difficulty level, determine the target game difficulty level of the subject corresponding to the target cognitive training game; Wherein, the game difficulty-related parameters of the subject include at least one of the following: The user attribute information of the subject, and the user attribute information includes user age or education level; The ability scores of the subject corresponding to various cognitive abilities; The index values of at least one ability evaluation index of the subject corresponding to at least one cognitive ability.

[0010] Optionally, the method further includes: Obtain the eye movement data of the subject in the target cognitive training game, and the eye movement data includes at least one of the fixation duration ratio of each interest area in at least one interest area in the game interface of the subject, the number of saccades when fixating on each interest area, and the pupil diameter change; When the eye movement data of the subject meets the first condition, re-determine the target cognitive training game according to the ability scores of the subject corresponding to various cognitive abilities; Wherein, the first condition includes at least one of the following: The fixation duration ratio of at least one interest area is less than the first threshold; The number of saccades of at least one interest area is lower than the second threshold; The pupil diameter change of at least one interest area is less than the third threshold.

[0011] Optionally, the method further includes: Obtaining eye movement data of the subject when observing the game interface of the target cognitive training game; Determining, according to the eye movement data of the subject, an index value of the subject under at least one preset eye movement observation index, where the at least one eye movement observation index includes at least one of fixation time, saccade frequency, and pupil diameter change; Determining the cognitive state of the subject through a state evaluation model according to the index value of the subject under the at least one eye movement observation index.

[0012] Optionally, the method further includes: Obtaining the game performance data of the subject; wherein the game performance data includes at least one of game completion time or game correct rate; Evaluating the game difficulty of the target cognitive training game for the subject based on the game performance data and a preset game difficulty evaluation strategy to obtain a difficulty evaluation result; Adjusting the game difficulty level of the target cognitive training game according to the difficulty evaluation result.

[0013] Optionally, the at least one cognitive ability includes at least one of memory, attention, executive function, orientation, motor control ability, and abstract logical reasoning ability; Wherein, at least one ability evaluation index of memory includes at least one of memory capacity, memory correct rate, or landing error; At least one ability evaluation index of attention includes at least one of saccade latency, saccade speed, or probability of abnormal saccade interruption; At least one ability evaluation index of executive function includes at least one of the incidence of reflexive saccades or average tracking error; At least one ability evaluation index of orientation includes at least one of saccade correct rate or landing error; At least one ability evaluation index of abstract logical reasoning ability includes at least one of saccade correct rate or classification correct rate; At least one ability evaluation index of motor control ability includes at least one of average tracking error, landing error, saccade correct rate, or saccade loss rate.

[0014] On the other hand, an embodiment of the present application further provides a device for cognitive training based on eye movement tracking control technology, and the device includes: An eye movement data acquisition module, configured to acquire test data of a subject corresponding to a plurality of eye movement test tasks; wherein, the plurality of eye movement test tasks include eye movement test tasks corresponding to each cognitive ability among at least one cognitive ability to be evaluated, and the test data of the eye movement test tasks corresponding to each cognitive ability includes eye movement test data related to at least one ability evaluation index of this cognitive ability; An ability evaluation index determination module, configured to, for each cognitive ability, determine the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability according to the eye movement test data of the subject related to this cognitive ability; A cognitive ability score determination module, configured to, for each cognitive ability, determine the ability score of the subject corresponding to this cognitive ability according to the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability; A game selection module, configured to determine a target cognitive training game matching the subject from a plurality of candidate cognitive training games according to the ability scores of the subject corresponding to various cognitive abilities.

[0015] On the other hand, an embodiment of the present application further provides a system for cognitive training based on eye movement tracking control technology. The system includes an eye movement tracking device, a memory, and a processor. The eye movement tracking device is configured to collect test data of the subject corresponding to a plurality of eye movement test tasks. A computer program is stored in the memory, and the processor executes the computer program to implement the method provided in any optional embodiment of the present application.

[0016] On the other hand, an embodiment of the present application further provides a computer-readable storage medium. A computer program is stored in the storage medium, and when the computer program is executed by a processor, the method provided in any optional embodiment of the present application is implemented.

[0017] On the other hand, an embodiment of the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the method provided in any optional embodiment of the present application is implemented.

[0018] The beneficial effects brought by the technical solution provided by the embodiment of the present application are as follows: The method for cognitive training based on eye movement tracking control technology provided by the embodiments of the present application obtains test data of the subject corresponding to multiple eye movement test tasks; among them, the multiple eye movement test tasks include eye movement test tasks corresponding to each cognitive ability to be evaluated. For each cognitive ability, according to the eye movement test data of the subject related to this cognitive ability, the index values of various ability evaluation indexes corresponding to the subject for this cognitive ability are determined, and then the ability score of the subject corresponding to this cognitive ability is determined. According to the ability scores of the subject corresponding to various cognitive abilities, a target cognitive training game matching the subject is determined. This method evaluates cognitive abilities based on the eye movement data of the subject, and selects a matching game according to the evaluation results to train cognitive abilities, enhancing the interest and effectiveness of the training, and significantly improving the training effect of cognitive training. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description in the embodiments of the present application.

[0020] Figure 1 It is a schematic flowchart of a method for cognitive training based on eye movement tracking control technology applicable to the embodiments of the present application; Figure 2 It is a schematic diagram of a cognitive ability evaluation and cognitive training provided by the embodiments of the present application; Figure 3 It is a schematic structural diagram of a cognitive training system provided by the embodiments of the present application; Figure 4 It is a schematic structural diagram of a training device for cognitive training based on eye movement tracking control technology provided by the embodiments of the present application; Figure 5 It is a schematic structural diagram of a training system for cognitive training based on eye movement tracking control technology provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The following describes the embodiments of the present application with reference to the drawings in the present application. It should be understood that the embodiments described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not constitute limitations on the technical solutions of the embodiments of the present application.

[0022] Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the terms "include" and "comprise" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or their combinations supported by the technical field of the present invention, etc. It should be understood that when we say an element is "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term. For example, "A and / or B" can be implemented as "A", or as "B", or as "A and B". When describing multiple (two or more) items, if the relationship between the multiple items is not clearly defined, the multiple items can refer to one, multiple or all of the multiple items. For example, for the description of "parameter A includes A1, A2, A3", it can be implemented that parameter A includes A1 or A2 or A3, or it can also be implemented that parameter A includes at least two of the three items of parameter A1, A2, and A3.

[0023] It should be noted that, in the alternative embodiments of the present application, for the data related to the object information involved (such as the eye data of the user, etc.), when the embodiments in the present application are applied to specific products or technologies, the permission or consent of the object needs to be obtained, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. That is to say, if the embodiments in the present application involve data related to the object, it needs to be obtained under the authorization and consent of the object, the authorization and consent of the relevant department, and in compliance with the relevant laws, regulations and standards of the country and region. In the embodiments, if personal information is involved, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained, and the embodiments also need to be implemented under the authorization and consent of the object.

[0024] First, some terms involved in the embodiments of the present application are explained and described: Fixation: The eyes stare at a certain point without moving and maintain a gaze.

[0025] Saccade: The saccadic movement of the eyes from one fixation point to the next. During the saccade, the visual input is suppressed, and actually, it is not clear to see things.

[0026] Microsaccades: Involuntary small-amplitude eye movements that occur during fixation, accompanied by nystagmus and ocular drift. When microsaccades disappear, human visual perception becomes blurred and then disappears within a very short time due to the adaptation of the visual nervous system. At the same time, microsaccades also play a role in precisely controlling the fixation position in high-precision visual processing.

[0027] Saccade (task): The initial fixation point is displayed at the initial position, and it is detected whether the subject always fixates on the initial fixation point. After a random delay for a certain time, the display of the initial fixation point is stopped, and a target is displayed at a first predetermined position different from the initial position. The subject needs to quickly and accurately look at the target, and saccade parameters such as saccade amplitude, saccade speed, saccade latency, saccade accuracy, and saccade direction can be detected.

[0028] Antisaccade (task): The initial fixation point is displayed at the initial position, and it is detected whether the subject always fixates on the initial fixation point. After a random delay for a certain time, the display of the initial fixation point is stopped, and a target is displayed at a first predetermined position different from the initial position. The subject needs to quickly look at the reverse position (mirror position) of the target, and antisaccade parameters such as antisaccade amplitude, antisaccade speed, antisaccade latency, error saccade rate, and saccade trajectory deviation can be detected.

[0029] Memory saccade (task): The initial fixation point is displayed at the initial position, and it is detected whether the subject always fixates on the initial fixation point. During the display of the initial fixation point, a target briefly flashes at a first predetermined position different from the initial position. After detecting that the subject has fixated on the initial fixation point for a preset time, the display of the initial fixation point is stopped, and the subject needs to look at the first predetermined position in memory. Saccade parameters such as saccade amplitude, saccade speed, saccade accuracy, saccade frequency, and fixation duration can be detected.

[0030] Double-step saccade (task): The initial fixation point is displayed at the initial position, and it is detected whether the subject always fixates on the initial fixation point. A target briefly flashes at a first predetermined position different from the initial position, and then a target briefly flashes at a second predetermined position. After detecting that the subject has fixated on the initial fixation point for a preset time, the display of the initial fixation point is stopped, and the subject needs to perform two consecutive saccades, looking at the first and second predetermined positions in memory in sequence. Saccade parameters such as saccade amplitude, saccade speed, saccade latency, saccade trajectory deviation, and saccade frequency can be detected.

[0031] Smooth pursuit (task): A task used to track continuously moving visual objects of interest in real time. For example, a target is displayed on a display screen according to a predetermined motion trajectory, and the subject is required to follow the target with their eyes as much as possible. Eye movement parameters such as the subject's tracking error, tracking gain (eye movement speed / target movement speed), eye movement speed, and acceleration can be detected.

[0032] N-back working memory task: A working memory test that requires the subject to compare whether the current stimulus is the same as the stimulus presented in the nth previous trial. GO-NO-GO saccade task: The task contains two different types of visual stimuli, namely "GO" stimuli and "NO-GO" stimuli. These two stimuli are significantly distinguishable in terms of appearance, properties, or presentation methods, etc. For example, they can be light points of different colors, figures of different shapes, or symbols in different positions, etc. The subject needs to make different responses according to the type of presented stimulus. When a "GO" stimulus appears, the subject needs to quickly make a saccade response and shift their gaze to the designated target position; while when a "NO-GO" stimulus appears, the subject has to suppress their saccade impulse and maintain fixation at the current position without making a saccade.

[0033] Principal Component Analysis (PCA): A commonly used data analysis technique for data dimensionality reduction, removing redundant information in the data while retaining the main features of the data. It transforms a set of variables that may be correlated through an orthogonal transformation into a set of linearly uncorrelated variables, and these new variables are called principal components.

[0034] The technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application will be described below through the description of several embodiments. It should be noted that the following embodiments can refer to, draw on, or combine with each other. For the same terms, similar features, and similar implementation steps in different embodiments, they will not be described repeatedly.

[0035] A method for cognitive training based on eye movement tracking control technology provided by an embodiment of the present application can be executed by any electronic device, such as a server or a terminal (such as an eye movement tracking device).

[0036] Figure 1 It is a schematic flowchart of a method for cognitive training based on eye movement tracking control technology provided by an embodiment of the present application. The method can include the following steps S110 - step S140, where: Step S110: Obtain the test data of the subject corresponding to multiple eye movement test tasks, where the multiple eye movement test tasks include the eye movement test tasks corresponding to each cognitive ability among at least one cognitive ability to be evaluated, and the test data of the eye movement test tasks corresponding to each cognitive ability includes the eye movement test data related to at least one ability evaluation index of this cognitive ability.

[0037] Eye movement is a fine movement precisely controlled by multiple brain regions and neural circuits in the brain. Therefore, the cognitive function level of the brain can be investigated by observing eye movement. In the embodiments of the present application, the at least one cognitive ability to be evaluated includes one or more cognitive domains among attention, memory, executive ability, orientation ability, motor control ability, and abstract logical reasoning ability. The ability evaluation indexes corresponding to each cognitive ability are constructed based on a large amount of clinical data and experimental data, and the ability evaluation indexes corresponding to different cognitive abilities are not completely the same. The eye movement test tasks corresponding to each cognitive ability include the eye movement test tasks related to the respective ability evaluation indexes of this cognitive ability.

[0038] Exemplarily, the eye movement test tasks corresponding to each cognitive ability and the corresponding ability evaluation indexes can be as shown in Table 1 below:

[0039] It can be understood that for each cognitive ability, when collecting the relevant eye movement test data based on the eye movement test tasks related to each ability evaluation index of this cognitive ability, the acquisition of the index value of each ability evaluation index can be regarded as an eye movement test task, or when some of the ability evaluation indexes correspond to the same eye movement test task, the eye movement test data related to each ability evaluation index among some ability evaluation indexes can be collected based on this eye movement test task. Taking the saccade latency and saccade velocity in the ability evaluation index of attention as an example, the forward saccade task can be used to collect the eye movement test parameters related to the saccade latency and saccade velocity of the subject. In specific implementation, these parameters can be obtained simultaneously through the same forward saccade task; it is also possible to set the acquisition of saccade latency as a separate eye movement test task and the acquisition of saccade velocity as another eye movement test task, and the acquisition of eye movement test parameters can be realized through different eye movement test tasks respectively. The specific division method of the eye movement test tasks in the present application is not limited and can be flexibly set according to actual research needs.

[0040] In the embodiments of the present application, as Figure 2As shown, the subject can perform various eye movement test tasks through an eye tracking device and collect the test data generated when the subject performs each eye movement test task. Among them, the various eye movement test tasks include one or more of tasks such as saccade, antisaccade, memory saccade, double-step saccade, smooth pursuit, working memory capacity, GO-NO-GO saccade, object classification, etc. The test data of the subject corresponding to each eye movement test task includes the eye movement trajectory data of the subject under this eye movement test task, such as the number of saccades, saccade amplitude, fixation point position, saccade speed, saccade acceleration, fixation duration, pupil diameter, saccade latency, spatial error / landing error, and so on.

[0041] Among them, before performing the eye movement test task using the eye tracking device, eye tracking calibration can be performed first. Specifically, the eye tracking device emits near-infrared light towards the subject's eyes, and N calibration points are sequentially displayed on the display screen for a preset duration (such as set to 2s). The subject sequentially fixates on the calibration points displayed on the screen. The camera of the eye tracking device captures the eye images of the subject when observing each calibration point. According to the relative relationship between the corneal reflection spot and the pupil position of the eye in the eye image when observing each calibration point, the least squares method is used to fit the actual position of the calibration point and the fixation point position collected by the eye tracking device, and a position mapping model is established to adjust the calibration parameters of the eye tracking device to ensure that the error between the collected eye movement data and the actual eye movement position is controlled within a preset range (such as 0.15° visual angle), ensuring the accuracy of the subsequent collected test data.

[0042] Step S120: For each cognitive ability, according to the eye movement test data of the subject related to this cognitive ability, determine the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability.

[0043] Optionally, since the subject may have behaviors such as blinking and shaking during the eye movement test task, resulting in inaccurate eye movement test data, before performing the cognitive ability evaluation, the obtained test data can be cleaned first to remove the noise and outliers caused by factors such as blinking and slight head shaking. According to the cleaned eye movement test data of the subject related to this cognitive ability, determine the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability. Among them, the type of denoising algorithm used in the embodiments of the present application is not limited. For example, the Kalman filter algorithm can be used for data cleaning.

[0044] Step S130: For each cognitive ability, according to the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability, determine the ability score of the subject corresponding to this cognitive ability.

[0045] Among them, each cognitive ability corresponds to at least one ability evaluation index (also known as eye movement behavior characteristics). Since different ability evaluation indexes reflect the performance of different brain functions, the importance of different ability evaluation indexes is different. When evaluating each cognitive ability, the index weight of each ability evaluation index of this cognitive ability can be obtained, and according to each index weight, the index values of various ability evaluation indexes corresponding to this cognitive ability of the subject are weighted and fused to obtain the ability score of the subject corresponding to this cognitive ability.

[0046] Exemplarily, the ability evaluation indexes corresponding to the inhibitory function include the incidence rate of reflex saccades and the average error of target tracking. Among them, the index weight of the incidence rate of reflex saccades is 0.6, and the index weight of the average error of target tracking is 0.4. Then, the ability score of the subject corresponding to the inhibitory function = 0.6 Incidence rate of reflex saccades + 0.4 Average error of target tracking.

[0047] Step S140: Determine a target cognitive training game that matches the subject from a variety of candidate cognitive training games according to the ability scores of the subject corresponding to various cognitive abilities.

[0048] Optionally, the game resource library includes a variety of candidate cognitive training games. Various candidate cognitive training games in the game resource library can realize the training of various cognitive abilities by guiding the subject to observe the game screen to generate various eye movement behaviors. For example Figure 2 As shown, the game screen of the target cognitive training game is displayed on the game terminal, and the subject observes the target object in the game screen and executes the corresponding game tasks to train various cognitive abilities. The embodiments of the present application do not limit the types of games included in the game resource library, as long as they can be used to improve at least one cognitive ability.

[0049] As an implementation manner, the game resource library may include a variety of cognitive training games for training different cognitive abilities, such as: attention concentration training games (such as focused target tracking games, attention switching challenge games, etc.), memory training games (such as memory matching games, memory sequence reconstruction games, etc.), executive function training games (such as reverse eye movement control games, task planning simulation games, information sorting and sequencing games, etc.).

[0050] After obtaining the ability scores of the subject corresponding to various cognitive abilities, for each cognitive ability, it is possible to determine whether the cognitive ability is impaired based on the ability score of the subject corresponding to this cognitive ability and the benchmark score corresponding to this cognitive ability. Among them, the benchmark score corresponding to each cognitive ability can be the average ability score or the lowest score of the statistically normal population corresponding to this cognitive ability. If the ability score of the subject corresponding to this cognitive ability is lower than the benchmark score, it can be considered that the subject's cognitive ability of this kind is impaired; otherwise, it is considered that the subject's cognitive ability of this kind is normal. Among them, the impaired cognitive ability is the cognitive ability in which the subject's performance is lower than that of the normal population. Based on the impaired cognitive ability of the subject, a target cognitive training game matching the subject is determined. For example, if the subject's attention is impaired and significantly lower than that of ordinary people, the target cognitive training games matching this subject include various attention concentration training games for improving the subject's attention.

[0051] In the embodiment of the present application, a cognitive assessment result of the subject can be generated based on the ability scores of the subject corresponding to various cognitive abilities and the impaired cognitive ability of the subject, so that the subject can understand his own cognitive situation.

[0052] Optionally, each candidate cognitive training game in the game resource library can be used for the collaborative training of multiple cognitive abilities. Each candidate cognitive training game has a basic weight (default training weight) corresponding to various cognitive abilities. The basic weight of a candidate cognitive training game corresponding to a cognitive ability represents the degree of training improvement of this candidate cognitive training game for this cognitive ability. The greater the basic weight, the greater the improvement. For example, the basic weight of game A corresponding to attention is 0.7, the basic weight corresponding to memory is 0.2, and the basic weight corresponding to executive function is 0.1, indicating that game A is mainly used to improve the user's attention, and also has a certain effect on improving memory, but has a limited effect on improving executive function.

[0053] When selecting a target cognitive training game matching the subject, for each candidate cognitive training game, the basic weight of this candidate cognitive training game corresponding to each cognitive ability can be adjusted according to the ability score of the subject corresponding to each cognitive ability to obtain the adjusted weight of this candidate cognitive training game corresponding to each cognitive ability; among them, the higher the ability score of a candidate cognitive training game corresponding to a cognitive ability, the smaller the weight adjustment amount of this candidate cognitive training game corresponding to this cognitive ability; the adjusted weights of this candidate cognitive training game corresponding to various cognitive abilities are fused to obtain the game matching score of this candidate cognitive training game; the candidate cognitive training game with the highest game matching score is used as the target cognitive training game.

[0054] Optionally, after determining the game matching scores of various candidate cognitive training games corresponding to the subject, for each candidate cognitive training game, if the game matching score of this candidate cognitive training game is greater than the preset game benchmark score, then this candidate cognitive training game is used as a game to be recommended. According to the game matching scores of each game to be recommended corresponding to the subject, determine the recommendation order of each game to be recommended, and in accordance with the recommendation order, sequentially recommend games to the subject.

[0055] Optionally, for each candidate cognitive training game, when adjusting the basic weights of this candidate cognitive training game corresponding to various cognitive abilities based on the ability scores of the subject corresponding to various cognitive abilities, it is possible to first determine the degree of impairment (also known as the degree of defect) of the subject corresponding to various cognitive abilities according to the ability scores of the subject corresponding to various cognitive abilities. Among them, the lower the ability score of the subject corresponding to a cognitive ability, the higher the degree of impairment of the subject corresponding to this cognitive ability; according to the degree of impairment of the subject corresponding to this cognitive ability and the basic weight of this candidate cognitive training game corresponding to this cognitive ability, determine the weight adjustment amount of this candidate cognitive training game corresponding to this cognitive ability; according to the basic weight of this candidate cognitive training game corresponding to this cognitive ability and the weight adjustment amount of this candidate cognitive training game corresponding to this cognitive ability, determine the adjusted weight of this candidate cognitive training game corresponding to this cognitive ability.

[0056] Optionally, in order to enable the subject's severely defective cognitive abilities to be trained and improved, the weight occupied by the subject in the game matching can be strengthened to enhance the importance of the severely defective cognitive abilities. After determining the degree of impairment of the subject corresponding to various cognitive abilities, it is possible to perform an amplification process on the degree of impairment of the subject corresponding to various cognitive abilities, and based on the amplified degree of impairment corresponding to various cognitive abilities, adjust the basic weights of this candidate cognitive training game corresponding to various cognitive abilities to obtain the adjusted weights of this candidate cognitive training game corresponding to each cognitive ability. Among them, adjusting the basic weights of the game corresponding to various cognitive abilities based on the amplified degree of impairment makes the adjusted weight of the cognitive ability with a severe degree of impairment higher, which is convenient for matching the target cognitive training game with a greater improvement in the severely impaired cognitive ability during game matching.

[0057] Suppose there are m games in the game resource library and n cognitive domains to be evaluated. The basic weight matrix of each game corresponding to each cognitive domain is , for each element in the basic weight matrix, it represents the basic weight of the j-th cognitive domain corresponding to the i-th game.

[0058] After determining the ability scores of the subject corresponding to each cognitive domain, the degree of deficiency of the subject corresponding to each cognitive domain can be calculated by the following formula (1): (1) where j represents the j-th cognitive domain, represents the ability score of the subject corresponding to the j-th cognitive domain, represents the degree of deficiency of the subject corresponding to the j-th cognitive domain.

[0059] After that, according to the degree of deficiency of the subject corresponding to each cognitive domain and the basic weight matrix of each game corresponding to each cognitive domain, the adjusted weight matrix of each game corresponding to the cognitive domain is determined , where each element in the weight matrix represents the adjusted weight of the i-th game corresponding to the j-th cognitive domain. By squaring the operation to strengthen the importance of the degree of deficiency, each element in the weight matrix can be expressed as: (2) Then, for each game, the sum of the adjusted weights of the game corresponding to each cognitive domain can be used as the recommended score of the game (i.e., the game matching score between the game and the subject), which can be expressed as: (3) where represents the recommended score of the i-th game, represents the adjusted weight of the i-th game corresponding to the j-th cognitive domain.

[0060] In summary, the recommended score of each game can be expressed as: (4) where m represents the total number of games, n represents the total number of cognitive domains, i represents the i-th game, j represents the j-th cognitive domain, represents the recommended score of the i-th game, represents the basic weight of the i-th game corresponding to the j-th cognitive domain, represents the ability score (vector) of the subject corresponding to the j-th cognitive domain.

[0061] Exemplarily, assume that there are two games in the game resource library, namely Game A and Game B, and the cognitive abilities to be evaluated include three types: attention, memory, and executive ability. Among them, the basic weight of Game A corresponding to attention is 0.7, the basic weight corresponding to memory is 0.2, and the basic weight corresponding to executive ability is 0.1; the basic weight of Game B corresponding to attention is 0.3, the basic weight corresponding to memory is 0.6, and the basic weight corresponding to executive ability is 0.1, as shown in Table 2 below:

[0062] Assume that the score of the subject for attention is 25 points, the score for memory is 50 points, and the score for executive ability is 85 points. Then, according to the degree of deficiency of the subject corresponding to attention, it is 1 - 25 / 100 = 0.75, the degree of deficiency corresponding to memory is 1 - 50 / 100 = 0.5, and the degree of deficiency corresponding to executive ability is 1 - 85 / 100 = 0.15, as shown in Table 3 below:

[0063] According to the degree of deficiency of the subject corresponding to various cognitive abilities, adjust the basic weights of each game corresponding to various cognitive abilities to obtain the adjusted weights of each game corresponding to various cognitive abilities, as shown in Table 4 below:

[0064] Determine the recommended scores (game matching scores) of each game according to the adjusted weights of each game corresponding to the various cognitive abilities of the subject: The recommended score of Game A = 1.09375 + 0.25 + 0.10225 = 1.446 The recommended score of Game B = 0.46875 + 0.75 + 0.10225 = 1.321 Since the recommended score of Game A is the highest, Game A is recommended as the game with the highest matching degree with the subject.

[0065] Optionally, the default game difficulty of each candidate cognitive training game in the game resource library can be set to a fixed value, or each candidate cognitive training game can also have a game difficulty level setting function, so as to make adaptive adjustments according to the cognitive ability evaluation results or personal ability level changes of the subject.

[0066] Specifically, before using the target cognitive training game for cognitive ability training, the game difficulty-related parameters of the subject can be determined. According to the game difficulty-related parameters of the subject and the corresponding relationship between the game difficulty-related parameters and each game difficulty level, the target game difficulty level corresponding to the subject is determined, and the game difficulty level of the target cognitive training game is adjusted to the target game difficulty level. The target cognitive training game with the adjusted difficulty level is used to train the cognitive ability of the subject.

[0067] Among them, the game difficulty-related parameters of the subject include at least one of the following: The user attribute information of the subject; among them, the user attribute information includes the user's age or education level. Among them, the higher the user's age, the lower the corresponding game difficulty level; the higher the user's education level, the higher the corresponding game difficulty level; The ability scores of the subject corresponding to various cognitive abilities. Among them, the lower the ability score of the subject, the lower the game difficulty level matched by the subject; The index values of at least one ability evaluation index corresponding to at least one cognitive ability of the subject.

[0068] In the embodiments of the present application, the difficulty levels of different games can be adjusted by adjusting the moving speed of the target object, the display size of the target object, the number of interfering objects, etc. in the game. Taking the target tracking game as an example, if the subject corresponds to a low difficulty level, the moving speed of the tracked target object is 0.5v, and the size of the target object is set to 1.5S; if the subject corresponds to a medium difficulty level, the moving speed of the tracked target object is 0.7v, and the size of the target object is set to S; if the subject corresponds to a high difficulty level, the moving speed of the tracked target object is v, and the size of the target object is set to 0.8S.

[0069] It can be understood that during the continuous training of the subject through the cognitive training game, the cognitive abilities in all aspects will be improved to a certain extent. In order to recommend the most suitable game for the user, the cognitive ability of the subject can be re-evaluated periodically, so as to recommend a game to the subject based on the latest cognitive ability evaluation result.

[0070] Based on Figure 1The cognitive training method shown in the figure evaluates various cognitive abilities based on the eye movement test data of the subjects under various eye movement test tasks, and selects a matching cognitive training game for targeted improvement according to the scores of the subjects corresponding to various cognitive abilities. This greatly improves the fun and challenge of cognitive training, increases the participation of the subjects, and also significantly improves the effect and efficiency of cognitive function rehabilitation training. It fills the gap in the existing technology in personalized cognitive rehabilitation training driven by eye movement data, meets the rehabilitation needs of the vast number of patients with cognitive dysfunction, and promotes the innovative development of technology in the field of cognitive rehabilitation.

[0071] Moreover, the cognitive training game can set different scenarios and tasks, covering collaborative training in multiple cognitive domains (such as attention, memory, executive ability, etc.), which helps to comprehensively improve the cognitive function of users.

[0072] During the game training process, it is also possible to evaluate whether the subject is interested in the game according to the performance of the subject in the game. When the interest level of the subject in the current game is low, switch to other games to ensure the effectiveness of the training.

[0073] Optionally, obtain the eye movement data of the subject in the target cognitive training game. Among them, the eye movement data during the game includes at least one of the fixation duration ratio of each area of interest in at least one area of interest in the game interface by the subject, the number of saccades when looking at different areas of interest, and the change in pupil diameter. When the eye movement data of the subject meets the first condition, re-determine the target cognitive training game according to the ability scores of the subject corresponding to various cognitive abilities.

[0074] Among them, the first condition includes at least one of the following: The fixation duration ratio of at least one area of interest is less than the first threshold; The number of saccades in at least one area of interest is lower than the second threshold; The change in pupil diameter in at least one area of interest is less than the third threshold.

[0075] Among them, the game interface can be divided into multiple areas of interest (AOIs). According to the eye movement data of the subject during the game, determine the fixation duration of the subject on each AOI, as well as the pupil diameter and the number of saccades generated when looking at different AOIs. According to the fixation duration of the subject on each AOI and the game training duration, determine the fixation duration ratio of the subject on each AOI. Among them, the longer the fixation duration, the more saccades, and the greater the change in pupil diameter (such as increasing or decreasing by more than 10%) indicate that the subject is more interested in the AOI. When dividing the AOIs, the game characters, task objectives, reward areas, etc. in the game interface can be used as an AOI respectively.

[0076] When the eye movement data of the subject meets the first condition, it indicates that the subject has a low interest in the current target cognitive training game, and other games matching the subject are re-determined. For example, if the proportion of the fixation duration of the subject within a certain AOI in the total game duration exceeds 30%, and the number of saccades within this AOI is greater than 15 times and the pupil diameter increases by more than 10%, it is determined that the subject is interested in the content of this AOI; if the proportion of the fixation duration of the subject within multiple AOIs on the game interface is less than 5% and the saccade frequency is low, it is determined that the subject is not interested in the content of this AOI, and other types of games can be switched, such as switching from an action game to a puzzle game.

[0077] In the embodiment of the present application, the game difficulty level can also be adjusted according to the performance of the subject in the game. Specifically, after the subject completes the target cognitive training game, the game performance data of the subject is obtained; wherein, the game performance data includes at least one of the game completion time or the game correct rate; based on the game performance data of the subject and the preset game difficulty evaluation strategy, the game difficulty of the target cognitive training game for the subject is evaluated to obtain a difficulty evaluation result; according to the difficulty evaluation result, the game difficulty level of the target cognitive training game is adjusted.

[0078] Among them, the game difficulty reference data includes the game completion reference time, the game reference correct rate, etc. The game completion reference time can be the average value calculated by statistically analyzing the game completion time of the normal population, and the game reference correct rate can be the average value calculated by statistically analyzing the game correct rate of the normal population.

[0079] Among them, the preset game difficulty evaluation strategy can include at least one of the following: Strategy 1: Determine the average game completion time of the subject; Determine the first completion time difference between the average game completion time and the game completion reference time; Determine the first time proportion of the first completion time difference in the game completion reference time; If the first time proportion exceeds the preset time proportion, lower the game difficulty level; Strategy 2: Determine the average game correct rate of the subject; If the average game correct rate is lower than the game reference correct rate, lower the game difficulty level; Strategy 3: Determine the second completion time difference between the game completion time and the game completion reference time; Determine the second time proportion of the second completion time difference in the game completion reference time; If the number of game times in which the second time ratio exceeds the preset time ratio reaches the preset number of times, lower the game difficulty level; Strategy Four: Determine the game accuracy rate of the subject; If the number of times the game accuracy rate is lower than the game reference accuracy rate reaches the preset number of times, lower the game difficulty level.

[0080] Exemplarily, assume that the game completion time of the subject is more than 30% longer than the game completion reference time for 3 consecutive times, and the game accuracy rate is lower than 70%. Then it is determined that the game difficulty level is too high, and the game difficulty level can be lowered by reducing the target movement speed, increasing the target size, etc.; if the game completion time is 30% shorter than the game completion reference time standard time for 3 consecutive times, and the accuracy rate is higher than 90%, then it is determined that the game difficulty level is too low, and the difficulty level can be increased by increasing the target movement speed, reducing the target size or adding interference items.

[0081] In the embodiment of the present application, the cognitive state (focused, distracted, thinking) of the subject can also be detected in real time based on the eye movement data of the subject during the game training process. Specifically, obtain the eye movement data of the subject when observing the game interface of the target cognitive training game; according to the eye movement data of the subject, determine the index values of the subject under at least one preset eye movement observation index, where at least one eye movement observation index includes at least one of fixation time, saccade frequency, and pupil diameter change; according to the index values of the subject under at least one eye movement observation index, determine the cognitive state of the subject through a state evaluation model. As an optional method, the state evaluation model can adopt a Hidden Markov Model (HMM).

[0082] Among them, when determining the cognitive state through the state evaluation model, the specific implementation steps include: Obtain the preset state transition probability matrix and observation probability matrix; where the state transition probability matrix includes the probabilities of state transitions between cognitive states, and the observation probability matrix includes the probabilities of combinations of index values of different eye movement observation indexes observed in each cognitive state; According to the index values of the subject in at least one eye movement observation index in the current period, the state probability distribution matrix corresponding to different cognitive states in the previous period, the state transition probability matrix, and the observation probability matrix, determine the state probability distribution matrix of the subject corresponding to different cognitive states in the current period; According to the state probability distribution matrix of the subject corresponding to different cognitive states in the current period, determine the cognitive state of the subject in the current period.

[0083] Among them, the multiple optional cognitive states of the subjects include concentration, distraction, and thinking. The state transition probability matrix and the observation probability matrix can be obtained through statistical analysis of a large amount of state data of users at different times.

[0084] When it is detected that the subject is in a distracted state more frequently during the game, it indicates that the subject has a relatively low interest in the current game, and other cognitive training games can be switched for display.

[0085] Optionally, the cognitive ability assessment result also includes the eye movement behavior pattern detection result. Among them, the eye movement behavior pattern detection result can be obtained through the following methods: Determine multiple eye movement behavior indicators related to eye movement behavior pattern detection; According to the test data of the subject corresponding to various eye movement test tasks, determine the eye movement test data of the subject related to multiple eye movement behavior indicators; According to the eye movement test data related to multiple eye movement behavior indicators, determine the indicator values of multiple eye movement behavior indicators; According to the indicator values of multiple eye movement behavior indicators and the reference indicator values of each eye movement behavior indicator, determine the eye movement behavior pattern detection result.

[0086] Among them, the eye movement behavior pattern detection includes the detection of multiple behavior patterns such as fixation stability detection and attention flexibility detection. The eye movement behavior indicators related to each behavior pattern are not completely the same. For example, the eye movement behavior indicators related to fixation stability detection include fixation duration, and the eye movement behavior indicators related to attention flexibility detection include the fluency of the saccade path, the accuracy of the saccade path, and so on.

[0087] For fixation stability detection, the fixation duration of the subject looking at different points can be determined from the test data of the subject corresponding to various eye movement test tasks. According to the fixation duration of the subject looking at different points in each eye movement test task and the fixation duration of the normal population under the same task, the fixation stability of the subject can be judged. Among them, the greater the difference between the fixation duration distribution range of the subject and the fixation duration distribution range of the normal population, the worse the fixation stability of the subject.

[0088] For the detection of attention flexibility, the saccade count, saccade acceleration, eye fixation points, target observation points and other eye movement test data of the subject in various eye movement test tasks can be determined from the test data of the subject corresponding to various eye movement test tasks; according to the saccade count and saccade acceleration of the subject in various eye movement test tasks, the smoothness of the saccade path of the subject in the eye movement test task is determined, wherein, the fewer the saccade count and the smaller the standard deviation of the saccade acceleration, the smoother the saccade path; according to the eye fixation points and target observation points of the subject in various eye movement test tasks, the accuracy of the saccade path of the subject in the eye movement test task is determined, wherein, the smaller the position deviation between the eye fixation points and the target observation points, the higher the accuracy of the saccade path; according to the smoothness and accuracy of the saccade path of the subject in various eye movement test tasks, the attention flexibility of the subject is judged.

[0089] It should be noted that the above eye movement test data can be directly used as the index value of the eye movement behavior index, or used as the index value of the eye movement behavior index after being processed. The embodiments of the present application do not limit this. For example, when the eye movement behavior index is the smoothness of the saccade path, the eye movement test data related to the smoothness of the saccade path includes the saccade count and saccade acceleration, and the smoothness of the saccade path can be determined according to the saccade count and saccade acceleration; when the eye movement behavior index is the fixation duration, the value of the fixation duration in the eye movement test data can be directly read as the index value of the fixation duration.

[0090] In the embodiments of the present application, the subject can continuously carry out cognitive training through the target cognitive training game. When the cognitive training duration reaches the preset duration, the training effect of the cognitive ability of the subject can be evaluated to detect whether the cognitive ability of the subject has been improved. Specifically, any of the following methods can be used for the evaluation of the training effect: Method 1: Redetect the test data of the subject in various eye movement test tasks, and determine the index values of various ability evaluation indexes corresponding to various cognitive abilities of the subject according to the eye movement test data of the subject related to various cognitive abilities. According to the index values of various ability evaluation indexes corresponding to various cognitive abilities of the subject, determine the ability scores corresponding to various cognitive abilities of the subject. According to the ability scores of each cognitive ability of the subject after training and the ability scores of each cognitive ability of the subject before training, evaluate the training effect of the subject in each cognitive ability.

[0091] Method 2: Obtain the eye movement data and game performance data of the test subject during the game training process. Among them, the eye movement data includes the number of saccades, saccade amplitude, fixation point position, fixation duration, etc. of the test subject when observing the game screen, and the game performance data includes the game completion duration, task accuracy rate, number of mistakes, etc. of the test subject; according to the game performance data of the test subject, extract the key feature data, and analyze the key feature data before and after training to evaluate the training effect of the test subject in various cognitive abilities.

[0092] Optionally, when extracting the key feature data from the game performance data, the principal component analysis (PCA) method can be used to extract the key feature data to reduce the data dimension.

[0093] Optionally, when analyzing the key feature data before and after training, the Support Vector Machine (SVM) algorithm can be used to classify the key feature data of the test subject during the entire training process to obtain the key feature data before training and the key feature data after training, and conduct a comparative analysis. For example, the key feature data includes the index values of the energy evaluation indicators corresponding to each cognitive ability. By comparing the index values of the energy evaluation indicators of each cognitive ability before training and the index values of the energy evaluation indicators of each cognitive ability after training, the improvement of the test subject in each energy evaluation indicator before and after training can be obtained.

[0094] Optionally, the training evaluation result can also include the comparative analysis result of the cognitive abilities of the test subject and the normal population of the same age group and the subsequent training suggestions. For example, for a test subject with obvious improvement in attention but slow improvement in memory, the proportion of memory training games in the subsequent training is increased, and the game difficulty is appropriately increased.

[0095] Figure 3 The figure is a schematic structural diagram of a cognitive training system provided by an embodiment of the present application. The cognitive training system includes an eye movement data acquisition module, a data processing and analysis module, a game control and training module, and an evaluation training model module. Among them, the eye movement data acquisition module can execute one or more of the eye movement paradigm tasks (such as forward saccade, reverse saccade, memory saccade, double-step saccade, smooth pursuit, working memory capacity eye movement task, exploratory eye movement task, etc.) through an eye movement tracking device (such as an eye tracker), and collect the eye movement images of the test subject when performing the eye movement paradigm tasks, and perform processing such as pupil search and positioning and eye movement trajectory analysis on the collected eye movement images to obtain the test data (such as the number of saccades, fixation duration, saccade speed, saccade acceleration, pupil position, etc.) of the test subject corresponding to various eye movement paradigm tasks.

[0096] Afterwards, the test data of the subjects corresponding to various eye movement paradigm tasks are sent to the data processing and analysis module. The data processing and analysis module can perform preprocessing such as cleaning and denoising on each test data, and extract the index values (eye movement parameters) of multiple ability evaluation indexes corresponding to each cognitive ability. According to the extracted index values of each ability evaluation index and the established cognitive domain evaluation model, the ability scores of the subjects corresponding to various cognitive domains are determined. Among them, the established cognitive domain evaluation model includes multiple ability evaluation indexes corresponding to each cognitive domain and the index weights corresponding to each ability evaluation index.

[0097] The game control and training module can determine the matching target cognitive training games from the game resource library according to the ability scores of the subjects corresponding to various cognitive domains, and determine the target game difficulty level corresponding to the subjects according to the game difficulty-related parameters of the subjects (such as age, ability scores of cognitive abilities, etc.), that is, generate the training game plan for the subjects. The target cognitive training game with the target game difficulty level is displayed on the game terminal. During the process of the subjects playing the cognitive training game, the eye movement data of the subjects (eye movement control and interaction data, including eye movement parameters such as fixation parameters, saccade parameters, smooth pursuit, etc.) and the game performance data (game completion duration, task completion rate, game accuracy rate, etc.) are recorded.

[0098] The evaluation and training model module can be continuously optimized based on big data analysis and machine learning algorithms. According to the eye movement data, game performance data, and personal basic information (age, gender, education level, etc.) of the subjects during the game training process, the levels of the subjects in each cognitive domain are re-evaluated, and the changes in the cognitive domain levels before and after training are compared to generate a training evaluation report. Moreover, the evaluation and training model module can also adjust the game difficulty level according to the game performance data of the subjects during the game process.

[0099] In order to make the purpose and advantages of the present invention clearer, the following specifically describes the present invention with specific embodiments. It should be understood that the following text only describes one or several specific implementation manners of the present invention and does not strictly limit the scope of protection of the specific claims of the present invention.

[0100] Embodiment 1: Patient A: Male, 65 years old, suffering from mild cognitive impairment First, an eye tracker is used to calibrate the eye movements of the subjects. After the calibration is completed, the initial eye movement data collection is started, including the collection of eye movement data under various eye movement tasks such as forward, backward, memory, double-step, smooth pursuit, N-back, etc.

[0101] After collecting the eye movement data of patient A under various eye movement tasks, the index values of the ability evaluation indexes corresponding to various cognitive abilities (attention, memory, executive ability, etc.) are determined. Based on the index values of the ability evaluation indexes of each cognitive ability, it is detected that the attention level of patient A is low or impaired. It is preliminarily judged that there are problems with the attention concentration and attention switching ability of patient A. Combining the age, gender and eye movement data of patient A, it is recommended to start with simple two-dimensional attention concentration training games, such as the "Target Fixation Stability Game", and the initial difficulty is set at a lower level.

[0102] During the game training process, patient A controls the survival state of the game character by fixing the gaze on the target point for a certain period of time, and switches the props in the game scene by saccading the prompt information on the screen edge. As the game training progresses, it is found that the trainer's fixation time gradually lengthens and the stability improves, and the saccade path is also smoother. Then, according to the eye movement data during the real-time training process, the game difficulty is increased, such as shortening the time interval of the target point appearance, increasing the number and complexity of the prompt information.

[0103] When the preset training cycle is reached, a comprehensive evaluation of the cognitive ability of patient A is carried out again. It is found that the trainer has made significant improvements in attention concentration and switching ability. Then, the subsequent training plan can be adjusted according to the training evaluation results, such as adding some memory training game elements to the subsequent training to further improve its overall cognitive function.

[0104] Example 2: Patient B: Female, 30 years old, with cognitive function decline after brain injury First, an eye tracker is used to calibrate the eye movements of the subjects. After calibration, the eye movement data of patient B under various eye movement tasks are collected.

[0105] By analyzing the eye movement data of patient B, it is obtained that patient B has a relatively low ability score in executive functions, such as planning and organizing ability, and the impairment is relatively obvious. According to the ability scores of patient B corresponding to various cognitive domains, and the basic weights of various candidate cognitive training games in the game resource library corresponding to various cognitive domains, the game matching scores of various candidate cognitive games corresponding to patient B are determined. Among them, the game with the highest game matching score is the "Three-dimensional Fishing" game, and the initial game difficulty level for patient B is set at medium difficulty.

[0106] In the game, patient B needs to control eye movements to catch various fish. During the catching process, it is necessary to continuously track and fixate on the swimming fish with the eyes. After stabilizing the tracking for a short period of time, the fish can be caught successfully, and points will be added after success. When some non-fish items appear (oil drums, plastic bottles, etc.), they cannot be caught and need to be avoided, otherwise points will be deducted.

[0107] During the training process, eye movement parameters such as the decision-making time, fixation accuracy, number of fish caught, number of errors, etc. of the patient B during the game are analyzed based on the eye movement trajectory, and the training plan is continuously optimized. For example, when it is found that the trainer has difficulty in catching fish with a relatively fast speed, the speed of the fish is reduced, thereby reducing the game difficulty level, or more guiding information is provided in the game interface or some task links are simplified. As the ability of the trainer improves, the complexity of the task can be gradually restored and new challenges can be added.

[0108] After a period of training, patient B has shown significant improvement in the cognitive domain related to executive function and is able to better handle planning and organizing tasks in daily life.

[0109] Based on the same principle as the Figure 1 method of cognitive training based on eye movement tracking control technology in Figure 4 shown, the embodiment of the present application provides a device for cognitive training based on eye movement tracking control technology. As shown in FIG., the training device 200 may include: an eye movement data acquisition module 210, an ability evaluation index determination module 220, a cognitive ability score determination module 230, and a game selection module 240, where: The eye movement data acquisition module 210 is configured to acquire test data of the subject corresponding to a variety of eye movement test tasks; wherein, the variety of eye movement test tasks include eye movement test tasks corresponding to each cognitive ability in at least one cognitive ability to be evaluated, and the test data of the eye movement test tasks corresponding to each cognitive ability includes eye movement test data related to at least one ability evaluation index of this cognitive ability; The ability evaluation index determination module 220 is configured to, for each cognitive ability, determine the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability according to the eye movement test data of the subject related to this cognitive ability; The cognitive ability score determination module 230 is configured to, for each cognitive ability, determine the ability score of the subject corresponding to this cognitive ability according to the index values of various ability evaluation indexes of the subject corresponding to this cognitive ability;

[0110] Optionally, each of the candidate cognitive training games has a basic weight corresponding to various cognitive abilities; the basic weight of a candidate cognitive training game corresponding to a cognitive ability represents the degree of improvement of this candidate cognitive training game for this cognitive ability, and the greater the basic weight, the greater the improvement; The game selection module 240 may be configured to: For each candidate cognitive training game, according to the ability scores of the subject corresponding to each cognitive ability, adjust the basic weight of the candidate cognitive training game corresponding to each cognitive ability to obtain the adjusted weight of the candidate cognitive training game corresponding to each cognitive ability; wherein, the higher the ability score of a candidate cognitive training game corresponding to a cognitive ability, the smaller the weight adjustment amount of the candidate cognitive training game corresponding to the cognitive ability. For each candidate cognitive training game, fuse the adjusted weights of the candidate cognitive training game corresponding to various cognitive abilities to obtain the game matching score of the candidate cognitive training game. Take the candidate cognitive training game with the highest game matching score as the target cognitive training game.

[0111] Optionally, the game selection module 240 can be used to: For each cognitive ability, according to the ability score of the subject corresponding to the cognitive ability, determine the degree of impairment of the subject corresponding to the cognitive ability, wherein the ability score of the subject corresponding to a cognitive ability is negatively correlated with the degree of impairment. For each cognitive ability, according to the degree of impairment of the subject corresponding to the cognitive ability and the basic weight of the candidate cognitive training game corresponding to the cognitive ability, determine the weight adjustment amount of the candidate cognitive training game corresponding to the cognitive ability. For each cognitive ability, according to the basic weight of the candidate cognitive training game corresponding to the cognitive ability and the weight adjustment amount of the candidate cognitive training game corresponding to the cognitive ability, determine the adjusted weight of the candidate cognitive training game corresponding to the cognitive ability.

[0112] Optionally, the cognitive ability score determination module 230 can be used to: Obtain the index weights of each ability evaluation index of the cognitive ability. According to the index weights, perform weighted fusion on the index values of various ability evaluation indexes of the subject corresponding to the cognitive ability to obtain the ability score of the subject corresponding to the cognitive ability.

[0113] Optionally, the game selection module can also be used to: Determine the game difficulty related parameters of the subject. According to the game difficulty related parameters of the subject and the corresponding relationship between the game difficulty related parameters and each game difficulty level, determine the target game difficulty level of the subject corresponding to the target cognitive training game. Among them, the game difficulty-related parameters of the subject include at least one of the following: The user attribute information of the subject, where the user attribute information includes the user's age or educational level; The ability scores of the subject corresponding to various cognitive abilities; The index values of at least one ability assessment index of the subject corresponding to at least one cognitive ability.

[0114] Optionally, the training device further includes a game feedback module, and the game feedback module can be used for: Obtain the eye movement data of the subject in the target cognitive training game, where the eye movement data includes at least one of the fixation duration ratio of each interest area in at least one interest area in the game interface, the number of saccades when fixating on each interest area, and the change in pupil diameter; When the eye movement data of the subject meets the first condition, re-determine the target cognitive training game according to the ability scores of the subject corresponding to various cognitive abilities; Among them, the first condition includes at least one of the following: The fixation duration ratio of at least one interest area is less than the first threshold; The number of saccades in at least one interest area is lower than the second threshold; The change in pupil diameter in at least one interest area is less than the third threshold.

[0115] Optionally, the game feedback module can be used for: Obtain the eye movement data of the subject when observing the game interface of the target cognitive training game; According to the eye movement data of the subject, determine the index values of the subject under at least one preset eye movement observation index, where the at least one eye movement observation index includes at least one of fixation time, saccade frequency, and change in pupil diameter; According to the index values of the subject under the at least one eye movement observation index, determine the cognitive state of the subject through a state assessment model.

[0116] Optionally, the game feedback module can be used for: Obtain the game performance data of the subject; among them, the game performance data includes at least one of the game completion time or the game correct rate; Based on the game performance data and a preset game difficulty assessment strategy, evaluate the game difficulty of the target cognitive training game for the subject to obtain a difficulty assessment result; According to the difficulty assessment result, adjust the game difficulty level of the target cognitive training game.

[0117] Optionally, the at least one cognitive ability includes at least one of memory, attention, executive ability, orientation, motor control ability, and abstract logical reasoning ability; Among them, at least one ability evaluation index of memory includes at least one of memory capacity, memory accuracy rate, or landing point error; At least one ability evaluation index of attention includes at least one of saccade latency, saccade speed, or the occurrence probability of abnormal saccade interruption; At least one ability evaluation index of executive ability includes at least one of the occurrence rate of reflexive saccades or the average tracking error; At least one ability evaluation index of orientation includes at least one of saccade accuracy rate or landing point error; At least one ability evaluation index of abstract logical reasoning ability includes at least one of saccade accuracy rate or classification accuracy rate; At least one ability evaluation index of motor control ability includes at least one of the average tracking error, landing point error, saccade accuracy rate, or saccade loss rate.

[0118] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the function of the module or unit.

[0119] In the embodiments of the present application, a training system for cognitive training based on eye movement tracking control technology is provided, as Figure 5 shown. The training system 300 includes an eye movement acquisition device 310, a memory 320, and a processor 330. Among them, the eye movement acquisition device 310 is used to acquire test data of the test subject corresponding to multiple eye movement test tasks. A computer program is stored in the memory 320. When the processor 330 executes the computer program stored in the memory 320, the method in any optional embodiment of the present application can be implemented.

[0120] Among them, the memory 320 can be used to store the operating system, application programs, etc. The application programs can include computer programs that implement the methods shown in the embodiments of the present invention when called by the processor 330, and can also include programs for implementing other functions or services. The memory 320 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and computer programs. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0121] The processor 330 is connected to the memory 320 through a bus and realizes corresponding functions by calling the application programs stored in the memory 320. Among them, the processor 330 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of the present invention. The processor 330 can also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0122] Optionally, for the solution provided by the embodiments of the present invention, the memory 320 can be used to store the computer program for executing the solution of the present invention and is run by the processor 330. When the processor 330 runs the computer program, it realizes the actions of the method or device provided by the embodiments of the present invention.

[0123] Based on the same principle as the method provided in the embodiments of the present application, the embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the corresponding content of the foregoing method embodiments can be implemented.

[0124] The embodiments of the present application also provide a computer program product, which includes a computer program. When the computer program is executed by a processor, the corresponding content of the foregoing method embodiments can be implemented.

[0125] It should be noted that the terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than the illustrated or textually described order.

[0126] It should be understood that although the flowchart in the embodiments of the present application indicates each operation step by an arrow, the execution order of these steps is not limited by the order indicated by the arrow. Unless there is a clear description in this article, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart can be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage among these sub-steps or stages can also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of the present application do not limit this.

[0127] The above are only optional implementation manners of some implementation scenarios of the present application. It should be pointed out that for those of ordinary skill in the art in the technical field, without departing from the technical concept of the solution of the present application, adopting other similar implementation means based on the technical idea of the present application also belongs to the protection scope of the embodiments of the present application.

Claims

1. A method for cognitive training based on eye tracking control technology, characterized in that: The method comprises: Acquire test data corresponding to a plurality of eye movement test tasks of the test person; wherein the plurality of eye movement test tasks include an eye movement test task corresponding to each cognitive ability of at least one cognitive ability to be evaluated, and the test data of the eye movement test task corresponding to each cognitive ability includes eye movement test data related to at least one ability evaluation indicator of the cognitive ability; For each cognitive ability, determining the index values ​​of various ability assessment indicators of the subject corresponding to the cognitive ability according to the eye movement test data of the subject related to the cognitive ability; For each cognitive ability, determining the ability score of the subject corresponding to the cognitive ability according to the index values ​​of various ability assessment indicators of the subject corresponding to the cognitive ability; According to the ability scores of the subject corresponding to various cognitive abilities, a target cognitive training game matching the subject is determined from a plurality of candidate cognitive training games.

2. The method according to claim 1, characterized in that Each candidate cognitive training game has a basic weight corresponding to each cognitive ability; a candidate cognitive training game corresponds to a basic weight of a cognitive ability, which represents the degree to which the candidate cognitive training game improves the cognitive ability, and the greater the basic weight, the greater the improvement; The step of determining a target cognitive ability training game that matches the subject from a plurality of candidate cognitive training games comprises: For each candidate cognitive training game, according to the ability score of the subject corresponding to each cognitive ability, the basic weight of the candidate cognitive training game corresponding to each cognitive ability is adjusted to obtain the adjusted weight of the candidate cognitive training game corresponding to each cognitive ability; wherein, the higher the ability score of a candidate cognitive training game corresponding to a cognitive ability, the smaller the weight adjustment amount of the candidate cognitive training game corresponding to the cognitive ability; For each candidate cognitive training game, the adjusted weights of the candidate cognitive training game corresponding to various cognitive abilities are integrated to obtain a game matching score of the candidate cognitive training game; The candidate cognitive training game with the highest game matching score is used as the target cognitive training game.

3. The method according to claim 2, characterized in that For each candidate cognitive training game, adjusting the basic weight of the candidate cognitive training game corresponding to each cognitive ability according to the ability score of the subject corresponding to each cognitive ability to obtain the adjusted weight of the candidate cognitive training game corresponding to the cognitive ability includes: For each cognitive ability, determining the degree of impairment of the subject corresponding to the cognitive ability according to the subject's ability score corresponding to the cognitive ability, wherein the subject's ability score corresponding to a cognitive ability is negatively correlated with the degree of impairment; For each cognitive ability, according to the degree of impairment of the subject corresponding to the cognitive ability and the basic weight of the candidate cognitive training game corresponding to the cognitive ability, determine the weight adjustment amount of the candidate cognitive training game corresponding to the cognitive ability; For each cognitive ability, the adjusted weight of the candidate cognitive training game corresponding to the cognitive ability is determined based on the basic weight of the candidate cognitive training game corresponding to the cognitive ability and the weight adjustment amount of the candidate cognitive training game corresponding to the cognitive ability.

4. The method according to claim 1 or 2, characterized in that: For each cognitive ability, determining the ability score of the subject corresponding to the cognitive ability according to the index values ​​of various ability evaluation indicators of the subject corresponding to the cognitive ability includes: Obtain the indicator weight of each ability assessment indicator of the cognitive ability; According to the weights of the indicators, the indicator values ​​of the various ability assessment indicators of the subject corresponding to the cognitive ability are weighted and fused to obtain the ability score of the subject corresponding to the cognitive ability.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Determining game difficulty related parameters of the subject; Determining a target game difficulty level of the subject corresponding to the target cognitive training game according to the game difficulty related parameters of the subject and the corresponding relationship between the game difficulty related parameters and each game difficulty level; The game difficulty related parameters of the test subjects include at least one of the following: User attribute information of the test subject, wherein the user attribute information includes the user's age or education level; The subject's ability scores corresponding to various cognitive abilities; The subject has an indicator value of at least one ability assessment indicator corresponding to at least one cognitive ability.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining eye movement data of the subject in the target cognitive training game, the eye movement data including at least one of a percentage of the subject's gaze time on each of at least one interest area in the game interface, a number of eye saccades when gazing at each interest area, and a change in pupil diameter; When the eye movement data of the subject meets the first condition, re-determining the target cognitive training game according to the ability scores of the subject corresponding to various cognitive abilities; The first condition includes at least one of the following: The percentage of fixation time of at least one area of ​​interest is less than a first threshold; The number of saccades in at least one area of ​​interest is below a second threshold; The pupil diameter variation of at least one region of interest is smaller than a third threshold.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquiring eye movement data of the subject when observing the game interface of the target cognitive training game; Determining, according to the eye movement data of the subject, an indicator value of the subject under at least one preset eye movement observation indicator, wherein the at least one eye movement observation indicator comprises at least one of fixation time, scanning frequency, and pupil diameter change; The cognitive state of the subject is determined according to the indicator value of the subject under the at least one eye movement observation indicator through a state assessment model.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Acquiring game performance data of the subject; wherein the game performance data includes at least one of game completion time or game accuracy rate; Based on the game performance data and a preset game difficulty assessment strategy, the game difficulty of the target cognitive training game for the subject is assessed to obtain a difficulty assessment result; According to the difficulty assessment result, the game difficulty level of the target cognitive training game is adjusted.

9. The method according to any one of claims 1 to 8, characterized in that: The at least one cognitive ability includes at least one of memory, attention, executive function, orientation, motor control, and abstract logical reasoning; The at least one memory ability evaluation index includes at least one of memory capacity, memory accuracy or placement error; At least one of the evaluation indicators of attention ability includes: at least one of saccadic latency, saccadic velocity or probability of abnormal saccadic interruption; At least one ability assessment indicator of executive function includes: at least one of the incidence of reflexive saccades or the mean tracking error; At least one ability evaluation index of orientation includes: at least one of saccade accuracy or landing point error; At least one ability assessment indicator of abstract logical reasoning ability includes: at least one of eye saccade accuracy or classification accuracy; At least one ability evaluation indicator of the motor control ability includes: at least one of the following average error, landing point error, saccade accuracy rate or saccade loss rate.

10. A device for cognitive training based on eye tracking control technology, characterized in that: The device comprises: An eye movement data acquisition module, used to acquire test data corresponding to a plurality of eye movement test tasks of the test person; wherein the plurality of eye movement test tasks include an eye movement test task corresponding to each cognitive ability of at least one cognitive ability to be evaluated, and the test data of the eye movement test task corresponding to each cognitive ability includes eye movement test data related to at least one ability evaluation indicator of the cognitive ability; An ability evaluation index determination module is used to determine, for each cognitive ability, the index values ​​of various ability evaluation indexes of the subject corresponding to the cognitive ability according to the eye movement test data of the subject related to the cognitive ability; A cognitive ability score determination module, for determining, for each cognitive ability, the ability score of the subject corresponding to the cognitive ability according to the index values ​​of various ability evaluation indicators of the subject corresponding to the cognitive ability; The game selection module is used to determine a target cognitive training game that matches the subject from a plurality of candidate cognitive training games according to the subject's ability scores corresponding to various cognitive abilities.

11. A system for cognitive training based on eye tracking control technology, characterized in that: The system includes an eye movement acquisition device, a memory and a processor. The eye movement acquisition device is used to collect test data of the subject corresponding to a variety of eye movement test tasks. The memory stores a computer program. The processor executes the computer program to implement the method described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images