A cognitive function evaluation system based on eye movement and its evaluation method
Through an eye movement-based cognitive function evaluation system, the eye movement data is evaluated using machine learning methods, and the problem of time-consuming and inaccurate cognitive function evaluation in the prior art is solved, and rapid and objective cognitive function evaluation is achieved. , providing more accurate evaluation results.
Patent Information
- Application Number
- CN202210654152.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-10
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-06-10
AI Technical Summary
The existing cognitive function assessment methods are affected by cultural differences, language, education level and other factors, and are time-consuming and lack objective and reliable methods suitable for large-scale popularization. It is difficult for existing eye movement technologies to accurately reflect changes in individual cognitive functions.
Using an eye movement-based cognitive function evaluation system, through experimental paradigm module, data acquisition module and intelligent evaluation module, combined with machine learning methods such as decision trees and neural networks, eye movement data is learned and trained to achieve cognitive function evaluation results. Automatic judgment and conclusion output.
It realizes rapid, objective and accurate cognitive function evaluation, which can comprehensively evaluate memory function, executive function, attention, cognitive flexibility and visual space function, and provides more accurate cognitive function status judgment.
Smart Images

Figure CN115047969B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of cognitive function assessment, and in particular to an eye movement-based cognitive function assessment system and an assessment method thereof. Background Art
[0002] Cognitive function is the intelligent processing process of the body to recognize and acquire knowledge, involving a series of psychological and social behaviors such as learning, memory, language, thinking, spirit, and emotion. Studies have shown that with the increase of age, the individual's brain and cognitive function are constantly changing. In the early stage, the individual's brain and cognitive level are on the rise; but as the age continues to increase, both will decline to varying degrees. Especially in the later stages of the life cycle, cognitive function may decline rapidly due to factors such as brain trauma and brain infection, thus affecting the individual's quality of life and life span. Assessing the cognitive function level of each age group can not only reflect the individual's current cognitive function status, but also explore the development of cognitive level with age. It also helps to identify abnormal conditions and is of great value.
[0003] At present, the assessment of cognitive function level is mainly based on scales, such as the Wechsler Intelligence Scale for Children, the Denver Developmental Screening Test, the McCarthy Children's Ability Scale for assessing childhood, the MMSE scale for assessing the cognitive level of the elderly, the MoCA scale, etc. However, these scale assessments are affected by many factors such as cultural differences, language, education level, environment, and the professional level of the assessor, and are time-consuming. Due to the limitations of the above-mentioned cognitive assessment functions, there is currently a lack of cognitive function assessment methods suitable for large-scale popularization and promotion in the community, so a more objective, reliable, and simple cognitive assessment method is needed.
[0004] Eye tracking refers to the collection of relevant data on the processing of visual information by recording the eye movement trajectory and pupil changes of people when processing visual information, such as reaction time, accuracy, number of eye saccades, fixation time, etc. These data are closely related to cognitive function and can reflect the individual's cognitive level. Therefore, eye tracking technology is considered to be a new technology and method for cognitive function assessment. The individual's cognitive level includes memory function, executive function, attention, visual-spatial function, etc. However, the existing eye movement technology mostly adopts universal task paradigms, which are mainly used to evaluate executive function, so it cannot accurately and comprehensively reflect the individual's current cognitive level and track subsequent changes in cognitive trajectory. In addition, although the indicators of eye movement technology are relatively clear, the indicators are independent of each other and cannot truly reflect the cognitive function of a certain field and the overall cognitive function level. For example, when indicator A is abnormal and indicator B is normal, it is rather one-sided to judge that the cognitive function is at a declining / normal level only by A abnormal / B normal. Therefore, it is necessary to use a more accurate and advanced algorithm to comprehensively reflect the "value" of the cognitive function and the overall cognitive function in a certain field according to the weight and score of each indicator, and to divide the cognitive function state. Summary of the invention
[0005] In order to solve the above technical problems, the present invention proposes a cognitive function assessment system based on eye movements and an assessment method thereof, which covers the assessment of memory function, executive function and attention, cognitive flexibility, visual-spatial function and abstract function based on eye movements, and uses machine learning methods such as decision tree-based ensemble learning methods and neural networks to conduct learning and training on the data collected by eye movement tracking, so as to realize automatic judgment of cognitive function assessment results and output of conclusions; it can quickly assess the cognitive function level of an individual, and give assessment results and conclusions, which has great practical significance.
[0006] An object of the present invention is to provide a cognitive function assessment system based on eye movement.
[0007] The eye movement-based cognitive function assessment system of the present invention comprises: an experimental paradigm module, a data acquisition module, a data processing module and an intelligent assessment module; wherein:
[0008] The experimental paradigm module includes: memory assessment experimental paradigm, executive function and attention experimental paradigm, cognitive flexibility experimental paradigm, visual-spatial function experimental paradigm and abstract function experimental paradigm. The experimental paradigm module provides experimental paradigms for users to perform cognitive function assessment;
[0009] The data acquisition module uses an eye tracker to collect the eye movement data generated by the user during the execution of the experimental paradigm through eye tracking technology, and synchronously sends the collected eye movement data to the data processing module; the corresponding collected eye movement data include: memory assessment data, executive function and attention data, cognitive flexibility data, visual-spatial function data and abstract function data;
[0010] The data processing module pre-processes the eye movement data collected by the data processing module to obtain characteristic parameters of the eye movement data;
[0011] The intelligent assessment module stores a machine learning SVR (support vector) regression model and a machine learning classification decision tree model. The intelligent assessment module uses the machine learning SVR regression model and the machine learning classification decision tree model to perform cognitive function assessment on the characteristic parameters of eye movement data to obtain cognitive function scores, and uses the classification model to divide cognitive function categories to achieve intelligent output of cognitive function scores and classification judgments.
[0012] The memory assessment experimental paradigm includes two experimental tasks: a memory scanning task and a picture memory task. In the memory scanning task, a central object and a polygonal figure are presented on the screen, and the polygonal figure appears randomly on both sides of the screen. The display time of the polygonal figure is set. The picture memory task uses materials consisting of 5 to 10 groups of unrelated pictures, each group contains 5 to 20 pictures. In the picture memory task, one picture is randomly selected from each group and presented in the center of the screen. These selected pictures are called picture memory targets. After 4 to 10 minutes, a new group of pictures is presented in the center of the screen. In this new group of pictures, some pictures are the same as the picture memory targets, that is, pictures that have appeared before, and some pictures are different from the picture memory targets, that is, pictures that have not appeared before, and pictures that have not appeared before are called picture memory interferers.
[0013] The executive function and attention experimental paradigm includes the executive function and attention forward saccade task and the executive function and attention reverse saccade task; during the executive function and attention forward saccade task and the reverse saccade task, a central object and polygonal figures are presented on the screen, and these polygonal figures will appear randomly on both sides of the screen, and the display time of the polygonal figures is set to 2 to 5 seconds.
[0014] The cognitive flexibility experimental paradigm includes the cognitive flexibility forward saccade task and the cognitive flexibility reverse saccade task; in the cognitive flexibility forward saccade task, a figure with a set color appears on the screen, which is called the cognitive flexibility forward saccade task prompt figure, and then polygonal figures appear randomly on the left and right sides of the screen, staying for 2s to 5s; in the cognitive flexibility reverse saccade task, a figure with a different color from the cognitive flexibility forward saccade task prompt figure appears on the screen, which is called the cognitive flexibility reverse saccade task prompt figure, and then a polygonal figure appears randomly on the left and right sides of the screen, staying for 2s to 5s.
[0015] The visuospatial function experimental paradigm adopts a visual search paradigm. A picture or graphic is presented at the top of the screen, which is called the visuospatial function target. At the same time, a group of pictures or graphics are presented at the bottom of the screen, which includes a picture or graphic that is exactly the same as the visuospatial function target; 3 to 5 pictures or graphics that are the same as the visuospatial function target but have one or more different pictures or graphics are called visuospatial function distractors.
[0016] The abstract function experimental paradigm adopts a visual search paradigm. A group of pictures, 4 to 8 pictures, are presented in the center of the screen. In this group of pictures, there is one picture that belongs to a different category from the rest. For example, among the four pictures of apple, banana, cantaloupe, and tomato, tomato is a vegetable, and the other three pictures are fruits. This picture is called the abstract function target, and the rest are called abstract function distractors.
[0017] When the user performs a memory scanning task, the data acquisition module collects the memory scanning data generated when the user looks at the location of the polygonal figure after the central objects disappear. The memory scanning data includes the memory scanning peak velocity, memory scanning latency, memory scanning accuracy, memory scanning redundant scanning incidence rate and memory scanning error correction rate; when the user performs a picture memory task, the data acquisition module collects the picture memory data generated when the user recalls and selects the picture memory target, including the number of eye movement fixation points, fixation duration, number of correct recalls, average reaction time of correct recalls and number of incorrect recalls; memory scanning data and picture memory data are collectively referred to as memory assessment data.
[0018] When the user performs the executive function and attention forward scanning task, the data collection module collects the executive function and attention forward scanning data generated in the process of the user looking at the location of the polygonal figure after the central objects disappear, and the executive function and attention forward scanning data include the forward executive function and attention peak velocity, the forward executive function and attention latency, the forward executive function and attention accuracy and the forward executive function and attention error correction rate; when the user performs the executive function and attention reverse scanning task, the data collection module collects the executive function and attention reverse scanning data generated in the process of the user looking at the opposite side of the location of the polygonal figure after the central objects disappear, and the executive function and attention reverse scanning data include the reverse executive function and attention peak velocity, the reverse executive function and attention latency, the reverse executive function and attention accuracy and the reverse execution function and attention error correction rate; the executive function and attention forward scanning data and the executive function and attention reverse scanning data are collectively referred to as executive function and attention data.
[0019] When the user performs a cognitive flexibility forward scanning task, the data collection module collects cognitive flexibility forward scanning data generated when the user looks at the location of the polygonal figure. The cognitive flexibility forward scanning data includes positive cognitive flexibility peak velocity, positive cognitive flexibility latency, positive cognitive flexibility accuracy and positive cognitive flexibility error correction rate; when the user performs a cognitive flexibility reverse scanning task, the data collection module collects cognitive flexibility reverse scanning data generated when the user looks at the opposite side of the location of the polygonal figure. The cognitive flexibility reverse scanning data includes reverse cognitive flexibility peak velocity, reverse cognitive flexibility latency, reverse cognitive flexibility accuracy and reverse cognitive flexibility error correction rate; cognitive flexibility forward scanning data and cognitive flexibility reverse scanning data are collectively referred to as cognitive flexibility data.
[0020] When the user performs the visual search paradigm of the visual-spatial function experimental paradigm, the data acquisition module collects the visual-spatial function data generated by the user in the process of selecting pictures or graphics that are exactly the same as the visual-spatial function target. The visual-spatial function data include the total fixation time of the visual-spatial function, the fixation time of the visual-spatial function target, the total number of visual-spatial function fixations, the number of fixations on the visual-spatial function target and the visual-spatial function accuracy.
[0021] When the user performs the visual search paradigm of the abstract function experimental paradigm, the data acquisition module collects the abstract function data generated by the user in the process of selecting the abstract function target. The abstract function data includes the total gaze time of the abstract function, the gaze time of the abstract function target, the total number of abstract function gazes, the number of gazes on the abstract function target and the abstract function accuracy.
[0022] Another object of the present invention is to provide a method for evaluating cognitive function based on eye movements.
[0023] The eye movement-based cognitive function assessment method of the present invention comprises the following steps:
[0024] 1) Design experimental paradigm module:
[0025] The experimental paradigm module includes five modules: memory assessment experimental paradigm, executive function and attention experimental paradigm, cognitive flexibility experimental paradigm, visual spatial function experimental paradigm and abstract function experimental paradigm. The experimental paradigm module provides users with experimental paradigms for cognitive function assessment.
[0026] 2) The data acquisition module collects eye movement data through eye tracking technology:
[0027] The data acquisition module uses an eye tracker to collect the eye movement data generated by the user during the execution of the experimental paradigm through eye tracking technology, and synchronously sends the collected eye movement data to the data processing module; the corresponding collected eye movement data include: memory assessment data, executive function and attention data, cognitive flexibility data, visual-spatial function data and abstract function data;
[0028] a) Collect memory assessment data:
[0029] When the user performs the memory saccade task, the data acquisition module acquires the memory saccade data generated when the user looks at the location of the polygonal figure after all the central objects presented on the screen disappear. The memory saccade data includes the memory saccade peak velocity, memory saccade latency, memory saccade accuracy, memory saccade redundant saccade occurrence rate and memory saccade error correction rate;
[0030] When the user performs the picture memory task, the data collection module collects the picture memory data generated by the user in the process of recalling and selecting the picture memory target, including the number of eye movement fixation points, fixation duration, number of correct recalls, average reaction time of correct recalls, and number of incorrect recalls;
[0031] b) Collect executive function and attention data:
[0032] When the user performs the executive function and attention forward scanning task, the data collection module collects the executive function and attention forward scanning data generated in the process of the user looking at the location of the polygonal figure after the central objects presented on the screen disappear. The executive function and attention forward scanning data include the forward executive function and attention peak velocity, the forward executive function and attention latency, the forward executive function and attention accuracy and the forward executive function and attention error correction rate;
[0033] When the user performs the executive function and attention reverse saccade task, the data acquisition module acquires the executive function and attention reverse saccade data generated when the user looks at the opposite side of the position of the polygonal figure after the central objects presented on the screen disappear. The executive function and attention reverse saccade data include reverse executive function and attention peak velocity, reverse executive function and attention latency, reverse executive function and attention accuracy rate and reverse executive function and attention error correction rate;
[0034] c) Collect cognitive flexibility data:
[0035] When the user performs the cognitive flexibility forward scanning task, the data collection module collects the cognitive flexibility forward scanning data generated when the user looks at the location of the polygonal figure. The cognitive flexibility forward scanning data includes the positive cognitive flexibility peak velocity, the positive cognitive flexibility latency, the positive cognitive flexibility accuracy and the positive cognitive flexibility error correction rate.
[0036] When the user performs the cognitive flexibility reverse saccade task, the data collection module collects cognitive flexibility reverse saccade data generated when the user looks at the opposite side of the polygonal figure. The cognitive flexibility reverse saccade data includes reverse cognitive flexibility peak velocity, reverse cognitive flexibility latency, reverse cognitive flexibility accuracy and reverse cognitive flexibility error correction rate.
[0037] d) Collecting visual-spatial function data:
[0038] When the user performs the visual search paradigm of the visual-spatial function experimental paradigm, the data acquisition module collects the visual-spatial function data generated by the user in the process of selecting a picture or graphic that is exactly the same as the visual-spatial function target. The visual-spatial function data includes the total visual-spatial function fixation time, the visual-spatial function target fixation time, the total number of visual-spatial function fixations, the number of fixations on the visual-spatial function target, and the visual-spatial function accuracy;
[0039] e) Collect abstract functional data:
[0040] When the user performs the visual search paradigm of the abstract function experiment paradigm, the data collection module collects the abstract function data generated by the user in the process of selecting the abstract function target object, and the abstract function data includes the total fixation time of the abstract function, the fixation time of the abstract function target object, the total number of fixations on the abstract function, the number of fixations on the abstract function target object and the accuracy rate of the abstract function;
[0041] 3) Data processing module performs data preprocessing:
[0042] The data processing module pre-processes the eye movement data collected by the data processing module to obtain characteristic parameters of the eye movement data, including filling missing values, smoothing noise data, smoothing or deleting outliers, deleting irrelevant data, and integrating and arranging data;
[0043] 4) Evaluate cognitive function scores using regression models:
[0044] The intelligent evaluation module constructs a machine learning SVR regression model, uses the eye movement data of a large number of people and the cognitive function score results obtained by the existing common cognitive function scale as samples and labels for training the machine learning SVR regression model, and trains the machine learning SVR regression model. During the training process, GridSearch is used to search for model parameters. For different model parameters, cyclic training is performed until the best regression result is obtained. A trained machine learning SVR regression model is obtained, and a genetic algorithm is used to perform feature selection on the eye movement data feature parameters to obtain the optimal feature parameter subset. Then, the machine learning SVR regression model is used to perform cognitive function evaluation on the feature parameter subset of the eye movement data feature parameters to obtain a cognitive function score.
[0045] 5) Use classification models to classify cognitive functions:
[0046] The intelligent assessment module uses the existing commonly used scale scores of cognitive functions as data labels, combines the eye movement data feature parameters as training data, uses a decision tree for classification calculation, and establishes and trains a machine learning classification decision tree model; in the process of establishing the machine learning classification decision tree model, all data of the above eye movement data feature parameters are used as features, starting from the root node, the information gain of all features of the node is calculated, and the feature with the largest information gain is selected as the root node feature, and sub-nodes are established according to different values of the root node feature; a new sub-node is generated for each sub-node in the same way until the information gain is very small or no feature can be selected, and finally a machine learning classification decision tree model is obtained; the cognitive function scores are classified through the machine learning classification decision tree model, and the expected classification of the cognitive function is output.
[0047] When the data scale increases significantly and there is a lot of noise data, the gradient decision tree (GBDT) method will be used. By adopting an additive model (i.e., a linear combination of basis functions) to integrate multiple machine learning classification decision tree models, the residual error generated by the training process is continuously reduced to achieve the algorithm for classifying data. The system uses the open source extreme gradient boosting (XGBoost, eXtreme Gradient Boosting) algorithm to implement GBDT, speed up learning, and improve model generalization.
[0048] Among them, in step 1), the memory assessment experimental paradigm includes two experimental tasks: a memory scanning task and a picture memory task; in the memory scanning task, a central object and a polygonal figure are presented on the screen, wherein the polygonal figure appears randomly on both sides of the screen; the display time of the polygonal figure is set; the picture memory task uses a material consisting of 5 to 10 groups of unrelated pictures, each group containing 5 to 20 pictures, and in the picture memory task, one picture is randomly selected from each group of pictures and presented in the center of the screen, and these selected pictures are called picture memory targets; after 4 minutes to 10 minutes, a new group of pictures is presented in the center of the screen, in this new group of pictures, some pictures are the same as the picture memory targets, that is, pictures that have appeared before, and some pictures are different from the picture memory targets, that is, pictures that have not appeared before, and pictures that have not appeared before are called picture memory interferers.
[0049] The executive function and attention experimental paradigm includes the executive function and attention forward saccade task and the executive function and attention reverse saccade task; during the executive function and attention forward saccade task and the reverse saccade task, a central object and polygonal figures are presented on the screen, and these polygonal figures will appear randomly on both sides of the screen, and the display time of the polygonal figures is set.
[0050] The cognitive flexibility experimental paradigm includes a cognitive flexibility forward saccade task and a cognitive flexibility reverse saccade task. In the cognitive flexibility forward saccade task, a graphic with a set color is presented on the screen. The graphic with the set color is called the cognitive flexibility forward saccade task prompt picture. Then, polygonal graphics appear randomly on the left and right sides of the screen and stay for 2s to 5s. In the cognitive flexibility reverse saccade task, a graphic with a different color from the cognitive flexibility forward saccade task prompt picture is presented on the screen. It is called the cognitive flexibility reverse saccade task prompt picture. Then, a polygonal graphic appears randomly on the left and right sides of the screen and stays for 2s to 5s. The eye movement data generated by the user looking at the opposite side of the polygonal graphic is recorded.
[0051] The visuospatial function experimental paradigm adopts a visual search paradigm. A picture or graphic is presented at the top of the screen, which is called the visuospatial function target. At the same time, a group of pictures or graphics are presented at the bottom of the screen, which includes a picture or graphic that is exactly the same as the visuospatial function target; 3 to 5 pictures or graphics that are similar to the visuospatial function target but have one or more different pictures or graphics, which are called visuospatial function distractors.
[0052] The abstract function experimental paradigm adopts a visual search paradigm. A group of pictures, 4 to 8 pictures, are presented in the center of the screen. In this group of pictures, there is one picture that belongs to a different category from the rest. For example, among the four pictures of apple, banana, cantaloupe, and tomato, tomato is a vegetable, and the other three pictures are fruits. This picture is called the abstract function target, and the rest are called abstract function distractors.
[0053] In step 2) of b), the peak velocity and latency can be found in the textbook "Eye Tracking User Experience Optimization Operation Guide"; the accuracy rate refers to the proportion of the number of tests in which the user correctly looks at the specified position as required to the total number of tests; the excess saccade incidence rate refers to the proportion of the number of tests in which the user mistakenly looks at the central object to the total number of tests.
[0054] In step 2) b), the error correction rate refers to the ratio of the number of tests in which the user corrects the error after the first scanning error in the reverse scanning task to the total number of tests.
[0055] Advantages of the present invention:
[0056] (1) The present invention uses eye tracking technology to evaluate cognitive function, which has good reliability, simple operation, easy implementation, objectivity and accuracy. It can solve the disadvantages of traditional cognitive assessment, which is time-consuming and labor-intensive, and has unstable results, and is convenient for large-scale evaluation and promotion;
[0057] (2) The present invention adopts a variety of experimental paradigms, which can evaluate cognitive functions in various fields, solve the drawback of existing eye movement technology that only evaluates executive functions, and is conducive to the comprehensive evaluation of the individual's cognitive function level;
[0058] (3) The present invention uses deep machine learning methods to model the multi-dimensional results of eye movement experiments and intelligently output cognitive assessment results to conduct cognitive function assessments in a more accurate, convenient and objective manner. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 is a structural block diagram of the eye movement-based cognitive function assessment system of the present invention;
[0060] Figure 2 The flowchart of the eye movement-based cognitive function assessment method of the present invention is shown in FIG. DETAILED DESCRIPTION
[0061] The present invention will be further described below through specific embodiments in conjunction with the accompanying drawings.
[0062] like Figure 1 As shown, the eye movement-based cognitive function assessment system of this embodiment includes: an experimental paradigm module, a data acquisition module, a data processing module and an intelligent assessment module; wherein,
[0063] The experimental paradigm module includes: memory assessment experimental paradigm, executive function and attention experimental paradigm, cognitive flexibility experimental paradigm, visual-spatial function experimental paradigm and abstract function experimental paradigm. The experimental paradigm module provides experimental paradigms for users to perform cognitive function assessment;
[0064] The data acquisition module uses an eye tracker to collect the eye movement data generated by the user during the execution of the experimental paradigm through eye tracking technology, and synchronously sends the collected eye movement data to the data processing module; the corresponding collected eye movement data include: memory assessment data, executive function and attention data, cognitive flexibility data, visual-spatial function data and abstract function data;
[0065] The data processing module pre-processes the eye movement data collected by the data processing module to obtain characteristic parameters of the eye movement data;
[0066] The intelligent assessment module stores a machine learning SVR (support vector) regression model and a machine learning classification decision tree model. The intelligent assessment module uses the machine learning SVR regression model and the machine learning classification decision tree model to perform cognitive function assessment on the characteristic parameters of eye movement data to obtain cognitive function scores and use classification models to divide cognitive function categories, thereby realizing intelligent output of cognitive function scores and classification judgments.
[0067] The eye movement-based cognitive function assessment method of this embodiment includes the following steps:
[0068] 1) Design experimental paradigm module:
[0069] The experimental paradigm module includes five: memory assessment experimental paradigm, executive function and attention experimental paradigm, cognitive flexibility experimental paradigm, visual spatial function experimental paradigm and abstract function experimental paradigm. The experimental paradigm module provides users with experimental paradigms for cognitive function assessment:
[0070] The memory assessment experimental paradigm includes two experimental tasks: memory scanning task and picture memory task. In the memory scanning task, the central object and polygonal figures are presented on the screen, and the polygonal figures appear randomly on both sides of the screen. The display time of the polygonal figures is set. The picture memory task uses materials composed of 5 groups of unrelated pictures, each group contains 5 pictures. In the picture memory task, one picture is randomly selected from each group and presented in the center of the screen. These selected pictures are called picture memory targets. After 4 to 10 minutes, a new group of pictures is presented in the center of the screen. In this new group of pictures, some pictures are the same as the picture memory targets, that is, pictures that have appeared before, and some pictures are different from the picture memory targets, that is, pictures that have not appeared before, and pictures that have not appeared before are called picture memory interference objects.
[0071] The executive function and attention experimental paradigm includes the executive function and attention forward saccade task and the executive function and attention reverse saccade task; during the executive function and attention forward saccade task and the reverse saccade task, the screen presents a central object and a polygonal figure, which will randomly appear on both sides of the screen. The display time of the polygonal figure is set, and after the central object disappears, the user looks at the location of the polygonal figure and the opposite side of the location of the polygonal figure respectively;
[0072] The cognitive flexibility experimental paradigm includes a cognitive flexibility forward saccade task and a cognitive flexibility reverse saccade task. In the cognitive flexibility forward saccade task, a graphic with a set color is presented on the screen. The graphic with the set color is called the cognitive flexibility forward saccade task prompt. Then, polygonal graphics appear randomly on the left and right sides of the screen for 2 seconds. In the cognitive flexibility reverse saccade task, a graphic with a different color from the cognitive flexibility forward saccade task prompt is presented on the screen. It is called the cognitive flexibility reverse saccade task prompt. Then, a polygonal graphic appears randomly on the left and right sides of the screen for 2 seconds. The eye movement data generated when the user looks at the opposite side of the polygonal graphic is recorded.
[0073] The visual search paradigm was used in the visuospatial function experiment. A picture or graphic was presented on the top of the screen, which was called the visuospatial function target. At the same time, a group of pictures or graphics were presented on the bottom of the screen. The group of pictures or graphics included one picture or graphic that was exactly the same as the visuospatial function target; three pictures or graphics that were similar to the visuospatial function target but had one or more different pictures or graphics, which were called visuospatial function distractors.
[0074] The abstract function experimental paradigm adopts a visual search paradigm. A group of 4 pictures are presented in the center of the screen. Among these pictures, one picture belongs to a different category from the other pictures. For example, among the four pictures of apple, banana, cantaloupe, and tomato, tomato belongs to the vegetable category, while the other three pictures belong to the fruit category. This picture is called the abstract function target, and the other pictures are called abstract function distractors.
[0075] 2) The data acquisition module collects eye movement data through eye tracking technology:
[0076] The data acquisition module uses an eye tracker to collect the eye movement data generated by the user during the execution of the experimental paradigm through eye tracking technology, and synchronously sends the collected eye movement data to the data processing module; the corresponding collected eye movement data include: memory assessment data, executive function and attention data, cognitive flexibility data, visual-spatial function data and abstract function data;
[0077] a) Collect memory assessment data:
[0078] When the user performs the memory saccade task, the data acquisition module acquires the memory saccade data generated when the user looks at the location of the polygonal figure after all the central objects presented on the screen disappear. The memory saccade data includes the memory saccade peak velocity, memory saccade latency, memory saccade accuracy, memory saccade redundant saccade occurrence rate and memory saccade error correction rate;
[0079] When the user performs the picture memory task, the data collection module collects the picture memory data generated by the user in the process of recalling and selecting the picture memory target, including the number of eye movement fixation points, fixation duration, number of correct recalls, average reaction time of correct recalls, and number of incorrect recalls;
[0080] b) Collect executive function and attention data:
[0081] When the user performs the executive function and attention forward scanning task, the data collection module collects the executive function and attention forward scanning data generated in the process of the user looking at the location of the polygonal figure after the central objects presented on the screen disappear. The executive function and attention forward scanning data include the forward executive function and attention peak velocity, the forward executive function and attention latency, the forward executive function and attention accuracy and the forward executive function and attention error correction rate;
[0082] When the user performs the executive function and attention reverse saccade task, the data acquisition module acquires the executive function and attention reverse saccade data generated when the user looks at the opposite side of the position of the polygonal figure after the central objects presented on the screen disappear. The executive function and attention reverse saccade data include reverse executive function and attention peak velocity, reverse executive function and attention latency, reverse executive function and attention accuracy rate and reverse executive function and attention error correction rate;
[0083] c) Collect cognitive flexibility data:
[0084] When the user performs the cognitive flexibility forward scanning task, the data collection module collects the cognitive flexibility forward scanning data generated when the user looks at the location of the polygonal figure. The cognitive flexibility forward scanning data includes the positive cognitive flexibility peak velocity, the positive cognitive flexibility latency, the positive cognitive flexibility accuracy and the positive cognitive flexibility error correction rate.
[0085] When the user performs the cognitive flexibility reverse saccade task, the data collection module collects cognitive flexibility reverse saccade data generated when the user looks at the opposite side of the polygonal figure. The cognitive flexibility reverse saccade data includes reverse cognitive flexibility peak velocity, reverse cognitive flexibility latency, reverse cognitive flexibility accuracy and reverse cognitive flexibility error correction rate.
[0086] d) Collecting visual-spatial function data:
[0087] When the user performs the visual search paradigm of the visual-spatial function experimental paradigm, the data acquisition module collects the visual-spatial function data generated by the user in the process of selecting a picture or graphic that is exactly the same as the visual-spatial function target. The visual-spatial function data includes the total visual-spatial function fixation time, the visual-spatial function target fixation time, the total number of visual-spatial function fixations, the number of fixations on the visual-spatial function target, and the visual-spatial function accuracy;
[0088] e) Collect abstract functional data:
[0089] When the user performs the visual search paradigm of the abstract function experiment paradigm, the data collection module collects the abstract function data generated by the user in the process of selecting the abstract function target object, and the abstract function data includes the total fixation time of the abstract function, the fixation time of the abstract function target object, the total number of fixations on the abstract function, the number of fixations on the abstract function target object and the accuracy rate of the abstract function;
[0090] 3) Data processing module performs data preprocessing:
[0091] The data processing module pre-processes the eye movement data collected by the data processing module to obtain characteristic parameters of the eye movement data, including filling missing values, smoothing noise data, smoothing or deleting outliers, deleting irrelevant data, and integrating and arranging data;
[0092] 4) Evaluate cognitive function scores using regression models:
[0093] The intelligent evaluation module constructs a machine learning SVR regression model, uses the eye movement data and cognitive function score results of a large number of people as samples and labels for training the machine learning SVR regression model, and trains the machine learning SVR regression model. During the training process, GridSearch is used to search for model parameters. For different parameters, cyclic training is performed until the best regression result is obtained. The genetic algorithm is used to perform feature selection on the characteristic parameters of the eye movement data to obtain the optimal characteristic parameter subset and the trained machine learning SVR regression model. Then, the machine learning SVR regression model is used to perform cognitive function evaluation on the characteristic parameter subset of the eye movement data characteristic parameters to obtain the cognitive function score.
[0094] 5) Use classification models to classify cognitive functions:
[0095] The intelligent assessment module uses the scores of the commonly used scales for assessing cognitive functions as data labels, combines the eye movement data feature parameters as training data, uses a decision tree for classification calculations, and establishes and trains a machine learning classification decision tree model; in the process of establishing the machine learning classification decision tree model, all data of the above eye movement data feature parameters are used as features, starting from the root node, the information gain of all features of the node is calculated, and the feature with the largest information gain is selected as the root node feature, and sub-nodes are established according to different values of the root node feature; a new sub-node is generated for each sub-node in the same way until the information gain is very small or no feature can be selected, and finally a machine learning classification decision tree model is obtained; through the machine learning classification decision tree model, the cognitive function scores are classified, and the expected classification of the cognitive function is output, and the expected classification includes "cognitive function is equivalent to the level of the same age group" and "cognitive function is lower than the level of the same age group", or three categories, four categories and more classifications.
[0096] When the data scale increases significantly and there is a lot of noise data, the gradient decision tree (GBDT) method will be used. By adopting an additive model (i.e., a linear combination of basis functions) to integrate multiple machine learning classification decision tree models, the residual error generated by the training process is continuously reduced to achieve the algorithm for classifying data. The system uses the open source XGBoost (eXtreme GradientBoosting) algorithm to implement GBDT, which speeds up learning and improves model generalization.
[0097] Finally, it should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.
Claims
1. A cognitive function assessment system based on eye movement, characterized in that: The eye movement-based cognitive function assessment system includes: an experimental paradigm module, a data acquisition module, a data processing module and an intelligent assessment module; wherein, The experimental paradigm module includes: memory assessment experimental paradigm, executive function and attention experimental paradigm, cognitive flexibility experimental paradigm, visual-spatial function experimental paradigm and abstract function experimental paradigm. The experimental paradigm module provides experimental paradigms for users to perform cognitive function assessment; The data acquisition module uses an eye tracker to collect the eye movement data generated by the user during the execution of the experimental paradigm through eye tracking technology, and synchronously sends the collected eye movement data to the data processing module; the corresponding collected eye movement data include: memory assessment data, executive function and attention data, cognitive flexibility data, visual-spatial function data and abstract function data; The data processing module pre-processes the eye movement data collected by the data processing module to obtain characteristic parameters of the eye movement data; The intelligent evaluation module stores a machine learning support vector SVR regression model and a machine learning classification decision tree model. The intelligent evaluation module uses the machine learning SVR regression model and the machine learning classification decision tree model to evaluate the cognitive function of the eye movement data feature parameters to obtain the cognitive function score, and uses the classification model to divide the cognitive function classification to achieve intelligent output of the cognitive function score and classification discrimination; The memory assessment experimental paradigm includes two experimental tasks: a memory scanning task and a picture memory task. In the memory scanning task, a central object and a polygonal figure are presented on the screen, wherein the polygonal figure appears randomly on both sides of the screen. The display time of the polygonal figure is set. The picture memory task uses a material composed of multiple groups of unrelated pictures, each group contains multiple pictures. In the picture memory task, one picture is randomly selected from each group of pictures and presented in the center of the screen. These selected pictures are called picture memory targets. After the set time, a new group of pictures is presented in the center of the screen. In this new group of pictures, some pictures are the same as the picture memory targets, i.e., pictures that have appeared before, and some pictures are different from the picture memory targets, i.e., pictures that have not appeared before, and pictures that have not appeared before are called picture memory interferers. The executive function and attention experimental paradigm includes an executive function and attention forward saccade task and an executive function and attention reverse saccade task; during the executive function and attention forward saccade task and the executive function and attention reverse saccade task, a central object and a polygonal figure are presented on the screen, and these polygonal figures will randomly appear on both sides of the screen; The cognitive flexibility experimental paradigm includes a cognitive flexibility forward saccade task and a cognitive flexibility reverse saccade task; in the cognitive flexibility forward saccade task, a graphic with a set color is presented on the screen, which is called the cognitive flexibility forward saccade task prompt picture, and then polygonal graphics appear randomly on the left and right sides of the screen; in the cognitive flexibility reverse saccade task, a graphic with a different color from the cognitive flexibility forward saccade task prompt picture is presented on the screen, which is called the cognitive flexibility reverse saccade task prompt picture, and then a polygonal graphic appears randomly on the left and right sides of the screen; The visual-spatial function experimental paradigm adopts a visual search paradigm, wherein a picture or graphic is presented at the top of the screen, and the picture or graphic is called the visual-spatial function target; at the same time, a group of pictures or graphics are presented at the bottom of the screen, and the group of pictures or graphics includes a picture or graphic that is exactly the same as the visual-spatial function target; multiple pictures or graphics that are the same as the visual-spatial function target but have one or more different pictures or graphics are called visual-spatial function distractors; The abstract function experimental paradigm adopts a visual search paradigm. A group of pictures are presented in the center of the screen. In this group of pictures, there is a picture that belongs to a different category from the rest of the pictures. This picture is called the abstract function target, and the rest of the pictures are called abstract function distractors.
2. A cognitive function assessment method based on the eye movement-based cognitive function assessment system as claimed in claim 1, characterized in that: The cognitive function assessment method comprises the following steps: 1) Design experimental paradigm module: The experimental paradigm module includes five modules: memory assessment experimental paradigm, executive function and attention experimental paradigm, cognitive flexibility experimental paradigm, visual spatial function experimental paradigm and abstract function experimental paradigm. The experimental paradigm module provides users with experimental paradigms for cognitive function assessment. 2) The data acquisition module collects eye movement data through eye tracking technology: The data acquisition module uses an eye tracker to collect the eye movement data generated by the user during the execution of the experimental paradigm through eye tracking technology, and synchronously sends the collected eye movement data to the data processing module; the corresponding collected eye movement data include: memory assessment data, executive function and attention data, cognitive flexibility data, visual-spatial function data and abstract function data; a) Collecting memory assessment data: When the user performs the memory saccade task, the data acquisition module acquires the memory saccade data generated when the user looks at the location of the polygonal figure after all the central objects presented on the screen disappear. The memory saccade data includes the memory saccade peak velocity, memory saccade latency, memory saccade accuracy, memory saccade redundant saccade occurrence rate and memory saccade error correction rate; When the user performs the picture memory task, the data collection module collects the picture memory data generated by the user in the process of recalling and selecting the picture memory target, including the number of eye movement fixation points, fixation duration, number of correct recalls, average reaction time of correct recalls, and number of incorrect recalls; b) Collecting executive function and attention data: When the user performs the executive function and attention forward scanning task, the data collection module collects the executive function and attention forward scanning data generated in the process of the user looking at the location of the polygonal figure after the central objects presented on the screen disappear. The executive function and attention forward scanning data include the forward executive function and attention peak velocity, the forward executive function and attention latency, the forward executive function and attention accuracy and the forward executive function and attention error correction rate; When the user performs the executive function and attention reverse saccade task, the data acquisition module acquires the executive function and attention reverse saccade data generated when the user looks at the opposite side of the position of the polygonal figure after the central objects presented on the screen disappear. The executive function and attention reverse saccade data include reverse executive function and attention peak velocity, reverse executive function and attention latency, reverse executive function and attention accuracy rate and reverse executive function and attention error correction rate; c) Collect cognitive flexibility data: When the user performs the cognitive flexibility forward scanning task, the data collection module collects the cognitive flexibility forward scanning data generated when the user looks at the location of the polygonal figure. The cognitive flexibility forward scanning data includes the positive cognitive flexibility peak velocity, the positive cognitive flexibility latency, the positive cognitive flexibility accuracy and the positive cognitive flexibility error correction rate. When the user performs the cognitive flexibility reverse saccade task, the data collection module collects cognitive flexibility reverse saccade data generated when the user looks at the opposite side of the polygonal figure. The cognitive flexibility reverse saccade data includes reverse cognitive flexibility peak velocity, reverse cognitive flexibility latency, reverse cognitive flexibility accuracy and reverse cognitive flexibility error correction rate. d) Collecting visual-spatial function data: When the user performs the visual search paradigm of the visual-spatial function experimental paradigm, the data acquisition module collects the visual-spatial function data generated by the user in the process of selecting a picture or graphic that is exactly the same as the visual-spatial function target. The visual-spatial function data includes the total visual-spatial function fixation time, the visual-spatial function target fixation time, the total number of visual-spatial function fixations, the number of fixations on the visual-spatial function target, and the visual-spatial function accuracy; e) Collect abstract functional data: When the user performs the visual search paradigm of the abstract function experiment paradigm, the data collection module collects the abstract function data generated by the user in the process of selecting the abstract function target object, and the abstract function data includes the total fixation time of the abstract function, the fixation time of the abstract function target object, the total number of fixations on the abstract function, the number of fixations on the abstract function target object and the accuracy rate of the abstract function; 3) Data processing module performs data preprocessing: The data processing module pre-processes the eye movement data collected by the data processing module to obtain characteristic parameters of the eye movement data; 4) Evaluate cognitive function scores using regression models: The intelligent evaluation module constructs a machine learning SVR regression model, uses the eye movement data of a large number of people and the cognitive function score results obtained from the existing common cognitive function scale as samples and labels for training the machine learning SVR regression model, and trains the machine learning SVR regression model. During the training process, grid search is used to search for model parameters. Circular training is performed for different model parameters until the best regression result is obtained. A trained machine learning SVR regression model is obtained, and a genetic algorithm is used to perform feature selection on the feature parameters of the eye movement data to obtain the optimal feature parameter subset. Then, the machine learning SVR regression model is used to perform cognitive function evaluation on the feature parameter subset of the eye movement data feature parameters to obtain a cognitive function score. 5) Use classification models to classify cognitive functions: The intelligent assessment module uses the existing commonly used scale scores of cognitive functions as data labels, combines the eye movement data feature parameters as training data, uses a decision tree for classification calculation, and establishes and trains a machine learning classification decision tree model; in the process of establishing the machine learning classification decision tree model, all data of the above eye movement data feature parameters are used as features, starting from the root node, the information gain of all features of the node is calculated, and the feature with the largest information gain is selected as the root node feature, and sub-nodes are established according to different values of the root node feature; a new sub-node is generated for each sub-node in the same way until the information gain is very small or no feature can be selected, and finally a machine learning classification decision tree model is obtained; the cognitive function scores are classified through the machine learning classification decision tree model, and the expected classification of the cognitive function is output.
3. The cognitive function assessment method according to claim 2, characterized in that: When the data scale increases significantly and there is a lot of noise data, the gradient decision tree GBDT method will be used. By adopting an additive model to integrate multiple machine learning classification decision tree models, the residual error generated by the training process is continuously reduced to achieve the algorithm for classifying data. The system uses the open source extreme gradient boosting algorithm to implement GBDT, which speeds up learning and improves the generalization of the model.
4. The cognitive function assessment method according to claim 2, characterized in that: In step 3), data preprocessing includes filling missing values, smoothing noisy data, smoothing or removing outliers, removing irrelevant data, data integration and tidying.
