Cognitive disorder diagnosis method based on eye movement tracking of network camera

Abstract

The invention discloses a cognitive impairment diagnosis method based on eye movement tracking of a network camera, and relates to the field of cognitive impairment detection.The method comprises the steps that S1, user data are obtained, and a user is numbered to generate a user ID; s2, performing an eye movement test on the user, collecting eye movement photos by adopting a network camera, and processing the eye movement photos to obtain sample data; s3, constructing an eye movement model, and processing the sample data through the eye movement model to obtain a fixation track which comprises a plurality of fixation site coordinates; s4, processing the coordinates of the plurality of fixation sites to obtain sample coordinates, constructing a transformer model, inputting the sample coordinates to evaluate a user, and outputting a cognitive impairment early screening diagnosis report, the method can reduce early senile dementia detection technical obstacles, and is suitable for cognitive impairment routine screening in various scenes such as home, medical and health care institutions.

Description

Technical Field

[0001] The present invention relates to the field of cognitive impairment detection, and in particular to a cognitive impairment diagnosis method based on eye movement tracking of a network camera. Background Art

[0002] Mainstream methods for diagnosing cognitive dysfunction include neuropsychological testing and biomarker analysis. Although these methods are widely used, they are invasive, costly and complex to operate, especially in resource-limited environments. The present invention significantly improves the convenience and popularity of diagnostic technology by using ordinary webcams and algorithms. The eye tracking technology currently used generally includes two main directions in diagnosing cognitive dysfunction: one is to test reaction delays by performing tasks such as reverse saccades, and the other is to analyze eye movement heat maps after viewing specific stimuli. These tasks usually require participants to learn in advance, which is particularly inconvenient in areas with low education levels or language barriers.

[0003] In addition, although heat map analysis is simple to operate, it does not fully exploit the potential of eye movement data because it does not fully explore the time series information in the eye movement data. Both methods also rely on expensive eye tracking equipment, which limits their popularity in economically underdeveloped areas.

[0004] Therefore, there is an urgent need for a cognitive impairment diagnosis method based on webcam eye tracking to solve the above problems. Summary of the invention

[0005] To solve the above problems, the present invention provides a method for diagnosing cognitive impairment based on eye tracking using a webcam, which uses the basic nature of gaze data to screen whether participants suffer from potential dementia.

[0006] The method for diagnosing cognitive impairment based on eye tracking of a webcam provided by the present invention comprises the following steps: S1. Obtain user data, number the user and generate a user ID; S2, conducting an eye movement test on the user and collecting eye movement photos using a webcam, and processing the eye movement photos to obtain sample data; S3, constructing an eye movement model, and processing the sample data through the eye movement model to obtain a gaze trajectory, wherein the gaze trajectory includes a plurality of gaze position coordinates; S4. Process the coordinates of several gaze sites to obtain sample coordinates, build a transformer model, input the sample coordinates to evaluate the user and output a cognitive impairment early screening diagnosis report.

[0007] Preferably, the eye movement test content of the user in S2 specifically includes: a matrix part and an art part; The matrix part includes 5 randomly generated 5-dot matrix, subjects were asked to count the number of dots; The art part includes face images, which include a number of faces, and the subjects are asked to count the number of faces in the images.

[0008] Preferably, the specific content of constructing the eye movement model in S3 is: Collect a number of facial images, perform feature analysis on the facial images to obtain sample facial features, where the sample facial features include key coordinates of the head and the position of the pupil in the eye; Use solvepnp function to calculate the head rotation vector; Establish correspondence between key coordinates of the head, the position of the pupil in the eye, and the head rotation vector; The eye movement model is trained according to the corresponding relationship; The sample data is fed into the eye movement model to extract facial features and obtain the gaze trajectory.

[0009] Preferably, the coordinates of several gaze sites are processed to obtain sample coordinates, a transformer model is constructed, the sample coordinates are input to score the user and the specific content of the diagnosis report of cognitive impairment is outputted as follows: A mini mental status assessment method is provided in the transformer model, and a user is scored by the mini mental status assessment method to obtain a score value; The transformer model divides the sample coordinates into five parts, uses four parts as training data and one part as validation data, and performs five-fold validation.

[0010] Preferably, the transformer model has input dimensions of a gaze point in x and y coordinates and a displayed image; The transformer model has a model dimension of 64, a number of layers of 3, a head nhead of 8, a feedforward dimension of 256, a loss function of BCELoss, and a training cycle of 100.

[0011] Preferably, the specific content of evaluating the user through the mini mental status assessment method is: A target score limit value is preset. If the score value is greater than or equal to the target score limit value, the user is judged to have cognitive impairment; If the score is less than the target score limit, it is judged that the user has no cognitive impairment.

[0012] In summary, the cognitive impairment diagnosis method based on eye tracking of a webcam of the present invention greatly reduces the cost compared with the traditional technology that uses ordinary webcams instead of professional eye tracking equipment. The use of simple equipment and algorithms makes this technology easier to popularize and apply in various scenarios, providing a non-invasive cognitive impairment diagnosis method with operation suggestions. Non-invasive, non-contact, increases the user's willingness to see a doctor, does not require professional equipment, and reduces the user's low cost of seeing a doctor.

[0013] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The present invention is a flow chart of a method for diagnosing cognitive impairment based on eye tracking using a network camera. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is further described below by means of the accompanying drawings and embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​described in these embodiments do not limit the scope of the present invention.

[0016] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0017] Technologies, systems, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, systems, and devices should be considered part of the specification.

[0018] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0019] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0020] like Figure 1 As shown, the present invention provides a method for diagnosing cognitive impairment based on eye tracking of a network camera, which is specifically as follows: S1, obtaining user data, and numbering the user to generate a user ID; S2, conducting an eye movement test on the user and collecting eye movement photos using a webcam, and processing the eye movement photos to obtain sample data; Preferably, the eye movement test content of the user in S2 specifically includes: a matrix part and an art part; The matrix part includes 5 randomly generated 5-dot matrix, subjects were asked to count the number of dots; The art part includes face images, which include a number of faces, and the subjects are asked to count the number of faces in the images.

[0021] S3, constructing an eye movement model, and processing the sample data through the eye movement model to obtain a gaze trajectory, wherein the gaze trajectory includes a plurality of gaze position coordinates; Preferably, the specific content of constructing the eye movement model in S3 is: Collect a number of facial images, perform feature analysis on the facial images to obtain sample facial features, where the sample facial features include key coordinates of the head and the position of the pupil in the eye; Use solvepnp function to calculate the head rotation vector; Establish correspondence between key coordinates of the head, the position of the pupil in the eye, and the head rotation vector; The eye movement model is trained according to the corresponding relationship; The sample data is fed into the eye movement model to extract facial features and obtain the gaze trajectory.

[0022] Preferably, the sample data is brought into the eye movement model, facial features are extracted, and the contents of the gaze trajectory are obtained including: Clean, correct and synchronize the collected eye movement photos; After taking a facial image with a webcam, use dlib's 68-point facial landmark detection model to extract the coordinates of key facial features; Construct the 3D coordinate points of the key areas of the face, and use the EPnP projection in OpenCV's solvePnP algorithm to calculate the rotation vector from the 2D image point to the 3D model; Get the orientation of the head in three-dimensional space based on the rotation vector; The relative position of the pupil in the eye is extracted through the GazeTracking library to obtain the gaze coordinates.

[0023] S4. Process the coordinates of several gaze sites to obtain sample coordinates, build a transformer model, input the sample coordinates to evaluate the user and output a diagnosis report of cognitive impairment.

[0024] Preferably, the coordinates of several gaze sites are processed to obtain sample coordinates, a transformer model is constructed, the sample coordinates are input to score the user and the specific content of the diagnosis report of cognitive impairment is outputted as follows: The transformer model is provided with a mini mental status assessment method, and the user is scored by the mini mental status assessment MMSE method to obtain a score value; The transformer model divides the sample coordinates into five parts, uses four parts as training data and one part as validation data, and performs five-fold validation.

[0025] Preferably, the transformer model has input dimensions of a gaze point in x and y coordinates and a displayed image; The transformer model has a model dimension of 64, a number of layers of 3, a head nhead of 8, a feedforward dimension of 256, a loss function of BCELoss, and a training cycle of 100.

[0026] Preferably, the specific content of evaluating the user through the mini mental status assessment method is: A target score limit value is preset. If the score value is greater than or equal to the target score limit value, the user is judged to have cognitive impairment; If the score is less than the target score limit, it is judged that the user has no cognitive impairment.

[0027] The information table of the subjects obtained from the MMSE questionnaire is as follows: Table 1

[0028] Healthy controls (HC) (=27) and MCI patients (=30) were identified based on the participants' scores on the MMSE assessment. Gender distribution showed that females generally had poorer cognitive function, but the difference was not statistically significant. Participants in the HC group tended to be younger than those in the MCI group, and the difference was statistically significant. The mean MMSE score was 27.3 for HC patients and 19.3 for MCI patients. There was also a difference in educational level between the HC and MCI groups, which was statistically different.

[0029] In this paper, we introduce a model for processing eye tracking data to detect mild cognitive impairment (MCI). The model shows good ability in classifying participants with cognitive impairment from normal controls, achieving a significant area under the curve.

[0030] The performance of the model was evaluated based on precision, accuracy, recall, and F1 score, and ROC-AUC.F1score was used to evaluate the balance between precision and accuracy of the model.

[0031] The expression for the F1 score is: The ROC-AUC of the model was also calculated to measure the performance of the model at different thresholds. The area under the curve was calculated using the receiver operating characteristic curve, and the AUC value obtained on the ROC curve was 0.91, which proved the effectiveness of the model.

[0032] Comparative analysis with previous studies shows that the model in the present invention has achieved significant improvement in performance. The AUC range of previous studies was 0.61-0.85, which is lower than 0.92 of the present invention.

[0033] Even compared with recent studies using advanced deep learning methods, the proposed model still achieves better performance. As shown in Table 2, the AUC of the study using VTnet is 0.75, while the AUC of the proposed model is 0.92. The f1 score of the study using MC-CNN is 0.81, while the f1 score of the proposed model is 0.97.

[0034] Table 2

[0035] The present invention's use of webcam-based eye tracking can also serve as a proof of concept for a more accessible MCI screening tool. While data collected from webcams provide lower fidelity compared to dedicated eye tracking hardware, they significantly reduce the cost and complexity of the setup. The present invention demonstrates that even with lower quality inputs, the present model can achieve a high level of diagnostic accuracy, making it suitable for routine screening of cognitive impairment in a variety of scenarios, such as home, medical, and health care institutions.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for diagnosing cognitive impairment based on eye tracking using a webcam, characterized in that: The following steps are involved: S1. Obtain user data, number the user and generate a user ID; S2, conducting an eye movement test on the user and collecting eye movement photos using a webcam, and processing the eye movement photos to obtain sample data; S3, constructing an eye movement model, and processing the sample data through the eye movement model to obtain a gaze trajectory, wherein the gaze trajectory includes a plurality of gaze position coordinates; S4. Process the coordinates of several gaze sites to obtain sample coordinates, build a transformer model, input the sample coordinates to evaluate the user and output a cognitive impairment early screening diagnosis report.

2. The method for diagnosing cognitive impairment based on eye tracking using a network camera according to claim 1, characterized in that: The eye movement test content of users in S2 specifically includes the matrix part and the art part; The matrix part includes 5 randomly generated 5-dot matrix, subjects were asked to count the number of dots; The art part includes face images, which include a number of faces, and the subjects are asked to count the number of faces in the images.

3. The method for diagnosing cognitive impairment based on eye tracking using a network camera according to claim 2, characterized in that: The specific contents of constructing the eye movement model in S3 are: Collect a number of facial images, perform feature analysis on the facial images to obtain sample facial features, where the sample facial features include key coordinates of the head and the position of the pupil in the eye; Use solvepnp function to calculate the head rotation vector; Establish correspondence between key coordinates of the head, the position of the pupil in the eye, and the head rotation vector; The eye movement model is trained according to the corresponding relationship; The sample data is fed into the eye movement model to extract facial features and obtain the gaze trajectory.

4. The method for diagnosing cognitive impairment based on eye tracking using a network camera according to claim 3, characterized in that: The coordinates of several gaze sites are processed to obtain sample coordinates, and a transformer model is constructed. The sample coordinates are input to score the user and output the specific content of the diagnosis report of cognitive impairment: A mini mental status assessment method is provided in the transformer model, and a user is scored by the mini mental status assessment method to obtain a score value; The transformer model divides the sample coordinates into five parts, uses four parts as training data and one part as validation data, and performs five-fold validation.

5. The method for diagnosing cognitive impairment based on eye tracking using a network camera according to claim 4, characterized in that: The transformer model takes as input the fixation point in x and y coordinates and the displayed image; The transformer model has a model dimension of 64, a number of layers of 3, a head nhead of 8, a feedforward dimension of 256, a loss function of BCELoss, and a training cycle of 100.

6. The method for diagnosing cognitive impairment based on eye tracking using a network camera according to claim 5, characterized in that: The specific content of the evaluation of users through the Mini Mental Status Assessment Method is as follows: A target score limit value is preset. If the score value is greater than or equal to the target score limit value, the user is judged to have cognitive impairment; If the score is less than the target score limit, it is judged that the user has no cognitive impairment.

Images