Application of portable three-dimensional eye tracker in cognitive function evaluation

Abstract

The invention relates to the technical field of early detection of neurological diseases, and discloses an application of a portable three-dimensional eye tracker in cognitive function assessment, comprising: a portable three-dimensional eye tracker used for collecting eye movement data, including fixation time, glancing trajectory, review frequency and eye jump speed; the electroencephalogram acquisition equipment is used for acquiring electroencephalogram data, and the electroencephalogram data comprises alpha wave energy E alpha, beta wave energy E beta and theta wave energy E theta; the heart rate variability monitoring equipment is used for collecting heart rate data, and the heart rate data comprises the ratio RLF / HF of low frequency (LF) to high frequency (HF). The accuracy of cognitive function evaluation is greatly improved through multi-modal data fusion, real-time monitoring and personalized feedback are provided, early discovery of cognitive impairment is facilitated, and the device is high in portability, supports long-term monitoring and testing anytime and anywhere, and is suitable for early detection and continuous evaluation of Alzheimer's diseases.

Description

Technical Field

[0001] The present invention relates to the technical field of early detection of neurological diseases, and in particular to the application of a portable three-dimensional eye tracker in cognitive function assessment. Background Art

[0002] With the aggravation of the aging problem in modern society, the incidence of neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease has increased significantly, and the demand for early diagnosis and cognitive function assessment of these diseases has become increasingly urgent. Cognitive function assessment methods are widely used in the fields of clinical medicine, neuroscience, and psychology, using behavioral tests, electroencephalogram (EEG), and functional magnetic resonance imaging (fMRI) techniques. However, traditional single-modality assessment methods are difficult to fully reflect the complex cognitive state of patients and have certain limitations.

[0003] In the existing technologies, eye trackers, EEG, heart rate variability (HRV) monitoring and galvanic skin response (GSR) physiological signal analysis methods are used to evaluate cognitive function. These technologies can each reflect part of cognitive activities. For example, eye trackers can evaluate concentration, EEG can monitor changes in brain electrical activity, and HRV can measure the response of the autonomic nervous system to stress and emotions.

[0004] However, the existing technology still has certain shortcomings. First, the evaluation of single-modality data is limited to specific physiological dimensions and is easily affected by noise or insufficient information. It is difficult to fully and accurately reflect the patient's cognitive function status, especially the detection accuracy of early neurodegenerative diseases is low; in addition, most of the existing evaluation methods lack real-time feedback and long-term monitoring capabilities, and cannot provide individuals with dynamic and continuous cognitive function evaluation. This limitation increases the risk of missed diagnosis and misdiagnosis, and cannot meet the needs of early detection and personalized intervention of cognitive dysfunction. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides an application of a portable three-dimensional eye tracker in cognitive function assessment, which solves the problem that single-modality assessment methods in the prior art are difficult to comprehensively and accurately detect cognitive dysfunction and lack real-time feedback and long-term monitoring capabilities.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: The application of a portable three-dimensional eye tracker in cognitive function assessment includes:

[0007] A portable 3D eye tracker for collecting eye movement data, including fixation time, saccade trajectory, number of return glances, and saccade velocity;

[0008] EEG data acquisition equipment, used to collect EEG data, the EEG data includes alpha wave energy E α , β wave energy E βand theta wave energy E θ ;

[0009] A heart rate variability monitoring device is used to collect heart rate data, wherein the heart rate data includes a ratio R of a low frequency LF to a high frequency HF. LF / HF ;

[0010] The skin galvanic response sensor is used to collect skin conductance change data, wherein the skin conductance change includes the skin conductivity amplitude A GSR ;

[0011] The data fusion module is used to synchronize and fuse the above multimodal data, and perform multimodal feature extraction and fusion based on time synchronization. The multimodal data fusion is processed by weighted averaging, and the formula is as follows:

[0012]

[0013] Among them, α is the fusion weight coefficient, ranging from 0.5 to 0.8.

[0014] Preferably, the eye movement data includes the following technical features:

[0015] Fixation duration T fixation , which is used to assess how long an individual can focus on a specific visual target;

[0016] Eye speed V saccade , used to assess changes in the speed of individual eye movements;

[0017] Number of re-looks N regressions , used to assess individuals’ information retrieval behavior.

[0018] Preferably, the EEG acquisition device further comprises an independent component analysis (ICA) module for removing artifacts in the EEG signal and calculating the power spectral density of the EEG wave band by the following formula:

[0019]

[0020] Among them, PSD is the power spectral density, N is the total number of samples of the signal, X(f n ) is the frequency f n signal value.

[0021] Preferably, the heart rate variability (HRV) monitoring device calculates the HRV low frequency LF and high frequency HF components by short time Fourier transform (STFT), and calculates their ratio by the following formula:

[0022]

[0023] Among them, P LF is the low frequency component power, PHF is the high frequency component power.

[0024] Preferably, the multimodal feature fusion module includes a combination of a convolutional neural network (CNN) and a long short-term memory network (LSTM), which is used to jointly process eye movement data and EEG data, wherein:

[0025] The CNN network is used to process the spatiotemporal feature extraction of eye movement data, and the convolution kernel size is 3×3;

[0026] The LSTM network is used to process EEG time series data and capture the temporal dependency of EEG signals.

[0027] Preferably, the system further comprises a data preprocessing module, which is used to perform the following processing on the multimodal data:

[0028] Perform median filtering on the eye movement data to remove sudden noise in the data;

[0029] The EEG data were band-pass filtered in the frequency range of 0.5 Hz to 30 Hz to remove low-frequency and power frequency interference;

[0030] The HRV data were processed to remove outliers, where outliers were defined as RR intervals greater than 2000 ms or less than 300 ms.

[0031] Preferably, the multimodal data fusion is performed by weighted averaging, wherein:

[0032] P eye EEG =w1T fixation +w2V saccade +w3E α +w4E β

[0033] in, is the weighted prediction result of eye movement and EEG, w1, w2, w3, w4 are the weights of eye movement and EEG features, and the weight range is 0.1 to 0.9.

[0034] Preferably, the system also includes a real-time feedback module for generating a cognitive function assessment report based on the patient's real-time multimodal data changes and issuing an early warning when an abnormality is detected.

[0035] Preferably, the real-time feedback module is further used to analyze the changing trend of the patient's cognitive function over time, and generate regular reports for tracking the changes in the patient's cognitive function.

[0036] Preferably, the system is used for early detection of neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease, by analyzing characteristic changes in eye movements, EEG, HRV and GSR in multimodal data and combining machine learning models to make early predictions of cognitive dysfunction.

[0037] The present invention provides an application of a portable three-dimensional eye tracker in cognitive function assessment. It has the following beneficial effects:

[0038] 1. The present invention significantly improves the accuracy of cognitive function assessment by integrating multimodal physiological signals such as eye movement data, EEG, HRV, and GSR. Multimodal data can capture the comprehensive physiological responses of individuals in different tasks, making up for the shortcomings of a single modality under insufficient information or noise. Experimental results show that the accuracy, sensitivity, and specificity of the multimodal fusion method in the detection of cognitive dysfunction are significantly higher than those of the traditional method that relies solely on a single modality, and is particularly suitable for the early detection of neurodegenerative diseases such as Alzheimer's disease.

[0039] 2. The present invention provides the ability to monitor individual cognitive functions in real time. The system can dynamically generate cognitive assessment reports based on comprehensive data of eye movements, EEG, heart rate and skin electrical response. When abnormal cognitive function is detected, the system can issue a warning signal in real time and provide immediate feedback to users or clinicians. This real-time and personalized feedback function helps to detect cognitive function degeneration problems at an earlier stage, support personalized intervention and treatment, and improve the patient's rehabilitation effect.

[0040] 3. The portable device design of the present invention enables it to be easily used in daily environments without relying on complex equipment in traditional laboratories. Its non-invasiveness and portability allow subjects to be tested in natural scenes, and the data is more real and reliable. At the same time, the system supports long-term monitoring and can track the changing trends of individual cognitive functions through regular data analysis. It is especially suitable for patients with Alzheimer's disease who need long-term tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a system architecture diagram of the present invention;

[0042] Figure 2 It is a structural schematic diagram of the data acquisition module of the present invention;

[0043] Figure 3 It is a structural schematic diagram of a data preprocessing module of the present invention;

[0044] Figure 4 It is a structural schematic diagram of the data fusion and analysis module of the present invention;

[0045] Figure 5 It is a structural diagram of the real-time feedback and monitoring module of the present invention;

[0046] Figure 6 It is a schematic diagram of the structure of the long-term tracking and report generation module of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Please see attached Figure 1 - Attachment Figure 6 The present invention relates to a multimodal data integration technology based on a portable three-dimensional eye tracker, which is applied to the evaluation of cognitive function and early detection of neurodegenerative diseases. The method integrates physiological data such as eye movement data, electroencephalogram (EEG), heart rate variability (HRV), and galvanic skin response (GSR) to construct a multimodal fusion model, which can evaluate the cognitive state of an individual in real time and perform early cognitive impairment detection. The system uses multimodal feature extraction and fusion algorithms to accurately capture the cognitive response of an individual during task execution and generate a detailed cognitive function evaluation report.

[0049] 1. Equipment selection and integration

[0050] The portable three-dimensional eye tracker, electroencephalogram device, heart rate variability monitoring device, and skin galvanic response sensor used in the present invention have the following characteristics:

[0051] Eye tracker: used to capture the eye movement characteristics of individuals during tasks, with a collection frequency of 60-120Hz. Eye movement data includes fixation time, scanning trajectory, number of return glances, eye saccade speed, etc.

[0052] Electroencephalogram (EEG) equipment: used to collect EEG signals, focusing on cognitive-related frequency bands (α, β, and θ waves), and using bandpass filtering (0.5-30 Hz) to remove low-frequency drift and high-frequency interference.

[0053] Heart rate variability (HRV) devices: monitor individual heart rate changes, focusing on low-frequency (LF) and high-frequency (HF) components, and use the LF / HF ratio to assess the activity of the autonomic nervous system.

[0054] Galvanic Skin Response (GSR) Sensor: Captures an individual's emotions and stress responses by detecting changes in skin conductivity, reflecting sympathetic nerve activity.

[0055] Through the integration of the above-mentioned devices, the present invention can simultaneously collect multimodal data and perform multi-faceted cognitive function assessment in a clinical environment.

[0056] 2. Data collection and synchronization processing

[0057] In practical applications, data collection from all devices needs to be time synchronized to ensure that multimodal data can reflect the individual cognitive state at the same time. The collection process mainly includes the following steps:

[0058] Synchronous acquisition: Use unified timestamp technology to synchronize multimodal data from eye movement, EEG, HRV, and GSR devices.

[0059] Sampling rate unification: Since the sampling rates of various devices are different, all modality data need to be aligned to the lowest sampling rate. For example, if the sampling rate of eye movement data is 120Hz, EEG is 256Hz, HRV is 1000Hz, and GSR is 50Hz, all data are synchronized to 120Hz through linear interpolation.

[0060] 3. Data preprocessing and feature extraction

[0061] 3.1 Eye Movement Data Preprocessing

[0062] The eye movement data is filtered through a median filter to remove burst noise and further extract the following key features:

[0063] Fixation duration T fixation : Used to assess how long an individual can focus on a specific visual target and reflect the ability to process visual information.

[0064] Eye speed V saccade : Used to evaluate the changes in eye movement speed and reflect the efficiency of visual search during task execution.

[0065] Number of re-looks N regressions : Assessing an individual's revisiting of previous information, commonly seen in reading or visual tasks.

[0066] 3.2 EEG data preprocessing

[0067] The EEG signal was bandpass filtered (0.5 Hz to 30 Hz) to remove low-frequency drift and high-frequency noise, and then independent component analysis (ICA) was used to eliminate artifacts (such as electrooculographic artifacts). The following power spectrum features were extracted from the EEG data:

[0068] Alpha wave energy E α : Reflects the state of attention and relaxation;

[0069] Beta wave energy E β : reflects the degree of active thinking and task engagement;

[0070] Theta wave energy E θ : Reflects deep thinking and memory processing.

[0071] The power spectral density (PSD) calculation formula is:

[0072]

[0073] Where N is the total number of samples of the signal, X(f n ) is the frequency f n The signal amplitude

[0074] 3.3HRV data preprocessing

[0075] HRV data is obtained by removing abnormal heart rate values ​​(such as RR intervals greater than 2000ms or less than 300ms), and then calculating the power of low-frequency (LF) and high-frequency (HF) components. The LF and HF components are obtained by short-time Fourier transform (STFT), and their ratio is calculated:

[0076]

[0077] Among them, P LF is the power of the low-frequency component, P HF The LF / HF ratio is used to assess the activity of the autonomic nervous system and reflects an individual's stress response.

[0078] 3.4 GSR data preprocessing

[0079] The skin electrical response data is low-pass filtered to remove high-frequency noise and extract the amplitude of skin conductivity change A. GSR and peak frequency f peaks These characteristics reflect the individual's emotional fluctuations and the activity of the sympathetic nervous system during the task.

[0080] 4. Multimodal feature fusion

[0081] 4.1 Unimodal Feature Extraction

[0082] Each modality data is passed through an independent feature extraction module to convert the eye movement, EEG, HRV, and GSR features into a single feature vector, representing the state of the individual in the cognitive task.

[0083] 4.2 Multimodal Fusion Method

[0084] The present invention uses a mid-term fusion strategy, that is, first extracting features from each modality independently, and then weighted fusion of the extracted features. The specific implementation is as follows:

[0085] Convolutional neural network (CNN) is used to process the spatiotemporal features in eye movement data;

[0086] Long short-term memory network (LSTM)** processes EEG time series data and captures the temporal dependency of EEG signals;

[0087] The random forest model was used to combine HRV and GSR features to assess emotional state and stress level.

[0088] During the fusion process, feature vectors are fused using the weighted average formula:

[0089] P eye EEG =w1T fixation +w2V saccade +w3E α +w4E β

[0090] Among them, α is the fusion weight coefficient, which ranges from 0.5 to 0.8, and the best value is determined by optimizing the validation set.

[0091] 5. Model training and optimization

[0092] 5.1 Training Process

[0093] The present invention uses the Adam optimizer to train the convolutional neural network (CNN), long short-term memory network (LSTM) and random forest models, with the initial learning rate set to 0.001 and the batch size to 64. During the training process, the accuracy and loss changes of the validation set are monitored to avoid overfitting of the model.

[0094] 5.2 Loss Function

[0095] The cross entropy loss function is used to optimize the classification task. The specific formula is:

[0096]

[0097] Among them, y i is the true label, p i is the predicted probability.

[0098] 6. System application and real-time feedback

[0099] The present invention provides a real-time feedback system that can continuously monitor changes in multimodal data during the execution of individual tasks and generate real-time cognitive function assessment results. When an abnormality is detected, the system will send an early warning signal to the user or clinician. The system can also generate regular cognitive function assessment reports to help users understand the changing trends of their cognitive status, which is suitable for early detection of neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease.

[0100] Example 1: Application of portable three-dimensional eye tracker in early detection of Alzheimer's disease Implementation steps:

[0101] Equipment preparation:

[0102] Portable 3D eye tracker: Tobii Pro Glasses 3, with a sampling frequency of 120 Hz, was used to capture the eye movements of the subjects during visual tasks.

[0103] EEG equipment: Emotiv Epoc X, 16-channel EEG, sampling frequency 256 Hz, monitoring the subjects’ brain electrical activity.

[0104] Heart rate variability (HRV) monitoring equipment: Polar H10, sampling frequency 1000 Hz, to monitor the heart rate changes of the subjects.

[0105] Galvanic Skin Response (GSR) sensor: Shimmer3 GSR+, sampling frequency 50Hz, monitors changes in skin conductivity.

[0106] Experimental design:

[0107] The subjects performed a 30-minute visual search task, which required them to find the target object in a complex background and remember the characteristics of the target object.

[0108] Task process: Change the difficulty of the visual task every 5 minutes, and monitor the changes in multimodal data.

[0109] The experimental subjects were healthy adults (control group) and early-stage Alzheimer's disease patients (experimental group), with 10 subjects recruited from each group.

[0110] Data collection and synchronization:

[0111] Time synchronization: Use unified timestamp technology to synchronize eye movement, EEG, HRV, and GSR data, and unify the sampling frequency to 120Hz.

[0112] Data collection cycle: During the collection period, each task lasts 30 minutes, and the data is stored and backed up in real time.

[0113] Data preprocessing:

[0114] Eye movement data: Median filtering was used to remove noise and extract fixation duration, saccade trajectory, number of return glances, and saccade velocity.

[0115] EEG data: ICA was used to remove electrooculographic artifacts and extract the power spectral density (PSD) of α, β, and θ waves.

[0116] HRV data: Abnormal RR intervals were removed and the LF / HF ratio was calculated.

[0117] GSR data: Filter to remove high-frequency noise and extract the amplitude and peak frequency of skin conductance changes.

[0118] Data fusion and model building:

[0119] CNN is used to extract spatiotemporal features from eye movement data, LSTM is used to process EEG time series data, and the random forest model is used to process HRV and GSR features.

[0120] Multimodal fusion uses the following weighted average formula:

[0121] P final =0.7P eyeEEG +0.3P HRVGSR

[0122] The model is trained using the Adam optimizer with a batch size of 64 and an initial learning rate of 0.001.

[0123] Real-time evaluation and feedback:

[0124] The system analyzes the changes in multimodal data in real time during the task execution and generates a cognitive function assessment report. If abnormal cognitive function (such as early symptoms of Alzheimer's disease) is detected, the system will issue an early warning.

[0125] Process parameters:

[0126] Eye tracker sampling frequency: 120Hz

[0127] EEG device sampling frequency: 256Hz

[0128] HRV device sampling frequency: 1000Hz

[0129] GSR device sampling frequency: 50Hz

[0130] Mission duration: 30 minutes

[0131] Task change frequency: 5 minutes / time

[0132] Comparative Example 1: Evaluation Method Using Eye Tracker Data Only

[0133] Implementation steps:

[0134] Equipment preparation: Only a portable 3D eye tracker (Tobii Pro Glasses 3) was used with a sampling frequency of 120 Hz.

[0135] Experimental design: Same as Example 1, subjects performed a 30-minute visual search task.

[0136] Data collection and processing: Only eye movement data were collected to extract fixation duration, scanning trajectory, number of return glances and saccade velocity.

[0137] Data processing and analysis: Multimodal data fusion was not used, and cognitive assessment was performed solely on eye movement features.

[0138] Process parameters:

[0139] Eye tracker sampling frequency: 120Hz

[0140] Mission duration: 30 minutes

[0141] Task change frequency: 5 minutes / time

[0142] Comparative Example 2: Evaluation Method Using EEG Data Only

[0143] Implementation steps:

[0144] Equipment preparation: Only EEG equipment (Emotiv Epoc X) was used with a sampling frequency of 256 Hz.

[0145] Experimental design: Same as Example 1, subjects performed a 30-minute visual search task.

[0146] Data acquisition and processing: Only EEG data were collected, and the power spectral density of α, β and θ waves was extracted.

[0147] Data processing and analysis: Cognitive assessment was performed solely on EEG features without combining eye movement, HRV, or GSR data.

[0148] Process parameters:

[0149] EEG device sampling frequency: 256Hz

[0150] Mission duration: 30 minutes

[0151] Task change frequency: 5 minutes / time

[0152] Comparative Example 3: Evaluation method using only HRV and GSR data

[0153] Implementation steps:

[0154] Equipment preparation: Only HRV equipment (Polar H10, sampling frequency 1000Hz) and GSR sensor (Shimmer3GSR+, sampling frequency 50Hz) were used.

[0155] Experimental design: Same as Example 1, subjects performed a 30-minute visual search task.

[0156] Data collection and processing: Only HRV and GSR data were collected, and the LF / HF ratio and skin conductivity change amplitude were extracted.

[0157] Data processing and analysis: Cognitive assessment was performed only on HRV and GSR features without combining eye movement or EEG data.

[0158] Process parameters:

[0159] HRV device sampling frequency: 1000Hz

[0160] GSR device sampling frequency: 50Hz

[0161] Mission duration: 30 minutes

[0162] Task change frequency: 5 minutes / time

[0163] Comparative experimental design:

[0164] Purpose:

[0165] By comparing the evaluation method of multimodal fusion (Example 1) with the evaluation method of a single modality (Comparative Examples 1-3), the superiority of the portable three-dimensional eye tracker and the multimodal data fusion method in cognitive function evaluation is verified.

[0166] Experimental group and control group:

[0167] Experimental group: 10 patients with early-stage Alzheimer's disease.

[0168] Control group: 10 healthy adults.

[0169] Experimental process:

[0170] All subjects participated in a 30-min visual search task, and eye movement, EEG, HRV, and GSR data were recorded.

[0171] Example 1 uses a multimodal fusion model for evaluation, while Examples 1-3 only rely on a single modality for evaluation.

[0172] After the task, the accuracy, sensitivity and specificity of each method in cognitive function assessment were analyzed.

[0173] Experimental data table:

[0174]

[0175]

[0176] It can be seen from the experimental data that the multimodal fusion method (Example 1) is significantly superior to the single-modal method (Comparative Examples 1-3) in terms of accuracy, sensitivity and specificity. Specific advantages include:

[0177] Multimodal fusion methods can capture the correlation between different modalities, thereby improving the detection accuracy of cognitive dysfunction;

[0178] The combination of multiple physiological signals such as eye movement, EEG, HRV, GSR, etc. can more comprehensively reflect the cognitive state of an individual, especially showing higher sensitivity in detecting early symptoms of Alzheimer's disease;

[0179] Compared with the single modality method, the multimodal fusion method in Example 1 shows higher specificity in the classification of healthy adults and early-stage Alzheimer's disease patients, and can more accurately distinguish healthy individuals from patients.

[0180] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. The application of portable three-dimensional eye tracker in cognitive function assessment is characterized by: include: A portable 3D eye tracker for collecting eye movement data, including fixation time, saccade trajectory, number of return glances, and saccade velocity; EEG data acquisition equipment, used to collect EEG data, the EEG data includes alpha wave energy E α , β wave energy E β and theta wave energy E θ ; A heart rate variability monitoring device is used to collect heart rate data, wherein the heart rate data includes a ratio R of a low frequency LF to a high frequency HF. LF / HF ; The skin galvanic response sensor is used to collect skin conductance change data, wherein the skin conductance change includes the skin conductivity amplitude A GSR ; The data fusion module is used to synchronize and fuse the above multimodal data, and perform multimodal feature extraction and fusion based on time synchronization. The multimodal data fusion is processed by weighted averaging, and the formula is as follows: Among them, α is the fusion weight coefficient, ranging from 0.5 to 0.

8.

2. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 1 is characterized in that: The eye movement data includes the following technical features: Fixation duration T fixation , which is used to assess how long an individual can focus on a specific visual target; Eye speed V saccade , used to assess changes in the speed of individual eye movements; Number of re-looks N regressions , used to assess individuals’ information retrieval behavior.

3. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 1, characterized in that: The EEG acquisition device further includes an independent component analysis (ICA) module for removing artifacts in the EEG signal and calculating the power spectral density of the EEG wave band using the following formula: Among them, PSD is the power spectral density, N is the total number of samples of the signal, X(f n ) is the frequency f n signal value.

4. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 1 is characterized in that: The heart rate variability (HRV) monitoring device calculates the HRV low frequency LF and high frequency HF components by short time Fourier transform (STFT), and calculates their ratio by the following formula: Among them, P LF is the low frequency component power, P HF is the high frequency component power.

5. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 1, characterized in that: The multimodal feature fusion module includes a combination of a convolutional neural network (CNN) and a long short-term memory network (LSTM), which is used to jointly process eye movement data and EEG data, wherein: The CNN network is used to process the spatiotemporal feature extraction of eye movement data, and the convolution kernel size is 3×3; The LSTM network is used to process EEG time series data and capture the temporal dependency of EEG signals.

6. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 1, characterized in that: The system also includes a data preprocessing module, which is used to perform the following processing on the multimodal data: Perform median filtering on the eye movement data to remove sudden noise in the data; The EEG data were band-pass filtered in the frequency range of 0.5 Hz to 30 Hz to remove low-frequency and power frequency interference; The HRV data were processed to remove outliers, where outliers were defined as RR intervals greater than 2000 ms or less than 300 ms.

7. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 1, characterized in that: The multimodal data fusion is performed by weighted averaging, where: P eyeEEG =w1T fixation +w2V saccade +w3E α +w4E β in, is the weighted prediction result of eye movement and EEG, w1, w2, w3, w4 are the weights of eye movement and EEG features, and the weight range is 0.1 to 0.

9.

8. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 1, characterized in that: The system also includes a real-time feedback module for generating a cognitive function assessment report based on the patient's real-time multimodal data changes and issuing an early warning when an abnormality is detected.

9. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 8, characterized in that: The real-time feedback module is further used to analyze the changing trend of the patient's cognitive function over time and generate regular reports for tracking the changes in the patient's cognitive function.

10. The application of the portable three-dimensional eye tracker in cognitive function assessment according to claim 1, characterized in that: The system is used for early detection of neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease. It analyzes the characteristic changes of eye movements, EEG, HRV and GSR in multimodal data and combines machine learning models to make early predictions of cognitive dysfunction.

Images