Cognitive disorder screening method and device based on VR eye tracker and brain-computer interface

Abstract

The invention discloses a cognitive impairment screening device and method based on a VR eye tracker and a brain-computer interface. The device comprises a virtual reality scene generation module, an eye tracker data acquisition module, a brain-computer interface data acquisition module, a data analysis processing module and a cognitive impairment screening module. The virtual reality scene generation module is used for generating a virtual reality scene containing a cognitive task; the data analyzing and processing module is used for analyzing and processing the collected eye movement data information set and electroencephalogram signal information set, and extracting to obtain a characteristic parameter sequence; the cognitive impairment screening module is used for performing cognitive impairment screening processing on the characteristic parameter sequence to obtain cognitive impairment information of the user; the cognitive impairment information comprises information about whether cognitive impairment exists or not and the degree of the cognitive impairment.

Description

Technical Field

[0001] The present invention relates to the fields of virtual reality and brain cognition, and in particular to a method and device for screening cognitive impairment based on a VR eye tracker and a brain-computer interface. Background Art

[0002] With the acceleration of the trend of an aging society, more and more people are experiencing cognitive impairment. Cognitive impairment is a common neurological abnormality, including dementia, mild cognitive impairment, etc. Early and accurate screening is crucial for the diagnosis and treatment of cognitive impairment. At present, commonly used screening methods for cognitive impairment include neuropsychological tests, imaging examinations, etc., but these methods have certain limitations, such as strong subjectivity, complex operation, and insensitivity to early manifestations of impairment.

[0003] How to achieve rapid, accurate and simple early screening of cognitive impairment is an urgent problem that needs to be solved in the current context of an aging society. Summary of the invention

[0004] The present invention mainly solves the problem of how to achieve rapid, accurate and simple early screening of cognitive impairment. The present invention discloses a method and device for screening cognitive impairment based on a VR eye tracker and a brain-computer interface. The purpose of the present invention is to provide a method and device for screening cognitive impairment based on a virtual reality eye tracker and a brain-computer interface to solve the limitations of existing cognitive impairment screening methods and improve the accuracy, objectivity and efficiency of screening.

[0005] In a first aspect of the embodiment of the present application, a cognitive impairment screening device based on a VR eye tracker and a brain-computer interface is disclosed, comprising: a virtual reality scene generation module, an eye tracker data acquisition module, a brain-computer interface data acquisition module, a data analysis and processing module, and a cognitive impairment screening module;

[0006] The virtual reality scene generation module is used to generate a virtual reality scene including cognitive tasks;

[0007] The eye tracker data acquisition module is used to acquire in real time an eye movement data information set when the user completes a cognitive task in a virtual reality scene; the eye movement data information set includes a fixation point information sequence, a fixation time information sequence, and an eye saccade frequency information sequence; the fixation point information sequence includes fixation point position information; the fixation time information sequence includes fixation time information; the eye saccade frequency information sequence includes eye saccade frequency information;

[0008] The brain-computer interface data acquisition module is used to acquire a set of EEG signal information when the user completes a cognitive task in a virtual reality scene; the set of EEG signal information includes an EEG signal sequence;

[0009] The data analysis and processing module is connected to the virtual reality scene generation module, the eye tracker data acquisition module, the brain-computer interface data acquisition module and the cognitive impairment screening module respectively, and is used to analyze and process the collected eye movement data information set and the electroencephalogram signal information set to extract a characteristic parameter sequence; the characteristic parameter sequence includes an eye movement pattern abnormality parameter sequence and an electroencephalogram signal spectrum characteristic sequence;

[0010] The cognitive impairment screening module is used to perform cognitive impairment screening processing on the characteristic parameter sequence to obtain the user's cognitive impairment information; the cognitive impairment information includes whether there is cognitive impairment and the degree of cognitive impairment.

[0011] The data analysis and processing module is used to analyze and process the collected eye movement data information set and the EEG signal information set to extract a characteristic parameter sequence, including:

[0012] The data analysis and processing module performs preprocessing operations on the collected eye movement data information set and the EEG signal information set respectively to obtain a preprocessed eye movement data information set and an EEG signal information set; performs abnormal feature analysis processing on the preprocessed eye movement data information set to obtain an eye movement pattern abnormal parameter sequence; performs frequency domain abnormality recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence;

[0013] The eye movement pattern abnormal parameter sequence and the EEG signal spectrum feature sequence are fused to obtain a feature parameter sequence.

[0014] The cognitive impairment screening module is used to perform cognitive impairment screening on the characteristic parameter sequence to obtain cognitive impairment information of the user, including:

[0015] The cognitive impairment screening module processes the characteristic parameter sequence using the trained cognitive impairment screening model to obtain a cognitive impairment probability value;

[0016] Determine whether the cognitive impairment probability value is greater than a preset impairment probability threshold to obtain screening result information; if the screening result information is yes, confirm that the user has cognitive impairment, confirm that the cognitive impairment degree information is the cognitive impairment probability value, and obtain cognitive impairment information; if the screening result information is no, confirm that the cognitive impairment information is that the user does not have cognitive impairment.

[0017] In a second aspect of the embodiment of the present application, a method for screening cognitive impairment based on a VR eye tracker and a brain-computer interface is disclosed, which is implemented using the cognitive impairment screening device based on a VR eye tracker and a brain-computer interface, including:

[0018] S1, using the virtual reality scene generation module to generate a virtual reality scene including a cognitive task; and displaying the virtual reality scene including the cognitive task to a user;

[0019] S2, using the eye tracker data acquisition module to acquire in real time a set of eye movement data information when the user completes a cognitive task in a virtual reality scene; using the brain-computer interface data acquisition module to acquire a set of electroencephalogram signal information when the user completes a cognitive task in a virtual reality scene;

[0020] S3, using the data analysis and processing module to analyze and process the collected eye movement data information set and the EEG signal information set, and extract a characteristic parameter sequence;

[0021] S4, using the cognitive impairment screening module to perform cognitive impairment screening processing on the characteristic parameter sequence to obtain cognitive impairment information of the user.

[0022] The step of analyzing and processing the collected eye movement data information set and the EEG signal information set to extract a characteristic parameter sequence includes:

[0023] S31, performing preprocessing operations on the collected eye movement data information set and the electroencephalogram signal information set to obtain a preprocessed eye movement data information set and an electroencephalogram signal information set;

[0024] S32, performing abnormal feature analysis on the preprocessed eye movement data information set to obtain an eye movement pattern abnormal parameter sequence;

[0025] S33, performing frequency domain anomaly recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence;

[0026] S34, fusing the eye movement pattern abnormal parameter sequence and the EEG signal spectrum feature sequence to obtain a feature parameter sequence.

[0027] The pre-processing operation comprises:

[0028] S311, normalizing each information set to obtain a corresponding normalized information set;

[0029] S312, performing data cleaning processing on each normalized information set to obtain a corresponding cleaned data set;

[0030] S313, performing pattern recognition and discrimination processing on each cleaned data set to obtain a corresponding pre-processed information set.

[0031] The pattern recognition and discrimination processing includes:

[0032] S3131, for each type of data attribute of the cleaned data set, the data collection information of the data is used as the independent variable, and the data value of the data is used as the dependent variable, and autoregression-sliding average modeling is performed to obtain the regression model of the data attribute of the class; using the regression model, the independent variable is calculated and processed to obtain the regression data value; it is determined whether the absolute value of the difference between the regression data value and the corresponding dependent variable value is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, the data is deleted from the cleaned data set; if it is less than or equal to the first regression discrimination threshold, the data is not processed;

[0033] S3132, performing fusion processing on all data after executing S3131 of the cleaned data set to obtain a pre-processed information set.

[0034] The performing abnormal feature analysis on the pre-processed eye movement data information set to obtain an abnormal eye movement pattern parameter sequence includes:

[0035] Performing deviation evaluation processing on the fixation point information sequence in the preprocessed eye movement data information set to obtain a deviation value of each fixation point information sequence;

[0036] Determine a gaze point information sequence whose deviation value is greater than a preset deviation threshold as a gaze abnormal parameter sequence;

[0037] The calculation expression of the deviation evaluation process is:

[0038]

[0039] Among them, p1 is the calculation parameter, a i is the deviation value of the i-th fixation point information sequence, (x0, y0, z0) is the position coordinate of the preset target fixation point in the cognitive task, (x ij ,y ij ,z ij ) is the jth fixation point position coordinate of the ith fixation point information sequence, and M is the number of fixation point position coordinates contained in a fixation point information sequence;

[0040] Performing joint abnormality assessment processing on the fixation time information sequence and the eye movement frequency information sequence in the preprocessed eye movement data information set to obtain a time-frequency deviation value of each fixation time information;

[0041] Determine the fixation time information and eye saccade frequency information whose time-frequency deviation value is greater than a preset time-frequency deviation threshold as time-frequency abnormality parameters;

[0042] All the abnormal gaze parameter sequences and time-frequency abnormal parameters are combined to obtain the abnormal eye movement pattern parameter sequence;

[0043] The calculation expression of the joint anomaly assessment process is:

[0044] c i =arctan(|t0-t i | / |t0+t i |+|f0-f i | / |f0+f i |) / pi+(|t0-t i |+|f0-f i |) / (t0+f0),

[0045] Among them, c i is the time-frequency abnormality parameter of the ith fixation time information in the fixation time information sequence, t0 and f0 are the standard fixation time value and the standard eye saccade frequency value respectively, t i and f i are the i-th fixation time information in the fixation time information sequence and the i-th saccade frequency information in the saccade frequency information sequence, and pi is the pi constant.

[0046] The performing frequency domain anomaly recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence includes:

[0047] Performing high-order feature quantity calculation processing on each EEG signal sequence of the preprocessed EEG signal information set to obtain an abnormality discrimination value;

[0048] Determine that the spectrum sequence of the EEG signal sequence whose abnormal discrimination value is greater than the set abnormal threshold value is an abnormal EEG sequence;

[0049] Using all abnormal EEG sequences, construct the EEG signal spectrum feature sequence;

[0050] The expression for the high-order feature quantity calculation process is:

[0051]

[0052] α=ω1R4+ω2R 23 +ω3R′ 56 ,

[0053] Where α is the abnormal discrimination value of the EEG signal sequence, FFT() represents the discrete Fourier transform operation, x(n) represents the nth element of the EEG signal sequence, and y(n) represents the nth element of the frequency domain signal of the EEG signal sequence. represents the conjugate element of y(n), ω1, ω2, ω3 represent the preset first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, N represents the length of the EEG signal sequence, R4 represents the fourth-order self-accumulative amount of the EEG signal sequence, R23 represents the second and third order cross-cumulants of the EEG signal sequence, and R5′6 represents the fifth and sixth order cross-conjugate cross-cumulants of the EEG signal sequence.

[0054] The cognitive impairment screening model includes: an input module, a feature extraction module, a feature fusion module, a feature processing module and a prediction module;

[0055] The input module is used to receive and obtain a characteristic parameter sequence;

[0056] The feature extraction module is connected to the input module and the feature fusion module respectively, and is used to extract features from the feature parameter sequence to obtain a feature signal set;

[0057] The feature fusion module is used to perform fusion processing on the feature signal set to obtain a fused feature signal set;

[0058] The feature processing module is connected to the feature fusion module and the prediction module respectively, and is used to perform time series correlation processing on the fused feature signal set to obtain a time series cross feature signal set;

[0059] The prediction module is used to perform prediction processing on the time series cross feature signal set to obtain a cognitive impairment probability value;

[0060] The feature extraction module includes four feature extractors corresponding to the signal modes; each feature extractor is used to perform feature extraction processing on the feature parameter sequence of the corresponding mode to obtain the corresponding feature signal;

[0061] The feature fusion module includes four multi-head attention sub-modules; each multi-head attention sub-module is used to process the feature signal of the corresponding modality; the dimension of each multi-head attention sub-module matches the dimension of the feature signal of the modality it processes; the input end of each multi-head attention sub-module is connected to the output end of the corresponding feature extractor;

[0062] The feature processing module includes four long short-term memory network submodules; each long short-term memory network submodule is used to process the feature parameter sequence of the corresponding modality; each long short-term memory network submodule is connected to the output end of the corresponding multi-head attention submodule;

[0063] The prediction module includes a fully connected network and an output end; the four input ends of the fully connected network are respectively connected to the output ends of the four long short-term memory network sub-modules; the output end of the prediction module is used to output the probability value of cognitive impairment.

[0064] The beneficial effects of the present invention are:

[0065] The present invention introduces virtual reality technology, eye tracker, and brain-computer interface technology into the field of cognitive impairment screening. Virtual reality technology can create a realistic virtual environment, simulate various daily life scenes, and improve the fun and participation of the test; the eye tracker can monitor the user's eye movement trajectory in real time and obtain information such as visual attention; brain-computer interface technology can directly detect the brain's neural activity and reflect the state of cognitive function. Through the fusion of multimodal physiological characteristics and abnormal feature detection, the present invention realizes early screening of users' cognitive impairment.

[0066] The present invention uses virtual reality technology to improve user participation and test authenticity, and can better simulate cognitive scenarios in daily life; combined with eye trackers and brain-computer interface technology, it obtains cognition-related information in multiple dimensions, thereby improving the accuracy and objectivity of screening; the present invention uses automated data analysis and establishes a screening model, thereby reducing manual intervention and improving screening efficiency.

[0067] The cognitive impairment screening model established by the present invention utilizes a feature extraction module to implement feature extraction processing of input multimodal abnormal signals, realizes the fusion of cognitive impairment-related quantities in multiple categories of features through a feature fusion module, and utilizes a feature processing module and a prediction module to perform abnormal analysis processing and prediction on the fused quantities, ultimately obtaining a probability value of the user's cognitive impairment; the present invention realizes rapid and accurate screening of user cognitive impairment through the cognitive impairment screening model. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a schematic diagram of the composition of the device of the present invention;

[0069] Figure 2 It is a flow chart for implementing the method of the present invention. DETAILED DESCRIPTION

[0070] In order to better understand the content of the present invention, an embodiment is given here.

[0071] Figure 1 It is a schematic diagram of the composition of the device of the present invention; Figure 2 It is a flow chart for implementing the method of the present invention.

[0072] In a first aspect of the embodiment of the present application, a cognitive impairment screening device based on a VR eye tracker and a brain-computer interface is disclosed, comprising: a virtual reality scene generation module, an eye tracker data acquisition module, a brain-computer interface data acquisition module, a data analysis and processing module, and a cognitive impairment screening module;

[0073] The virtual reality scene generation module is used to generate a virtual reality scene containing cognitive tasks; the cognitive tasks include object recognition, spatial navigation, memory test, etc.;

[0074] The eye tracker data acquisition module is used to acquire the user's eye movement data information set in real time in the virtual reality scene; the eye movement data information set includes a fixation point information sequence, a fixation time information sequence, an eye saccade frequency information sequence, etc. The fixation point information sequence includes fixation point position information; the fixation time information sequence includes fixation time information; the eye saccade frequency information sequence includes eye saccade frequency information;

[0075] The brain-computer interface data acquisition module is used to acquire a collection of EEG signal information when the user completes a cognitive task. The collection of EEG signal information includes an EEG signal sequence;

[0076] The data analysis and processing module is respectively connected to the virtual reality scene generation module, the eye tracker data acquisition module, the brain-computer interface data acquisition module and the cognitive impairment screening module, and is used to analyze and process the collected eye movement data information set and the electroencephalogram signal information set, and extract a characteristic parameter sequence, wherein the characteristic parameter sequence includes an eye movement pattern abnormality parameter sequence and an electroencephalogram signal spectrum characteristic sequence.

[0077] The cognitive impairment screening module is used to perform cognitive impairment screening on the characteristic parameter sequence to obtain cognitive impairment information of the user; the cognitive impairment information includes whether cognitive impairment exists and the degree of cognitive impairment;

[0078] The data analysis and processing module analyzes and processes the collected eye movement data information set and the EEG signal information set to extract a characteristic parameter sequence, including:

[0079] The data analysis and processing module performs preprocessing operations on the collected eye movement data information set and the electroencephalogram signal information set to obtain a preprocessed eye movement data information set and an electroencephalogram signal information set; performs abnormal feature analysis processing on the preprocessed eye movement data information set to obtain an eye movement pattern abnormal parameter sequence;

[0080] Performing frequency domain anomaly recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence;

[0081] The eye movement pattern abnormal parameter sequence and the EEG signal spectrum feature sequence are fused to obtain a feature parameter sequence.

[0082] The cognitive impairment screening module is used to perform cognitive impairment screening on the characteristic parameter sequence to obtain cognitive impairment information of the user, including:

[0083] The cognitive impairment screening module processes the characteristic parameter sequence using the trained cognitive impairment screening model to obtain a cognitive impairment probability value;

[0084] Determine whether the cognitive impairment probability value is greater than a preset impairment probability threshold to obtain screening result information; if the screening result information is yes, confirm that the user has cognitive impairment, confirm that the cognitive impairment degree information is the cognitive impairment probability value, and obtain cognitive impairment information; if the screening result information is no, confirm that the cognitive impairment information is that the user does not have cognitive impairment.

[0085] The pre-processing operation comprises:

[0086] Normalizing each information set to obtain a corresponding normalized information set;

[0087] Perform data cleaning on each normalized information set to obtain a corresponding cleaned data set;

[0088] Perform pattern recognition and discrimination processing on each cleaned data set to obtain a corresponding pre-processed information set;

[0089] The pattern recognition and discrimination processing includes:

[0090] S201, for each type of data attribute of the cleaned data set, the data collection information of the data is used as an independent variable, and the data value of the data is used as a dependent variable, and autoregression-sliding average modeling is performed to obtain regression models of the data attributes of the class; using the regression model, the independent variable is calculated and processed to obtain regression data values; it is determined whether the absolute value of the difference between the regression data value and the corresponding dependent variable value is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, the data is deleted from the cleaned data set; if it is less than or equal to the first regression discrimination threshold, the data is not processed;

[0091] S202, performing fusion processing on all the data after executing S201 on the cleaned data set to obtain a pre-processed information set;

[0092] The performing abnormal feature analysis on the pre-processed eye movement data information set to obtain an abnormal eye movement pattern parameter sequence includes:

[0093] Performing deviation evaluation processing on the fixation point information sequence in the preprocessed eye movement data information set to obtain a deviation value of each fixation point information sequence;

[0094] Determine a gaze point information sequence whose deviation value is greater than a preset deviation threshold as a gaze abnormal parameter sequence;

[0095] The calculation expression of the deviation evaluation process is:

[0096]

[0097] Among them, p1 is the calculation parameter, a i is the deviation value of the i-th fixation point information sequence, (x0, y0, z0) is the position coordinate of the preset target fixation point in the cognitive task, (x ij ,y ij ,z ij ) is the jth fixation point position coordinate of the ith fixation point information sequence, and M is the number of fixation point position coordinates contained in a fixation point information sequence;

[0098] Performing joint abnormality assessment processing on the fixation time information sequence and the eye movement frequency information sequence in the preprocessed eye movement data information set to obtain a time-frequency deviation value of each fixation time information;

[0099] Determine the fixation time information and eye saccade frequency information whose time-frequency deviation value is greater than a preset time-frequency deviation threshold as time-frequency abnormality parameters;

[0100] All the abnormal gaze parameter sequences and time-frequency abnormal parameters are combined to obtain the abnormal eye movement pattern parameter sequence;

[0101] The calculation expression of the joint anomaly assessment process is:

[0102] c i =arctan(|t0-t i | / |t0+t i |+|f0-f i | / |f0+f i |) / pi+(|t0-t i |+|f0-f i |) / (t0+f0),

[0103] Among them, c i is the time-frequency abnormality parameter of the ith fixation time information in the fixation time information sequence, t0 and f0 are the standard fixation time value and the standard eye saccade frequency value respectively, t i and f i are the i-th fixation time information in the fixation time information sequence and the i-th saccade frequency information in the saccade frequency information sequence, respectively, and pi is the pi constant;

[0104] The performing frequency domain anomaly recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence includes:

[0105] Performing high-order feature quantity calculation processing on each EEG signal sequence of the preprocessed EEG signal information set to obtain an abnormality discrimination value;

[0106] Determine that the spectrum sequence of the EEG signal sequence whose abnormal discrimination value is greater than the set abnormal threshold value is an abnormal EEG sequence;

[0107] Using all abnormal EEG sequences, construct the EEG signal spectrum feature sequence;

[0108] The expression for the high-order feature quantity calculation process is:

[0109]

[0110] α=ω1R4+ω2R 23 +ω3R′ 56 ,

[0111] Where α is the abnormal discrimination value of the EEG signal sequence, FFT() represents the discrete Fourier transform operation, x(n) represents the nth element of the EEG signal sequence, and y(n) represents the nth element of the frequency domain signal of the EEG signal sequence. represents the conjugate element of y(n), ω1, ω2, ω3 represent the preset first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, N represents the length of the EEG signal sequence, R4 represents the fourth-order self-accumulative amount of the EEG signal sequence, R 23 Represents the second and third order cross-cumulants of the EEG signal sequence, R′ 56 Represents the fifth and sixth order mutual conjugate cumulants of the EEG signal sequence.

[0112] The cognitive impairment screening model includes: an input module, a feature extraction module, a feature fusion module, a feature processing module and a prediction module;

[0113] The input module is used to receive and obtain a characteristic parameter sequence;

[0114] The feature extraction module is connected to the input module and the feature fusion module respectively, and is used to extract features from the feature parameter sequence to obtain a feature signal set;

[0115] The feature fusion module is used to perform fusion processing on the feature signal set to obtain a fused feature signal set;

[0116] The feature processing module is connected to the feature fusion module and the prediction module respectively, and is used to perform time series correlation processing on the fused feature signal set to obtain a time series cross feature signal set;

[0117] The prediction module is used to perform prediction processing on the time series cross feature signal set to obtain a cognitive impairment probability value.

[0118] The feature extraction module includes four feature extractors corresponding to the signal modes; each feature extractor is used to perform feature extraction processing on the feature parameter sequence of the corresponding mode to obtain the corresponding feature signal;

[0119] The signals of the four signal modes of the characteristic parameter sequence include a gaze abnormality parameter sequence, gaze time information of a time-frequency abnormality parameter, eye saccade frequency information of a time-frequency abnormality parameter, and an abnormal EEG sequence;

[0120] The feature extraction module includes 4 feature extraction submodules for extracting feature parameter sequences of corresponding modalities; for the gaze abnormality parameter sequence, the corresponding feature extraction submodule is implemented by using a Transformer network; for the gaze time information of the time-frequency abnormality parameter, the corresponding feature extraction submodule is implemented by using a pre-trained VGG16net model; for the eye saccade frequency information of the time-frequency abnormality parameter, the corresponding feature extraction submodule is implemented by using a CNN network; for the abnormal EEG sequence, the corresponding feature extraction submodule is implemented by using an RNN network;

[0121] The feature fusion module includes four multi-head attention sub-modules; each multi-head attention sub-module is used to process the feature signal of the corresponding modality; the dimension of each multi-head attention sub-module matches the dimension of the feature signal of the modality it processes; the input end of each multi-head attention sub-module is connected to the output end of the corresponding feature extractor;

[0122] The feature processing module includes four long short-term memory network submodules; each long short-term memory network submodule is used to process the feature parameter sequence of the corresponding modality; each long short-term memory network submodule is connected to the output end of the corresponding multi-head attention submodule;

[0123] The prediction module includes a fully connected network and an output end; the four input ends of the fully connected network are respectively connected to the output ends of the four long short-term memory network sub-modules; the output end of the prediction module is used to output the probability value of cognitive impairment.

[0124] The fully connected network is used to implement the Softmax function.

[0125] The multi-head attention submodule is used to perform modal cross-feature calculation processing on the corresponding feature signal to obtain the modal cross-feature; the output of a multi-head attention submodule is 4 modal cross-feature vectors; each multi-head attention submodule includes n attention heads;

[0126] The modal cross-feature calculation process is calculated as follows:

[0127]

[0128]

[0129] Among them, F l-uis the modal cross feature vector of the feature signal of the current multi-head attention submodule and the nth feature signal, l is the serial number of the current multi-head attention submodule, l = 1, 2, ..., Nu = 1, 2, ..., N, h i is the output of the i-th attention head of the current multi-head attention submodule, are the key, value, and weighted value of the query vector of the i-th attention head of the current multi-head attention submodule, i=1,2,..·,n, q, k, and v are the key, value, and query vector input by the current multi-head attention submodule, respectively; the values ​​of v and k are the l-th feature signal, and the value of q is the u-th feature signal; f is the first calculation function, and its expression is:

[0130]

[0131]

[0132] Among them, d is a preset calculation constant, and s(q2, q1) is a calculation intermediate quantity.

[0133] The training process of the cognitive impairment screening model includes:

[0134] A training data set is constructed using the historical feature parameter sequence and the corresponding cognitive impairment probability value; the training data in the training data set is the historical feature parameter sequence, and the cognitive impairment probability value corresponding to the historical feature parameter sequence is the label information corresponding to the training data;

[0135] Initialize the number of training iterations;

[0136] Inputting the training data in the training data set as input data into the cognitive impairment screening model;

[0137] Using the cognitive impairment screening model, the input data is processed to obtain a predicted value;

[0138] Performing difference calculation processing on the obtained predicted value and the label information corresponding to the input data to obtain a difference value;

[0139] Determine whether the difference value satisfies a convergence condition, and obtain a first determination result;

[0140] When the first judgment result is no, judging whether the training iteration number value is equal to the training number threshold, and obtaining a second judgment result;

[0141] When the second judgment result is no, determining that the model training state does not meet the training termination condition;

[0142] When the second judgment result is yes, determining that the model training state satisfies the training termination condition;

[0143] When the first judgment result is yes, determining that the model training state satisfies a training termination condition;

[0144] When the model training state does not meet the training termination condition, the parameters of the cognitive impairment screening model are updated using the parameter update model, the training iteration number value is increased by 1, and the execution is triggered to input the training data in the training data set as input data into the cognitive impairment screening model;

[0145] When the model training state satisfies the training termination condition, the training process of the cognitive impairment screening model is completed to obtain a trained cognitive impairment screening model.

[0146] The parameter updating model is:

[0147]

[0148] θ←θ+v;

[0149] In the formula, is the difference value calculated for the i-th training data in the training data set, v is the parameter update value, θ is the parameter of the module to be updated, η is the initial parameter learning rate, α is the momentum angle parameter, 0≤α≤π / 4, It means to find partial derivatives with respect to variable θ;

[0150] The difference value satisfies the convergence condition, which means that the difference value is less than a preset convergence threshold; the difference value does not satisfy the convergence condition, which means that the difference value is not less than the preset convergence threshold.

[0151] The difference calculation process may be implemented using a loss function.

[0152] The loss function may adopt a cross entropy loss function.

[0153] When using the cognitive impairment screening device based on VR eye tracker and brain-computer interface to perform cognitive impairment screening, the steps include:

[0154] (1) Preparation

[0155] Select appropriate virtual reality equipment, eye trackers, and brain-computer interface devices, and perform calibration and debugging. Invite a group of healthy volunteers and users with known cognitive impairment to participate in the test.

[0156] (2) Virtual Reality Scene Generation

[0157] Design virtual reality scenarios that contain different cognitive tasks, such as a virtual supermarket where subjects need to find specific products.

[0158] (3) Data collection

[0159] The test subject wears the device to enter the virtual reality scene, and the eye tracker and brain-computer interface device begin to collect data.

[0160] (4) Data analysis

[0161] The collected data is preprocessed to remove noise and outliers, and eye movement features (such as gaze point distribution and eye saccade speed) and EEG features (such as power spectral density in specific frequency bands) are extracted.

[0162] (5) Establishing a screening model

[0163] A cognitive impairment screening model was established based on the extracted features.

[0164] (6) Model Verification

[0165] The model was validated using the reserved test set to evaluate the accuracy, sensitivity, and specificity of the model; users were screened using the cognitive impairment screening model.

[0166] The virtual reality scene generation module can be implemented by using virtual reality glasses; the eye tracker data acquisition module can be implemented by using an eye tracker or a VR eye tracker developed by Xi'an Blue Brain Technology Co., Ltd.; the brain-computer interface data acquisition module can be implemented by using a BCI acquisition device.

[0167] In a second aspect of the embodiment of the present application, a method for screening cognitive impairment based on a VR eye tracker and a brain-computer interface is disclosed, which is implemented using the cognitive impairment screening device based on a VR eye tracker and a brain-computer interface, including:

[0168] S1, using the virtual reality scene generation module to generate a virtual reality scene including a cognitive task; and displaying the virtual reality scene including the cognitive task to a user;

[0169] S2, using the eye tracker data acquisition module to acquire a set of eye movement data information of the user in the virtual reality scene in real time; using the brain-computer interface data acquisition module to acquire a set of EEG signal information of the user when completing a cognitive task;

[0170] S3, using the data analysis and processing module to analyze and process the collected eye movement data information set and the EEG signal information set, and extract a characteristic parameter sequence;

[0171] S4, using the cognitive impairment screening module to perform cognitive impairment screening processing on the characteristic parameter sequence to obtain cognitive impairment information of the user.

[0172] The pre-processing operation comprises:

[0173] Normalizing each information set to obtain a corresponding normalized information set;

[0174] Perform data cleaning on each normalized information set to obtain a corresponding cleaned data set;

[0175] Perform pattern recognition and discrimination processing on each cleaned data set to obtain a corresponding pre-processed information set;

[0176] The pattern recognition and discrimination processing includes:

[0177] S201, for each type of data attribute of the cleaned data set, the data collection information of the data is used as an independent variable, and the data value of the data is used as a dependent variable, and autoregression-sliding average modeling is performed to obtain regression models of the data attributes of the class; using the regression model, the independent variable is calculated and processed to obtain regression data values; it is determined whether the absolute value of the difference between the regression data value and the corresponding dependent variable value is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, the data is deleted from the cleaned data set; if it is less than or equal to the first regression discrimination threshold, the data is not processed;

[0178] S202, performing fusion processing on all the data after executing S201 on the cleaned data set to obtain a pre-processed information set;

[0179] The data collection information is the data collection time information;

[0180] The performing abnormal feature analysis on the pre-processed eye movement data information set to obtain an abnormal eye movement pattern parameter sequence includes:

[0181] Performing deviation evaluation processing on the fixation point information sequence in the preprocessed eye movement data information set to obtain a deviation value of each fixation point information sequence;

[0182] Determine a gaze point information sequence whose deviation value is greater than a preset deviation threshold as a gaze abnormal parameter sequence;

[0183] The calculation expression of the deviation evaluation process is:

[0184]

[0185] Among them, p1 is the calculation parameter, a iis the deviation value of the i-th fixation point information sequence, (x0, y0, z0) is the position coordinate of the preset target fixation point in the cognitive task, (x ij ,y ij ,z ij ) is the jth fixation point position coordinate of the ith fixation point information sequence, and M is the number of fixation point position coordinates contained in a fixation point information sequence;

[0186] Performing joint abnormality assessment processing on the fixation time information sequence and the eye movement frequency information sequence in the preprocessed eye movement data information set to obtain a time-frequency deviation value of each fixation time information;

[0187] Determine the fixation time information and eye saccade frequency information whose time-frequency deviation value is greater than a preset time-frequency deviation threshold as time-frequency abnormality parameters;

[0188] All the abnormal gaze parameter sequences and time-frequency abnormal parameters are combined to obtain the abnormal eye movement pattern parameter sequence;

[0189] The calculation expression of the joint anomaly assessment process is:

[0190] c i =arctan(|t0-t i | / |t0+t i |+|f0-f i | / |f0+f i |) / pi+(|t0-t i |+|f0-f i |) / (t0+f0),

[0191] Among them, c i is the time-frequency abnormality parameter of the ith fixation time information in the fixation time information sequence, t0 and f0 are the standard fixation time value and the standard eye saccade frequency value respectively, t i and f i are the i-th fixation time information in the fixation time information sequence and the i-th saccade frequency information in the saccade frequency information sequence, respectively, and pi is the pi constant;

[0192] The performing frequency domain anomaly recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence includes:

[0193] Performing high-order feature quantity calculation processing on each EEG signal sequence of the preprocessed EEG signal information set to obtain an abnormality discrimination value;

[0194] Determine that the spectrum sequence of the EEG signal sequence whose abnormal discrimination value is greater than the set abnormal threshold value is an abnormal EEG sequence;

[0195] Using all abnormal EEG sequences, construct the EEG signal spectrum feature sequence;

[0196] The expression for the high-order feature quantity calculation process is:

[0197]

[0198] α=ω1R4+ω2R 23 +ω3R′ 56 ,

[0199] Where α is the abnormal discrimination value of the EEG signal sequence, FFT() represents the discrete Fourier transform operation, x(n) represents the nth element of the EEG signal sequence, and y(n) represents the nth element of the frequency domain signal of the EEG signal sequence. represents the conjugate element of y(n), ω1, ω2, ω3 represent the preset first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, N represents the length of the EEG signal sequence, R4 represents the fourth-order self-accumulative amount of the EEG signal sequence, R 23 Represents the second and third order cross-cumulants of the EEG signal sequence, R′ 56 Represents the fifth and sixth order mutual conjugate cumulants of the EEG signal sequence.

[0200] The cognitive impairment screening module performs cognitive impairment screening processing on the characteristic parameter sequence to obtain cognitive impairment information of the user, including:

[0201] The cognitive impairment screening module processes the characteristic parameter sequence using the trained cognitive impairment screening model to obtain a cognitive impairment probability value;

[0202] Determine whether the cognitive impairment probability value is greater than a preset impairment probability threshold value, and obtain screening result information; if the screening result information is yes, confirm that the user has cognitive impairment, and confirm that the cognitive impairment degree information is the cognitive impairment probability value; if the screening result information is no, confirm that the user does not have cognitive impairment;

[0203] The cognitive impairment screening model includes: an input module, a feature extraction module, a feature fusion module, a feature processing module and a prediction module;

[0204] The input module is used to receive and obtain a characteristic parameter sequence;

[0205] The feature extraction module is connected to the input module and the feature fusion module respectively, and is used to extract features from the feature parameter sequence to obtain a feature signal set;

[0206] The feature fusion module is used to perform fusion processing on the feature signal set to obtain a fused feature signal set;

[0207] The feature processing module is connected to the feature fusion module and the prediction module respectively, and is used to perform time series correlation processing on the fused feature signal set to obtain a time series cross feature signal set;

[0208] The prediction module is used to perform prediction processing on the time series cross feature signal set to obtain a cognitive impairment probability value.

[0209] The feature extraction module includes four feature extractors corresponding to the signal modes; each feature extractor is used to perform feature extraction processing on the feature parameter sequence of the corresponding mode to obtain the corresponding feature signal;

[0210] The signals of the four signal modes of the characteristic parameter sequence include a gaze abnormality parameter sequence, gaze time information of a time-frequency abnormality parameter, eye saccade frequency information of a time-frequency abnormality parameter, and an abnormal EEG sequence;

[0211] The feature extraction module includes 4 feature extraction submodules for extracting feature parameter sequences of corresponding modalities; for the gaze abnormality parameter sequence, the corresponding feature extraction submodule is implemented by using a Transformer network; for the gaze time information of the time-frequency abnormality parameter, the corresponding feature extraction submodule is implemented by using a pre-trained VGG16net model; for the eye saccade frequency information of the time-frequency abnormality parameter, the corresponding feature extraction submodule is implemented by using a CNN network; for the abnormal EEG sequence, the corresponding feature extraction submodule is implemented by using an RNN network;

[0212] The feature fusion module includes four multi-head attention sub-modules; each multi-head attention sub-module is used to process the feature signal of the corresponding modality; the dimension of each multi-head attention sub-module matches the dimension of the feature signal of the modality it processes; the input end of each multi-head attention sub-module is connected to the output end of the corresponding feature extractor;

[0213] The feature processing module includes four long short-term memory network submodules; each long short-term memory network submodule is used to process the feature parameter sequence of the corresponding modality; each long short-term memory network submodule is connected to the output end of the corresponding multi-head attention submodule;

[0214] The prediction module includes a fully connected network and an output end; the four input ends of the fully connected network are respectively connected to the output ends of the four long short-term memory network sub-modules; the output end of the prediction module is used to output the probability value of cognitive impairment.

[0215] The fully connected network is used to implement the Softmax function.

[0216] The multi-head attention submodule is used to perform modal cross-feature calculation processing on the corresponding feature signal to obtain the modal cross-feature; the output of a multi-head attention submodule is 4 modal cross-feature vectors; each multi-head attention submodule includes n attention heads;

[0217] The modal cross-feature calculation process is calculated as follows:

[0218]

[0219]

[0220] Among them, F l-u is the modal cross feature vector of the feature signal of the current multi-head attention submodule and the nth feature signal, l is the serial number of the current multi-head attention submodule, l = 1, 2, ..., N, u = 1, 2, ..., N, h i is the output of the i-th attention head of the current multi-head attention submodule, are the key, value, and weighted value of the query vector of the i-th attention head of the current multi-head attention submodule, i = 1, 2, ..., n, q, k, and v are the key, value, and query vector input by the current multi-head attention submodule, respectively; the values ​​of v and k are the l-th feature signal, and the value of q is the u-th feature signal; f is the first calculation function, and its expression is:

[0221]

[0222]

[0223] Among them, d is a preset calculation constant, and s(q2, q1) is a calculation intermediate quantity.

[0224] The training process of the cognitive impairment screening model includes:

[0225] A training data set is constructed using the historical feature parameter sequence and the corresponding cognitive impairment probability value; the training data in the training data set is the historical feature parameter sequence, and the cognitive impairment probability value corresponding to the historical feature parameter sequence is the label information corresponding to the training data;

[0226] Initialize the number of training iterations;

[0227] Inputting the training data in the training data set as input data into the cognitive impairment screening model;

[0228] Using the cognitive impairment screening model, the input data is processed to obtain a predicted value;

[0229] Performing difference calculation processing on the obtained predicted value and the label information corresponding to the input data to obtain a difference value;

[0230] Determine whether the difference value satisfies a convergence condition, and obtain a first determination result;

[0231] When the first judgment result is no, judging whether the training iteration number value is equal to the training number threshold, and obtaining a second judgment result;

[0232] When the second judgment result is no, determining that the model training state does not meet the training termination condition;

[0233] When the second judgment result is yes, determining that the model training state satisfies the training termination condition;

[0234] When the first judgment result is yes, determining that the model training state satisfies a training termination condition;

[0235] When the model training state does not meet the training termination condition, the parameters of the cognitive impairment screening model are updated using the parameter update model, the training iteration number value is increased by 1, and the execution is triggered to input the training data in the training data set as input data into the cognitive impairment screening model;

[0236] When the model training state satisfies the training termination condition, the training process of the cognitive impairment screening model is completed to obtain a trained cognitive impairment screening model.

[0237] The parameter updating model is:

[0238]

[0239] θ←θ+v;

[0240] In the formula, is the difference value calculated for the i-th training data in the training data set, v is the parameter update value, θ is the parameter of the module to be updated, η is the initial parameter learning rate, α is the momentum angle parameter, 0≤α≤π / 4, It means to find partial derivatives with respect to variable θ;

[0241] The difference value satisfies the convergence condition, which means that the difference value is less than a preset convergence threshold; the difference value does not satisfy the convergence condition, which means that the difference value is not less than the preset convergence threshold.

[0242] The difference calculation process may be implemented using a loss function.

[0243] The loss function may adopt a cross entropy loss function.

[0244] The normalization process is to map data in different value ranges to a specified value range, and its mathematical expression is:

[0245]

[0246] Among them, t is a data value in the data set, t max is the maximum value of all data in the data set, t min is the minimum value of all data in the data set, t 归一化 is the value of data t after normalization;

[0247] The data cleaning process includes filling missing values, smoothing noise data, and smoothing or deleting outliers; the smoothed noise data is obtained by first determining the noise data, and then smoothing the noise data according to the data before and after the noise data; the noise data is a value whose value is less than the detection sensitivity of the sensor of the observed data, or greater than the measurement upper limit of the sensor of the observed data. The outlier point can be determined by the Kalman filter method. The filling value of the missing value can be determined by averaging the measured values ​​within a certain sampling interval before and after the missing value.

[0248] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A cognitive impairment screening device based on a VR eye tracker and a brain-computer interface, characterized in that: include: Virtual reality scene generation module, eye tracker data acquisition module, brain-computer interface data acquisition module, data analysis and processing module, cognitive impairment screening module; The virtual reality scene generation module is used to generate a virtual reality scene including cognitive tasks; The eye tracker data acquisition module is used to acquire in real time an eye movement data information set when the user completes a cognitive task in a virtual reality scene; the eye movement data information set includes a fixation point information sequence, a fixation time information sequence, and an eye saccade frequency information sequence; the fixation point information sequence includes fixation point position information; the fixation time information sequence includes fixation time information; the eye saccade frequency information sequence includes eye saccade frequency information; The brain-computer interface data acquisition module is used to acquire a set of EEG signal information when the user completes a cognitive task in a virtual reality scene; the set of EEG signal information includes an EEG signal sequence; The data analysis and processing module is connected to the virtual reality scene generation module, the eye tracker data acquisition module, the brain-computer interface data acquisition module and the cognitive impairment screening module, respectively, and is used to analyze and process the collected eye movement data information set and the electroencephalogram signal information set to extract a characteristic parameter sequence; The characteristic parameter sequence includes an eye movement pattern abnormality parameter sequence and an EEG signal spectrum characteristic sequence; The cognitive impairment screening module is used to perform cognitive impairment screening processing on the characteristic parameter sequence to obtain cognitive impairment information of the user; The cognitive impairment information includes whether there is cognitive impairment and the degree of cognitive impairment.

2. The cognitive impairment screening device based on VR eye tracker and brain-computer interface as claimed in claim 1, characterized in that: The data analysis and processing module is used to analyze and process the collected eye movement data information set and the EEG signal information set to extract a characteristic parameter sequence, including: The data analysis and processing module performs preprocessing operations on the collected eye movement data information set and the EEG signal information set respectively to obtain a preprocessed eye movement data information set and an EEG signal information set; performs abnormal feature analysis processing on the preprocessed eye movement data information set to obtain an eye movement pattern abnormal parameter sequence; performs frequency domain abnormality recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence; The eye movement pattern abnormal parameter sequence and the EEG signal spectrum feature sequence are fused to obtain a feature parameter sequence.

3. The cognitive impairment screening device based on VR eye tracker and brain-computer interface as claimed in claim 1, characterized in that: The cognitive impairment screening module is used to perform cognitive impairment screening on the characteristic parameter sequence to obtain cognitive impairment information of the user, including: The cognitive impairment screening module processes the characteristic parameter sequence using the trained cognitive impairment screening model to obtain a cognitive impairment probability value; Determine whether the cognitive impairment probability value is greater than a preset impairment probability threshold to obtain screening result information; if the screening result information is yes, confirm that the user has cognitive impairment, confirm that the cognitive impairment degree information is the cognitive impairment probability value, and obtain cognitive impairment information; if the screening result information is no, confirm that the cognitive impairment information is that the user does not have cognitive impairment.

4. A method for screening cognitive impairment based on a VR eye tracker and a brain-computer interface, implemented by using the device for screening cognitive impairment based on a VR eye tracker and a brain-computer interface as claimed in any one of claims 1 to 3, comprising: S1, using the virtual reality scene generation module to generate a virtual reality scene including a cognitive task; Displaying the virtual reality scene containing the cognitive task to the user; S2, using the eye tracker data acquisition module to acquire in real time a set of eye movement data information when the user completes a cognitive task in a virtual reality scene; using the brain-computer interface data acquisition module to acquire a set of electroencephalogram signal information when the user completes a cognitive task in a virtual reality scene; S3, using the data analysis and processing module to analyze and process the collected eye movement data information set and the EEG signal information set, and extract a characteristic parameter sequence; S4, using the cognitive impairment screening module to perform cognitive impairment screening processing on the characteristic parameter sequence to obtain cognitive impairment information of the user.

5. The method for screening cognitive impairment based on VR eye tracker and brain-computer interface as claimed in claim 4, characterized in that: The step of analyzing and processing the collected eye movement data information set and the EEG signal information set to extract a characteristic parameter sequence includes: S31, performing preprocessing operations on the collected eye movement data information set and the electroencephalogram signal information set to obtain a preprocessed eye movement data information set and an electroencephalogram signal information set; S32, performing abnormal feature analysis on the preprocessed eye movement data information set to obtain an eye movement pattern abnormal parameter sequence; S33, performing frequency domain anomaly recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence; S34, fusing the eye movement pattern abnormal parameter sequence and the EEG signal spectrum feature sequence to obtain a feature parameter sequence.

6. The method for screening cognitive impairment based on VR eye tracker and brain-computer interface as claimed in claim 5, characterized in that: The pre-processing operation comprises: S311, normalizing each information set to obtain a corresponding normalized information set; S312, performing data cleaning processing on each normalized information set to obtain a corresponding cleaned data set; S313, performing pattern recognition and discrimination processing on each cleaned data set to obtain a corresponding pre-processed information set.

7. The method for screening cognitive impairment based on VR eye tracker and brain-computer interface as claimed in claim 6, characterized in that: The pattern recognition and discrimination processing includes: S3131, for each type of data attribute of the cleaned data set, the data collection information of the data is used as the independent variable, and the data value of the data is used as the dependent variable, and autoregression-sliding average modeling is performed to obtain the regression model of the data attribute of the class; using the regression model, the independent variable is calculated and processed to obtain the regression data value; it is determined whether the absolute value of the difference between the regression data value and the corresponding dependent variable value is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, the data is deleted from the cleaned data set; if it is less than or equal to the first regression discrimination threshold, the data is not processed; S3132, performing fusion processing on all data after executing S3131 of the cleaned data set to obtain a pre-processed information set.

8. The method for screening cognitive impairment based on VR eye tracker and brain-computer interface as claimed in claim 5, characterized in that: The performing abnormal feature analysis on the pre-processed eye movement data information set to obtain an abnormal eye movement pattern parameter sequence includes: Performing deviation evaluation processing on the fixation point information sequence in the preprocessed eye movement data information set to obtain a deviation value of each fixation point information sequence; Determine a gaze point information sequence whose deviation value is greater than a preset deviation threshold as a gaze abnormal parameter sequence; The calculation expression of the deviation evaluation process is: Among them, p1 is the calculation parameter, a i is the deviation value of the i-th fixation point information sequence, (x0, y0, z0) is the position coordinate of the preset target fixation point in the cognitive task, (x ij ,y ij ,z ij ) is the jth fixation point position coordinate of the ith fixation point information sequence, and M is the number of fixation point position coordinates contained in a fixation point information sequence; Performing joint abnormality assessment processing on the fixation time information sequence and the eye movement frequency information sequence in the preprocessed eye movement data information set to obtain a time-frequency deviation value of each fixation time information; Determine the fixation time information and eye saccade frequency information whose time-frequency deviation value is greater than a preset time-frequency deviation threshold as time-frequency abnormality parameters; All the abnormal gaze parameter sequences and time-frequency abnormal parameters are combined to obtain the abnormal eye movement pattern parameter sequence; The calculation expression of the joint anomaly assessment process is: c i =arctan(|t0-t i | / |t0+t i |+|f0-f i | / |f0+f i |) / pi+(|t0-t i |+|f0-f i |) / (t0+f0), where c i is the time-frequency abnormality parameter of the ith fixation time information in the fixation time information sequence, t0 and f0 are the standard fixation time value and the standard eye saccade frequency value respectively, t i and f i are the i-th fixation time information in the fixation time information sequence and the i-th saccade frequency information in the saccade frequency information sequence, and pi is the pi constant.

9. The method for screening cognitive impairment based on VR eye tracker and brain-computer interface as claimed in claim 5, characterized in that: The performing frequency domain anomaly recognition processing on the preprocessed EEG signal information set to obtain an EEG signal spectrum feature sequence includes: Performing high-order feature quantity calculation processing on each EEG signal sequence of the preprocessed EEG signal information set to obtain an abnormality discrimination value; Determine that the spectrum sequence of the EEG signal sequence whose abnormal discrimination value is greater than the set abnormal threshold value is an abnormal EEG sequence; Using all abnormal EEG sequences, construct the EEG signal spectrum feature sequence; The expression for the high-order feature quantity calculation process is: α=ω1R4+ω2R 23 +ω3R′ 56 , Where α is the abnormal discrimination value of the EEG signal sequence, FFT() represents the discrete Fourier transform operation, x(n) represents the nth element of the EEG signal sequence, and y(n) represents the nth element of the frequency domain signal of the EEG signal sequence. represents the conjugate element of y(n), ω1, ω2, ω3 represent the preset first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, N represents the length of the EEG signal sequence, R4 represents the fourth-order self-accumulative amount of the EEG signal sequence, R 23 represents the second and third order cross-cumulants of the EEG signal sequence, and R5′6 represents the fifth and sixth order cross-conjugate cross-cumulants of the EEG signal sequence.

10. The method for screening cognitive impairment based on VR eye tracker and brain-computer interface as claimed in claim 5, characterized in that: The cognitive impairment screening model includes: an input module, a feature extraction module, a feature fusion module, a feature processing module and a prediction module; The input module is used to receive and obtain a characteristic parameter sequence; The feature extraction module is connected to the input module and the feature fusion module respectively, and is used to extract features from the feature parameter sequence to obtain a feature signal set; The feature fusion module is used to perform fusion processing on the feature signal set to obtain a fused feature signal set; The feature processing module is connected to the feature fusion module and the prediction module respectively, and is used to perform time series correlation processing on the fused feature signal set to obtain a time series cross feature signal set; The prediction module is used to perform prediction processing on the time series cross feature signal set to obtain a cognitive impairment probability value; The feature extraction module includes four feature extractors corresponding to the signal modes; each feature extractor is used to perform feature extraction processing on the feature parameter sequence of the corresponding mode to obtain the corresponding feature signal; The feature fusion module includes four multi-head attention sub-modules; each multi-head attention sub-module is used to process the feature signal of the corresponding modality; the dimension of each multi-head attention sub-module matches the dimension of the feature signal of the modality it processes; the input end of each multi-head attention sub-module is connected to the output end of the corresponding feature extractor; The feature processing module includes four long short-term memory network submodules; each long short-term memory network submodule is used to process the feature parameter sequence of the corresponding modality; each long short-term memory network submodule is connected to the output end of the corresponding multi-head attention submodule; The prediction module includes a fully connected network and an output end; the four input ends of the fully connected network are respectively connected to the output ends of the four long short-term memory network sub-modules; the output end of the prediction module is used to output the probability value of cognitive impairment.

Images