Cognitive disorder screening method and device based on VR eye tracker and near-infrared brain imaging

Abstract

The invention discloses a cognitive impairment screening method and device based on a VR eye tracker and near-infrared brain imaging. The device comprises a virtual reality scene generation module, an eye tracker data acquisition module, a near-infrared brain imaging module, a data evaluation processing module and a cognitive impairment screening module. The eye tracker data acquisition module is used for acquiring an eye movement data information set of a user in a virtual reality scene in real time; the eye movement data information set comprises a fixation point information sequence, a pupil area information sequence and an eye hopping frequency information sequence; the near-infrared brain imaging module is used for acquiring a blood oxygen information set of cerebral cortex when a user completes a cognitive task; the blood oxygen information set of the cerebral cortex comprises a brain blood oxygen information sequence; the data evaluation processing module is used for carrying out evaluation processing on the collected eye movement data information set and the blood oxygen information set of cerebral cortex to obtain cognitive disorder information of the user; the cognitive impairment information comprises information about whether cognitive impairment exists or not and the degree of the cognitive impairment.

Description

Technical Field

[0001] The present invention relates to the fields of virtual reality and brain cognition, and in particular to a method and device for screening cognitive impairment based on a VR eye tracker and near-infrared brain imaging. Background Art

[0002] With the acceleration of the trend of aging society, more and more people are suffering from cognitive impairment. Cognitive impairment is a common neurological abnormality, including dementia, mild cognitive impairment, etc. Early and accurate screening is crucial for the diagnosis and treatment of cognitive impairment. At present, commonly used screening methods for cognitive impairment include neuropsychological tests, imaging examinations, etc., but these methods have certain limitations, such as strong subjectivity, complex operation, and insensitivity to early lesions.

[0003] In recent years, with the development of hardware technology, eye tracking technology and brain imaging technology have been widely used in cognitive research. Virtual reality technology can provide an immersive interactive environment, increasing the fun and authenticity of the test. However, the current methods and devices that combine these technologies for cognitive impairment screening have the problems of high implementation cost and large data processing volume. How to achieve rapid, real-time and low-cost detection of brain cognitive impairment is a problem that needs to be solved at present. Summary of the invention

[0004] The present invention mainly solves the problem of how to achieve rapid, real-time and low-cost detection of brain cognitive disorders. The present invention discloses a method and device for screening cognitive disorders based on a VR eye tracker and near-infrared brain imaging.

[0005] The purpose of the present invention is to provide a method and device for screening cognitive impairment based on a virtual reality eye tracker and near-infrared brain imaging, so as to address the limitations of existing cognitive impairment screening methods, improve the accuracy, objectivity and efficiency of screening, and reduce screening costs.

[0006] In a first aspect of the embodiment of the present application, a cognitive impairment screening device based on a VR eye tracker and near-infrared brain imaging is disclosed, comprising: a virtual reality scene generation module, an eye tracker data acquisition module, a near-infrared brain imaging module, a data evaluation and processing module, and a cognitive impairment screening module;

[0007] The eye tracker data acquisition module is used to acquire in real time an eye movement data information set when the user completes a cognitive task in a virtual reality scene; the eye movement data information set includes a fixation point information sequence, a pupil area information sequence, and an eye saccade frequency information sequence; the fixation point information sequence includes fixation point position information; the pupil area information sequence includes pupil area information; the eye saccade frequency information sequence includes eye saccade frequency information;

[0008] The near-infrared brain imaging module is used to collect in real time a set of blood oxygen information of the cerebral cortex when the user completes a cognitive task in a virtual reality scene; the set of blood oxygen information of the cerebral cortex includes a brain blood oxygen information sequence;

[0009] The data evaluation and processing module is connected to the virtual reality scene generation module, the eye tracker data acquisition module, and the near-infrared brain imaging module respectively, and is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information; the cognitive impairment information includes whether there is a cognitive impairment and the degree of cognitive impairment.

[0010] The data evaluation and processing module is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information, including:

[0011] Performing a preprocessing operation on the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain a preprocessing information set;

[0012] Performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence; the feature parameter sequence includes an eye movement pattern abnormality parameter sequence and a brain blood oxygen information spectrum feature sequence;

[0013] The characteristic parameter sequence is subjected to cognitive impairment screening processing to obtain cognitive impairment information of the user; the cognitive impairment information includes whether cognitive impairment exists and information on the degree of cognitive impairment.

[0014] The step of performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence includes:

[0015] Performing eye movement abnormality analysis on the preprocessed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence;

[0016] Performing blood oxygen abnormality recognition processing on the preprocessed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence;

[0017] The eye movement pattern abnormal parameter sequence and the brain blood oxygen information spectrum feature sequence are fused to obtain a feature parameter sequence.

[0018] In a second aspect of the embodiment of the present application, a method for screening cognitive impairment based on a VR eye tracker and near-infrared brain imaging is disclosed, which is implemented using the cognitive impairment screening device based on a VR eye tracker and near-infrared brain imaging, including:

[0019] Using the virtual reality scene generation module, generate and display a virtual reality scene containing cognitive tasks;

[0020] Using the eye tracker data acquisition module, a collection of eye movement data information of the user when completing cognitive tasks in a virtual reality scene is acquired in real time;

[0021] By using the near-infrared brain imaging module, a collection of blood oxygen information of the cerebral cortex of the user when completing cognitive tasks in a virtual reality scene is collected in real time;

[0022] The data evaluation and processing module is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information; the cognitive impairment information includes whether there is cognitive impairment and the degree of cognitive impairment.

[0023] The data evaluation and processing module is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information, including:

[0024] Performing a preprocessing operation on the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain a preprocessing information set;

[0025] Performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence; the feature parameter sequence includes an eye movement pattern abnormality parameter sequence and a brain blood oxygen information spectrum feature sequence;

[0026] The characteristic parameter sequence is subjected to cognitive impairment screening processing to obtain the user's cognitive impairment information.

[0027] The step of performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence includes:

[0028] Performing eye movement abnormality analysis on the preprocessed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence;

[0029] Performing blood oxygen abnormality recognition processing on the preprocessed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence;

[0030] The eye movement pattern abnormal parameter sequence and the brain blood oxygen information spectrum feature sequence are fused to obtain a feature parameter sequence.

[0031] The pre-processing operation comprises:

[0032] Normalizing each information set to obtain a corresponding normalized information set;

[0033] Perform data cleaning on each normalized information set to obtain a corresponding cleaned data set;

[0034] Perform feature pre-analysis processing on each cleaned data set to obtain a corresponding pre-processed information set;

[0035] The preprocessed eye movement data information set and the preprocessed cerebral cortex blood oxygen information set are fused to obtain a preprocessed information set.

[0036] The performing eye movement abnormality analysis on the pre-processed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence includes:

[0037] Using the pre-processed eye movement data information set, an eye movement collection data matrix A is constructed; the row vector of the eye movement collection data matrix is ​​a fixation point information sequence, a pupil area information sequence, and an eye saccade frequency information sequence;

[0038] Obtaining a standard eye movement data vector; the standard eye movement data vector includes a standard value of gaze point information, a standard value of pupil area, and a standard value of eye saccade frequency;

[0039] The standard eye movement data vector is used as a column vector, and is expanded and replicated in the row direction to obtain a standard data matrix B having the same dimension as the eye movement acquisition data matrix;

[0040] Performing difference calculation processing on the eye movement data matrix and the standard data matrix to obtain an eye movement pattern abnormal parameter sequence;

[0041] The expression for the difference calculation process is:

[0042]

[0043] Among them, h i is the i-th element in the eye movement pattern abnormal parameter sequence, is the mean of the i-th row of the eye movement data matrix A, A ij is the element of the i-th row and j-th column of the eye movement data matrix A, B ij is the element in the i-th row and j-th column of the standard data matrix B, and N is the column dimension of the eye movement data matrix.

[0044] The performing blood oxygen abnormality recognition processing on the pre-processed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence includes:

[0045] Performing a two-dimensional transformation process on each blood oxygen information sequence of the preprocessed cerebral cortex blood oxygen information set to obtain a corresponding time-frequency matrix;

[0046] Perform feature solving processing on each time-frequency matrix to obtain the corresponding eigenvalue;

[0047] Using the eigenvalues ​​of all time-frequency matrices, a characteristic sequence of brain blood oxygen information spectrum is constructed;

[0048] The expression of the feature solving process is:

[0049] W -1 Xv=λv,

[0050] Among them, W is the orthogonal transformation matrix, X is the time-frequency matrix, for W -1 The solution of Xv=λv is achieved by adopting the generalized eigenvalue solution algorithm; v represents the eigenvector obtained by solving the generalized eigenvalue operation, and λ represents the eigenvalue obtained by solving the generalized eigenvalue operation; the element with the largest absolute value among all eigenvalues ​​is taken as the eigenvalue corresponding to the time-frequency matrix X obtained by the eigensolution process.

[0051] The calculation expression of the two-dimensional transformation processing is:

[0052]

[0053] Among them, s(k) is the kth element of the blood oxygen information sequence, NS is the length of the blood oxygen information sequence, y1(z,x) is the element of the zth row and xth column of the time-frequency matrix obtained by the two-dimensional transform processing, p1() is the transformation function corresponding to the two-dimensional transform processing, p1(kT1-zT1) is the value of p1() at kT1-zT1, T1 and F1 are the first transform length and the second transform length of the two-dimensional transform processing, respectively.

[0054] The beneficial effects of the present invention are:

[0055] The present invention introduces virtual reality technology, eye tracker and near-infrared brain imaging technology into the field of cognitive impairment screening. Virtual reality technology can create a realistic virtual environment, simulate various daily life scenes, and improve the fun and participation of the test; the eye tracker can monitor the user's eye movement trajectory in real time and obtain information such as visual attention; near-infrared brain imaging technology can monitor the brain's neural activity by monitoring the blood in the brain, reflecting the state of cognitive function.

[0056] The present invention significantly reduces the amount and cost of data collection by introducing near-infrared brain imaging technology, which helps to achieve real-time and low-cost monitoring of cognitive impairment.

[0057] The present invention performs evaluation and processing based on the collected eye movement data information set and the blood oxygen information set of the cerebral cortex. First, abnormal feature data is extracted, and abnormal data extraction models corresponding to the blood oxygen signal and the eye movement signal are established respectively, thereby achieving accurate extraction of abnormal data and greatly reducing the amount of data used for cognitive evaluation.

[0058] Based on abnormal data, the present invention establishes an evaluation model that integrates time-frequency transformation and feature normalization processing, fully utilizes multi-source heterogeneous abnormal information, and improves the accuracy of cognitive impairment assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 1 is a flow chart for implementing the method of the present invention;

[0060] Figure 2 It is a composition diagram of the device of the present invention. DETAILED DESCRIPTION

[0061] In order to better understand the content of the present invention, an embodiment is given here.

[0062] Figure 1 1 is a flow chart for implementing the method of the present invention; Figure 2 It is a composition diagram of the device of the present invention.

[0063] In a first aspect of the embodiment of the present application, a cognitive impairment screening device based on a VR eye tracker and near-infrared brain imaging is disclosed, comprising: a virtual reality scene generation module, an eye tracker data acquisition module, a near-infrared brain imaging module, a data evaluation and processing module, and a cognitive impairment screening module;

[0064] The virtual reality scene generation module is used to generate and display a virtual reality scene containing cognitive tasks; the cognitive tasks include object recognition, spatial navigation, memory test, etc.;

[0065] The eye tracker data acquisition module is used to acquire in real time the eye movement data information set of the user when completing cognitive tasks in the virtual reality scene; the eye movement data information set includes a fixation point information sequence, a pupil area information sequence, an eye saccade frequency information sequence, etc. The fixation point information sequence includes fixation point position information; the pupil area information sequence includes pupil area information; the eye saccade frequency information sequence includes eye saccade frequency information;

[0066] The near-infrared brain imaging module is used to collect in real time a set of blood oxygen information of the cerebral cortex when the user completes a cognitive task in a virtual reality scene. The set of blood oxygen information of the cerebral cortex includes a brain blood oxygen information sequence;

[0067] The data evaluation and processing module is connected to the virtual reality scene generation module, the eye tracker data acquisition module, and the near-infrared brain imaging module, respectively, and is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information; the cognitive impairment information includes whether there is cognitive impairment and the degree of cognitive impairment;

[0068] The data evaluation and processing module is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information, including:

[0069] Performing a preprocessing operation on the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain a preprocessing information set;

[0070] The preprocessing information set is subjected to feature extraction processing to obtain a feature parameter sequence; the feature parameter sequence includes an eye movement pattern abnormality parameter sequence and a brain blood oxygen information spectrum feature sequence.

[0071] Perform cognitive impairment screening on the characteristic parameter sequence to obtain cognitive impairment information of the user; the cognitive impairment information includes whether cognitive impairment exists and the degree of cognitive impairment;

[0072] The step of performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence includes:

[0073] Performing eye movement abnormality analysis on the preprocessed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence;

[0074] Performing blood oxygen abnormality recognition processing on the preprocessed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence;

[0075] The eye movement pattern abnormal parameter sequence and the brain blood oxygen information spectrum feature sequence are fused to obtain a feature parameter sequence.

[0076] The virtual reality scene generation module can be implemented by using virtual reality glasses; the eye tracker data acquisition module can be implemented by using an eye tracker or a VR eye tracker developed by Xi'an Blue Brain Technology Co., Ltd.; the near-infrared brain imaging module can be implemented by using a near-infrared brain imaging acquisition device.

[0077] The pre-processing operation comprises:

[0078] Normalizing each information set to obtain a corresponding normalized information set;

[0079] Perform data cleaning on each normalized information set to obtain a corresponding cleaned data set;

[0080] Perform feature pre-analysis processing on each cleaned data set to obtain a corresponding pre-processed information set;

[0081] The preprocessed eye movement data information set and the preprocessed cerebral cortex blood oxygen information set are fused to obtain a preprocessed information set;

[0082] The performing eye movement abnormality analysis on the pre-processed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence includes:

[0083] Using the pre-processed eye movement data information set, an eye movement collection data matrix A is constructed; the row vector of the eye movement collection data matrix is ​​a fixation point information sequence, a pupil area information sequence, and an eye saccade frequency information sequence;

[0084] Obtaining a standard eye movement data vector; the standard eye movement data vector includes a standard value of gaze point information, a standard value of pupil area, and a standard value of eye saccade frequency;

[0085] The standard eye movement data vector is used as a column vector, and is expanded and replicated in the row direction to obtain a standard data matrix B having the same dimension as the eye movement acquisition data matrix;

[0086] Performing difference calculation processing on the eye movement data matrix and the standard data matrix to obtain an eye movement pattern abnormal parameter sequence;

[0087] The expression for the difference calculation process is:

[0088]

[0089] Among them, h i is the i-th element in the eye movement pattern abnormal parameter sequence, is the mean of the i-th row of the eye movement data matrix A, A ij is the element of the i-th row and j-th column of the eye movement data matrix A, B ij is the element of the i-th row and j-th column of the standard data matrix B, and N is the column dimension of the eye movement data matrix;

[0090] The performing blood oxygen abnormality recognition processing on the pre-processed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence includes:

[0091] Performing a two-dimensional transformation process on each blood oxygen information sequence of the preprocessed cerebral cortex blood oxygen information set to obtain a corresponding time-frequency matrix;

[0092] Perform feature solving processing on each time-frequency matrix to obtain the corresponding eigenvalue;

[0093] Using the eigenvalues ​​of all time-frequency matrices, a characteristic sequence of brain blood oxygen information spectrum is constructed;

[0094] The calculation expression of the two-dimensional transformation processing is:

[0095]

[0096] Wherein, s(k) is the kth element of the blood oxygen information sequence, NS is the length of the blood oxygen information sequence, y1(z,x) is the element of the zth row and xth column of the time-frequency matrix obtained by the two-dimensional transform processing, p1() is the transformation function corresponding to the two-dimensional transform processing, p1(kT1-zT1) is the value of p1() at kT1-zT1, and T1 and F1 are respectively the first transformation length and the second transformation length of the two-dimensional transform processing;

[0097] The expression of the feature solving process is:

[0098] W -1 Xv=λv,

[0099] Where W is the orthogonal transformation matrix, which can be a Hermite matrix, and X is the time-frequency matrix. -1 The solution of Xν=λν is realized by adopting a generalized eigenvalue solution algorithm; v represents an eigenvector obtained by solving the generalized eigenvalue operation, and λ represents an eigenvalue obtained by solving the generalized eigenvalue operation; the element with the largest absolute value among all eigenvalues ​​is taken as the eigenvalue corresponding to the time-frequency matrix X obtained by the eigenvalue solution process;

[0100] The cognitive impairment screening process is performed on the characteristic parameter sequence to obtain the cognitive impairment information of the user, including:

[0101] Performing time-frequency transformation processing on the abnormal eye movement pattern parameter sequence to obtain an eye movement feature vector; performing normalization processing on the eye movement feature vector to obtain a normalized eye movement feature vector;

[0102] Performing feature normalization processing on the brain blood oxygen information spectrum feature sequence to obtain a normalized blood oxygen feature vector;

[0103] The calculation expression of the feature normalization process is:

[0104]

[0105] Where T is the number of elements contained in the brain blood oxygen information spectrum feature sequence, t i is the i-th element of the brain blood oxygen information spectrum feature sequence, is the i-th element of the normalized blood oxygen feature vector, t max is the largest element of the brain blood oxygen information spectrum characteristic sequence, and pi is the pi constant;

[0106] Perform weighted summation on the normalized eye movement feature vector and the normalized blood oxygen feature vector to obtain information on the degree of cognitive impairment;

[0107] Determine whether the cognitive impairment degree information is greater than a set discrimination threshold, if it is greater than the set discrimination threshold, determine that the user has cognitive impairment; if it is not greater than the set discrimination threshold, determine that the user does not have cognitive impairment;

[0108] Cognitive impairment information is constructed using the result information of whether cognitive impairment exists and the degree of cognitive impairment.

[0109] The cognitive impairment screening process is performed on the characteristic parameter sequence to obtain the cognitive impairment information of the user, including:

[0110] Perform cross-feature quantity calculation on the feature parameter sequence to obtain information on the degree of cognitive impairment;

[0111] Determine whether the cognitive impairment degree information is greater than a set discrimination threshold, if it is greater than the set discrimination threshold, determine that the user has cognitive impairment; if it is not greater than the set discrimination threshold, determine that the user does not have cognitive impairment;

[0112] Cognitive impairment information is constructed using the result information of whether cognitive impairment exists and the degree of cognitive impairment.

[0113] The expression for the cross feature quantity calculation process is:

[0114]

[0115] α=ω1U 12 +ω2U 23 +ω3U 34 ,

[0116] Among them, α is the information of cognitive impairment degree, FFT() represents discrete Fourier transform operation, x(n) represents the nth element of eye movement pattern abnormal parameter sequence, y(n) represents the nth element of brain blood oxygen information spectrum feature sequence, represents the conjugate element of y(n), ω1, ω2, ω3 represent the preset first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, N represents the length of the eye movement pattern abnormal parameter sequence, U 12 represents the first feature quantity, U 23 represents the second feature quantity, U 34 represents the third feature quantity.

[0117] In a second aspect of the embodiment of the present application, a method for screening cognitive impairment based on a VR eye tracker and near-infrared brain imaging is disclosed, comprising:

[0118] Using the virtual reality scene generation module, generate and display a virtual reality scene containing cognitive tasks;

[0119] Using the eye tracker data acquisition module, a collection of eye movement data information of the user when completing cognitive tasks in a virtual reality scene is acquired in real time;

[0120] By using the near-infrared brain imaging module, a set of blood oxygen information of the cerebral cortex of the user when completing a cognitive task in a virtual reality scene is collected in real time; the set of blood oxygen information of the cerebral cortex includes a brain blood oxygen information sequence;

[0121] Using the data evaluation and processing module, the collected eye movement data information set and the cerebral cortex blood oxygen information set are evaluated and processed to obtain the user's cognitive impairment information; the cognitive impairment information includes whether there is cognitive impairment and the degree of cognitive impairment;

[0122] The data evaluation and processing module is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information, including:

[0123] Performing a preprocessing operation on the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain a preprocessing information set;

[0124] The preprocessing information set is subjected to feature extraction processing to obtain a feature parameter sequence; the feature parameter sequence includes an eye movement pattern abnormality parameter sequence and a brain blood oxygen information spectrum feature sequence.

[0125] Perform cognitive impairment screening on the characteristic parameter sequence to obtain cognitive impairment information of the user; the cognitive impairment information includes whether cognitive impairment exists and the degree of cognitive impairment;

[0126] The step of performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence includes:

[0127] Performing eye movement abnormality analysis on the preprocessed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence;

[0128] Performing blood oxygen abnormality recognition processing on the preprocessed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence;

[0129] The eye movement pattern abnormal parameter sequence and the brain blood oxygen information spectrum feature sequence are fused to obtain a feature parameter sequence.

[0130] The pre-processing operation comprises:

[0131] Normalizing each information set to obtain a corresponding normalized information set;

[0132] Perform data cleaning on each normalized information set to obtain a corresponding cleaned data set;

[0133] Perform feature pre-analysis processing on each cleaned data set to obtain a corresponding pre-processed information set;

[0134] The preprocessed eye movement data information set and the preprocessed cerebral cortex blood oxygen information set are fused to obtain a preprocessed information set;

[0135] The performing eye movement abnormality analysis on the pre-processed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence includes:

[0136] Using the pre-processed eye movement data information set, an eye movement collection data matrix A is constructed; the row vector of the eye movement collection data matrix is ​​a fixation point information sequence, a pupil area information sequence, and an eye saccade frequency information sequence;

[0137] Obtaining a standard eye movement data vector; the standard eye movement data vector includes a standard value of gaze point information, a standard value of pupil area, and a standard value of eye saccade frequency;

[0138] The standard eye movement data vector is used as a column vector, and is expanded and replicated in the row direction to obtain a standard data matrix B having the same dimension as the eye movement acquisition data matrix;

[0139] Performing difference calculation processing on the eye movement data matrix and the standard data matrix to obtain an eye movement pattern abnormal parameter sequence;

[0140] The expression for the difference calculation process is:

[0141]

[0142] Among them, h i is the i-th element in the eye movement pattern abnormal parameter sequence, is the mean of the i-th row of the eye movement data matrix A, A ij is the element of the i-th row and j-th column of the eye movement data matrix A, B ij is the element of the i-th row and j-th column of the standard data matrix B, and N is the column dimension of the eye movement data matrix;

[0143] The performing blood oxygen abnormality recognition processing on the pre-processed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence includes:

[0144] Performing a two-dimensional transformation process on each blood oxygen information sequence of the preprocessed cerebral cortex blood oxygen information set to obtain a corresponding time-frequency matrix;

[0145] Perform feature solving processing on each time-frequency matrix to obtain the corresponding eigenvalue;

[0146] Using the eigenvalues ​​of all time-frequency matrices, a characteristic sequence of brain blood oxygen information spectrum is constructed;

[0147] The calculation expression of the two-dimensional transformation processing is:

[0148]

[0149] Wherein, s(k) is the kth element of the blood oxygen information sequence, NS is the length of the blood oxygen information sequence, y1(z,x) is the element of the zth row and xth column of the time-frequency matrix obtained by the two-dimensional transform processing, p1() is the transformation function corresponding to the two-dimensional transform processing, p1(kT1-zT1) is the value of p1() at kT1-zT1, and T1 and F1 are respectively the first transformation length and the second transformation length of the two-dimensional transform processing;

[0150] The expression of the feature solving process is:

[0151] W -1 Xv=λv,

[0152] Where W is the orthogonal transformation matrix, which can be a Hermite matrix, and X is the time-frequency matrix. -1 The solution of Xv=λv is realized by adopting a generalized eigenvalue solution algorithm; ν represents an eigenvector obtained by solving the generalized eigenvalue operation, and λ represents an eigenvalue obtained by solving the generalized eigenvalue operation; the element with the largest absolute value among all eigenvalues ​​is taken as the eigenvalue corresponding to the time-frequency matrix X obtained by the eigenvalue solution process;

[0153] The cognitive impairment screening process is performed on the characteristic parameter sequence to obtain the cognitive impairment information of the user, including:

[0154] Performing time-frequency transformation processing on the abnormal eye movement pattern parameter sequence to obtain an eye movement feature vector; performing normalization processing on the eye movement feature vector to obtain a normalized eye movement feature vector;

[0155] Performing feature normalization processing on the brain blood oxygen information spectrum feature sequence to obtain a normalized blood oxygen feature vector;

[0156] The calculation expression of the feature normalization process is:

[0157]

[0158] Where T is the number of elements contained in the brain blood oxygen information spectrum feature sequence, t i is the i-th element of the brain blood oxygen information spectrum feature sequence, is the i-th element of the normalized blood oxygen feature vector, t max is the largest element of the brain blood oxygen information spectrum characteristic sequence, and pi is the pi constant;

[0159] Perform weighted summation on the normalized eye movement feature vector and the normalized blood oxygen feature vector to obtain information on the degree of cognitive impairment;

[0160] Determine whether the cognitive impairment degree information is greater than a set discrimination threshold, if it is greater than the set discrimination threshold, determine that the user has cognitive impairment; if it is not greater than the set discrimination threshold, determine that the user does not have cognitive impairment;

[0161] Cognitive impairment information is constructed using the result information of whether cognitive impairment exists and the degree of cognitive impairment.

[0162] The cognitive impairment screening process is performed on the characteristic parameter sequence to obtain the cognitive impairment information of the user, including:

[0163] Perform cross-feature quantity calculation on the feature parameter sequence to obtain information on the degree of cognitive impairment;

[0164] Determine whether the cognitive impairment degree information is greater than a set discrimination threshold, if it is greater than the set discrimination threshold, determine that the user has cognitive impairment; if it is not greater than the set discrimination threshold, determine that the user does not have cognitive impairment;

[0165] Cognitive impairment information is constructed using the result information of whether cognitive impairment exists and the degree of cognitive impairment.

[0166] The expression for the cross feature quantity calculation process is:

[0167]

[0168] α=ω1U 12 +ω2U 23 +ω3U 34 ,

[0169] Among them, α is the information of cognitive impairment degree, FFT() represents discrete Fourier transform operation, x(n) represents the nth element of eye movement pattern abnormal parameter sequence, y(n) represents the nth element of brain blood oxygen information spectrum feature sequence, represents the conjugate element of y(n), ω1, ω2, ω3 represent the preset first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, N represents the length of the eye movement pattern abnormal parameter sequence, U 12 represents the first feature quantity, U 23 represents the second feature quantity, U 34 represents the third feature quantity.

[0170] The weight values ​​used for weighted summing of the normalized eye movement feature vector and the normalized blood oxygen feature vector may be 0.5 and 0.5, or 0.4 and 0.6.

[0171] The time-frequency transformation may adopt wavelet transformation, orthogonal filter bank or Cohen-type time-frequency distribution method;

[0172] The normalization process is to map data in different value ranges to a specified value range, and its mathematical expression is:

[0173]

[0174] Among them, t is a data value in the data set, t max is the maximum value of all data in the data set, t min is the minimum value of all data in the data set, t 归一化 is the value of data t after normalization;

[0175] The data cleaning process includes filling missing values, smoothing noise data, and smoothing or deleting outliers; the smoothed noise data is obtained by first determining the noise data, and then smoothing the noise data according to the data before and after the noise data; the noise data is a value whose value is less than the detection sensitivity of the sensor of the observed data, or greater than the measurement upper limit of the sensor of the observed data. The outlier point can be determined by the Kalman filter method. The filling value of the missing value can be determined by averaging the measured values ​​within a certain sampling interval before and after the missing value.

[0176] The transformation function corresponding to the two-dimensional transformation process may be a Gaussian function.

[0177] The feature pre-analysis processing includes:

[0178] For each type of data attribute of the cleaned data set, the data collection information of the data is used as a known independent variable, and the data value of the data is used as a known dependent variable. The known independent variable and the known dependent variable are used to construct a curve to be approximated, and the function approximation method is used to perform curve fitting on the curve to be approximated to obtain the best consistent approximation polynomial f(Ix) of the type of data attribute; the best consistent approximation polynomial f(Ix) is used to calculate and process the known independent variables to obtain an approximate dependent variable; it is determined whether the absolute value of the difference between the approximate dependent variable and the corresponding known dependent variable is greater than a set second regression discrimination threshold; if it is greater than the second regression discrimination threshold, the data is deleted from the cleaned data set; if it is less than or equal to the second regression discrimination threshold, the data is not processed;

[0179] Performing fusion processing on all data of the cleaned data set that perform feature pre-analysis to obtain a corresponding pre-processed information set;

[0180] The curve fitting of the curve to be approximated by the function approximation method may be performed by using the best consistent linear approximation method. The best consistent approximation polynomial f(Ix) is expressed as:

[0181] f(Ix)=α P1 (Ix) P1 +α P1-1 (Ix) P1-1 +…+α2(Ix) 2 +α1(Ix)+α0,

[0182] Where P1 is the order of the best consistent approximation polynomial f(Ix), α0, α1, α2, …, α P1 are the coefficients of the optimal consistent approximation polynomial f(Ix);

[0183] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A device for screening cognitive impairment based on VR eye tracker and near-infrared brain imaging, characterized in that: include: Virtual reality scene generation module, eye tracker data acquisition module, near-infrared brain imaging module, data evaluation and processing module, cognitive impairment screening module; The virtual reality scene generation module is used to generate and display a virtual reality scene containing cognitive tasks; The eye tracker data acquisition module is used to acquire in real time an eye movement data information set when the user completes a cognitive task in a virtual reality scene; the eye movement data information set includes a fixation point information sequence, a pupil area information sequence, and an eye saccade frequency information sequence; the fixation point information sequence includes fixation point position information; The pupil area information sequence includes pupil area information; the eye saccade frequency information sequence includes eye saccade frequency information; The near-infrared brain imaging module is used to collect in real time a set of blood oxygen information of the cerebral cortex when the user completes a cognitive task in a virtual reality scene; the set of blood oxygen information of the cerebral cortex includes a brain blood oxygen information sequence; The data evaluation and processing module is connected to the virtual reality scene generation module, the eye tracker data acquisition module, and the near-infrared brain imaging module, respectively, and is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information; The cognitive impairment information includes whether there is cognitive impairment and the degree of cognitive impairment.

2. The cognitive impairment screening device based on VR eye tracker and near-infrared brain imaging according to claim 1, characterized in that: The data evaluation and processing module is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information, including: Performing a preprocessing operation on the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain a preprocessing information set; Performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence; the feature parameter sequence includes an eye movement pattern abnormality parameter sequence and a brain blood oxygen information spectrum feature sequence; The characteristic parameter sequence is subjected to cognitive impairment screening processing to obtain cognitive impairment information of the user; the cognitive impairment information includes whether cognitive impairment exists and information on the degree of cognitive impairment.

3. The cognitive impairment screening device based on VR eye tracker and near-infrared brain imaging according to claim 1, characterized in that: The step of performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence includes: Performing eye movement abnormality analysis on the preprocessed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence; Performing blood oxygen abnormality recognition processing on the preprocessed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence; The eye movement pattern abnormal parameter sequence and the brain blood oxygen information spectrum feature sequence are fused to obtain a feature parameter sequence.

4. A method for screening cognitive impairment based on VR eye tracker and near-infrared brain imaging, which is implemented by using the cognitive impairment screening device based on VR eye tracker and near-infrared brain imaging as described in any one of claims 1 to 3, characterized in that: include: Using the virtual reality scene generation module, generate and display a virtual reality scene containing cognitive tasks; Using the eye tracker data acquisition module, a collection of eye movement data information of the user when completing cognitive tasks in a virtual reality scene is acquired in real time; By using the near-infrared brain imaging module, a collection of blood oxygen information of the cerebral cortex of the user when completing cognitive tasks in a virtual reality scene is collected in real time; Using the data evaluation and processing module, the collected eye movement data information set and the cerebral cortex blood oxygen information set are evaluated and processed to obtain the user's cognitive impairment information; The cognitive impairment information includes whether there is cognitive impairment and the degree of cognitive impairment.

5. The method for screening cognitive impairment based on VR eye tracker and near-infrared brain imaging as claimed in claim 4, characterized in that: The data evaluation and processing module is used to evaluate and process the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain the user's cognitive impairment information, including: Performing a preprocessing operation on the collected eye movement data information set and the cerebral cortex blood oxygen information set to obtain a preprocessing information set; Performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence; the feature parameter sequence includes an eye movement pattern abnormality parameter sequence and a brain blood oxygen information spectrum feature sequence; The characteristic parameter sequence is subjected to cognitive impairment screening processing to obtain the user's cognitive impairment information.

6. The method for screening cognitive impairment based on VR eye tracker and near-infrared brain imaging as claimed in claim 5, characterized in that: The step of performing feature extraction processing on the preprocessing information set to obtain a feature parameter sequence includes: Performing eye movement abnormality analysis on the preprocessed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence; Performing blood oxygen abnormality recognition processing on the preprocessed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence; The eye movement pattern abnormal parameter sequence and the brain blood oxygen information spectrum feature sequence are fused to obtain a feature parameter sequence.

7. The method for screening cognitive impairment based on VR eye tracker and near-infrared brain imaging as claimed in claim 6, characterized in that: The pre-processing operation comprises: Normalizing each information set to obtain a corresponding normalized information set; Perform data cleaning on each normalized information set to obtain a corresponding cleaned data set; Perform feature pre-analysis processing on each cleaned data set to obtain a corresponding pre-processed information set; The preprocessed eye movement data information set and the preprocessed cerebral cortex blood oxygen information set are fused to obtain a preprocessed information set.

8. The method for screening cognitive impairment based on VR eye tracker and near-infrared brain imaging as claimed in claim 7, characterized in that: The performing eye movement abnormality analysis on the pre-processed eye movement data information set to obtain an eye movement pattern abnormality parameter sequence includes: Using the pre-processed eye movement data information set, an eye movement collection data matrix A is constructed; the row vector of the eye movement collection data matrix is ​​a fixation point information sequence, a pupil area information sequence, and an eye saccade frequency information sequence; Obtaining a standard eye movement data vector; the standard eye movement data vector includes a standard value of gaze point information, a standard value of pupil area, and a standard value of eye saccade frequency; The standard eye movement data vector is used as a column vector, and is expanded and replicated in the row direction to obtain a standard data matrix B having the same dimension as the eye movement acquisition data matrix; Performing difference calculation processing on the eye movement data matrix and the standard data matrix to obtain an eye movement pattern abnormal parameter sequence; The expression for the difference calculation process is: Among them, h i is the i-th element in the eye movement pattern abnormal parameter sequence, is the mean of the i-th row of the eye movement data matrix A, A ij is the element of the i-th row and j-th column of the eye movement data matrix A, B ij is the element in the i-th row and j-th column of the standard data matrix B, and N is the column dimension of the eye movement data matrix.

9. The method for screening cognitive impairment based on VR eye tracker and near-infrared brain imaging as claimed in claim 8, characterized in that: The performing blood oxygen abnormality recognition processing on the pre-processed cerebral cortex blood oxygen information set to obtain a brain blood oxygen information spectrum feature sequence includes: Performing a two-dimensional transformation process on each blood oxygen information sequence of the preprocessed cerebral cortex blood oxygen information set to obtain a corresponding time-frequency matrix; Perform feature solving processing on each time-frequency matrix to obtain the corresponding eigenvalue; Using the eigenvalues ​​of all time-frequency matrices, a characteristic sequence of brain blood oxygen information spectrum is constructed; The expression of the feature solving process is: W -1 Xν=λν, Among them, W is the orthogonal transformation matrix, X is the time-frequency matrix, for W -1 The solution of Xν=λν is achieved by using a generalized eigenvalue solution algorithm; ν represents the eigenvector obtained by solving the generalized eigenvalue operation, and λ represents the eigenvalue obtained by solving the generalized eigenvalue operation; the element with the largest absolute value among all eigenvalues ​​is taken as the eigenvalue corresponding to the time-frequency matrix X obtained by the eigensolution process.

10. The method for screening cognitive impairment based on VR eye tracker and near-infrared brain imaging as claimed in claim 9, characterized in that: The calculation expression of the two-dimensional transformation processing is: Among them, s(k) is the kth element of the blood oxygen information sequence, NS is the length of the blood oxygen information sequence, y1(z,x) is the element of the zth row and xth column of the time-frequency matrix obtained by the two-dimensional transform processing, p1() is the transformation function corresponding to the two-dimensional transform processing, p1(kT1-zT1) is the value of p1() at kT1-zT1, T1 and F1 are the first transform length and the second transform length of the two-dimensional transform processing, respectively.

Images