Multi-modal cognitive screening method and device based on brain-computer interface and galvanic skin bracelet

Abstract

The invention discloses a multi-modal cognitive screening method and device based on a brain-computer interface and a galvanic skin bracelet. The device comprises a brain-computer interface module, a galvanic skin bracelet, a signal feature extraction module and a cognitive evaluation module. The brain-computer interface module is used for acquiring an electroencephalogram signal sequence when a user performs a cognitive test; the skin electric bracelet is used for acquiring a skin electric signal sequence when a user performs a cognitive test; the signal feature extraction module is used for performing feature extraction processing on the acquired electroencephalogram signal sequence and the acquired skin electrical signal sequence to obtain an electroencephalogram feature sequence and a skin electrical feature sequence; the cognitive assessment module is used for performing cognitive impairment assessment processing on the electroencephalogram feature sequence, the galvanic skin feature sequence, the skin electric signal sequence and the electroencephalogram signal sequence to obtain a cognitive impairment assessment result.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface and brain cognition, and in particular to a multimodal cognitive screening method and device based on a brain-computer interface and an electrodermal bracelet. Background Art

[0002] Cognitive dysfunction seriously affects an individual's daily life and work ability, including but not limited to dementia, mild cognitive impairment, etc. Early and accurate cognitive screening is of great significance for its intervention and early recovery.

[0003] At present, common cognitive screening methods mainly rely on questionnaires, neuropsychological tests, etc. These methods have problems such as strong subjectivity and limited accuracy. As an emerging neurotechnology, brain-computer interface technology can directly obtain neural activity signals from the brain, providing a new way to evaluate cognitive function. However, in actual applications, a single brain-computer interface technology may be affected by a variety of environmental factors and various interference signals, resulting in insufficient stability and reliability of cognitive screening results.

[0004] Electrodermal bracelets can monitor skin electrical activity in real time, reflecting an individual's emotions and physiological arousal state, and are also related to cognitive function. How to organically integrate brain-computer interfaces and electrodermal bracelets to build multimodal cognitive screening methods and devices, thereby effectively improving the accuracy and comprehensiveness of cognitive screening, is an urgent problem that needs to be solved. Summary of the invention

[0005] The present invention mainly solves the problem of how to organically integrate a brain-computer interface and an electric skin bracelet to construct a multimodal cognitive screening method and device, thereby effectively improving the accuracy and comprehensiveness of cognitive screening. The present invention discloses a multimodal cognitive screening method and device based on a brain-computer interface and an electric skin bracelet.

[0006] In a first aspect of the embodiment of the present application, a multimodal cognitive screening device based on a brain-computer interface and an electrodermal bracelet is disclosed, comprising: a brain-computer interface module, an electrodermal bracelet, a signal feature extraction module, and a cognitive assessment module;

[0007] The brain-computer interface module is used to collect and obtain the EEG signal sequence of the user when performing cognitive testing;

[0008] The electrodermal bracelet is used to collect and obtain the user's electrodermal signal sequence when taking cognitive tests;

[0009] The signal feature extraction module is connected to the brain-computer interface module, the skin electrode bracelet and the cognitive assessment module respectively, and is used to perform feature extraction processing on the collected EEG signal sequence and skin electrode signal sequence to obtain an EEG feature sequence and a skin electrode feature sequence;

[0010] The cognitive assessment module is used to perform cognitive impairment assessment processing on the EEG feature sequence, the electrodermal feature sequence, the electrodermal signal sequence and the EEG signal sequence to obtain a cognitive impairment assessment result.

[0011] The signal feature extraction module is used to perform feature extraction processing on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence, including:

[0012] The signal feature extraction module preprocesses the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence;

[0013] The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence.

[0014] The signal feature extraction module preprocesses the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence, including:

[0015] The signal feature extraction module performs data cleaning processing on the collected EEG signal sequence and skin electrical signal sequence to obtain a cleaned EEG signal sequence and a cleaned skin electrical signal sequence;

[0016] The cleaned EEG signal sequence and the cleaned skin electrical signal sequence are respectively subjected to data normalization processing to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence.

[0017] In a second aspect of an embodiment of the present invention, a multimodal cognitive screening method based on a brain-computer interface and an electric skin bracelet is disclosed, which is implemented using the multimodal cognitive screening device based on a brain-computer interface and an electric skin bracelet, and includes:

[0018] Using the brain-computer interface module, the brain electrical signal sequence of the user when taking a cognitive test is collected;

[0019] Using the electrodermal bracelet, a skin electrical signal sequence of the user when taking a cognitive test is collected;

[0020] Using the signal feature extraction module, feature extraction is performed on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence;

[0021] The cognitive assessment module is used to perform cognitive impairment assessment processing on the EEG feature sequence, the electrodermal feature sequence, the electrodermal signal sequence and the EEG signal sequence to obtain a cognitive impairment assessment result.

[0022] The signal feature extraction module is used to perform feature extraction processing on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence, including:

[0023] Using the signal feature extraction module, preprocessing the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence;

[0024] The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence.

[0025] The method of using the signal feature extraction module to preprocess the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence includes:

[0026] Using the signal feature extraction module, the collected EEG signal sequence and skin electrical signal sequence are respectively subjected to data cleaning processing to obtain a cleaned EEG signal sequence and a cleaned skin electrical signal sequence;

[0027] The cleaned EEG signal sequence and the cleaned skin electrical signal sequence are respectively subjected to data normalization processing to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence.

[0028] The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence, including:

[0029] Obtain standard EEG signal sequence;

[0030] Subtracting the preprocessed EEG signal sequence from the standard EEG signal sequence to obtain a difference signal sequence;

[0031] Obtaining a set of statistical characteristic values ​​of the difference signal sequence by statistics; the set of statistical characteristic values ​​includes a median value, a mode value, a range value and a variance value;

[0032] Using a matrix dimension estimation model, the length and statistical characteristic value set of the difference signal sequence are calculated and processed to obtain a matrix row dimension value and a matrix column dimension value;

[0033] Based on the matrix row dimension value and the matrix column dimension value, using the difference signal sequence, construct a difference matrix;

[0034] Performing eigenvalue calculation processing on the difference matrix to obtain an EEG feature sequence;

[0035] Performing time-frequency feature statistical processing on the preprocessed skin electrical signal sequence to obtain a set of time-frequency statistical feature quantities; the set of time-frequency statistical feature quantities includes mean, variance, average rise time, harmonic mean, fluidity value, and average derivative;

[0036] The electrical skin feature calculation process is performed on the time-frequency statistical feature quantity set to obtain an electrical skin feature sequence.

[0037] The expression of the matrix dimension estimation model is:

[0038]

[0039] Wherein, κ is the median value, λ is the mode value, α is the variance value, β is the range value, N is the length of the difference signal sequence, L1 and L2 are the matrix row dimension value and the matrix column dimension value, respectively. Indicates rounding up. Indicates rounding down.

[0040] The step of performing eigenvalue calculation on the difference matrix to obtain an EEG feature sequence includes:

[0041] The difference matrix is ​​decomposed to obtain a feature matrix; the calculation expression of the decomposition process is:

[0042] Y=UAV,

[0043] Among them, U is the left decomposition matrix, Y is the difference matrix, A is the feature matrix, V is the right decomposition matrix, U and V are both orthogonal matrices, and A is a diagonal matrix;

[0044] Extracting the diagonal elements of the feature matrix to obtain a feature vector;

[0045] Performing linear fitting processing on the elements and element sequence values ​​of the feature vector to obtain a feature estimation polynomial;

[0046] For each row vector of the difference matrix, the average value of the row vector is calculated; for each element in the row vector, the absolute value of the difference between the row vector and the average value is calculated, and the absolute value of the difference is determined as the offset of the element;

[0047] For each row vector, find the element with the maximum offset, and determine the sequence number of the element in the row vector, which is the deviation sequence number of the row vector;

[0048] Using the characteristic estimation polynomial, the deviation sequence value of each row vector is calculated and processed to obtain the EEG characteristic value corresponding to each row vector;

[0049] The EEG feature sequence is constructed using the EEG feature values ​​corresponding to all row vectors.

[0050] The step of performing skin electrical feature calculation processing on the time-frequency statistical feature quantity set to obtain a skin electrical feature sequence includes:

[0051] Using the mean and variance, performing a first feature calculation process on the preprocessed skin electrical signal sequence to obtain a first skin electrical feature value H1;

[0052] Performing a second characteristic calculation process on the average rise time and the harmonic mean to obtain a second skin electrical characteristic value H2;

[0053] Performing a third characteristic calculation process on the fluidity value, the average derivative, the mean value and the preprocessed skin electrical signal sequence to obtain a third skin electrical characteristic value H3;

[0054] Constructing a skin electrical feature sequence using the first skin electrical feature, the second skin electrical feature and the third skin electrical feature;

[0055] The expression of the first feature calculation process is:

[0056]

[0057] Among them, r i is the i-th element of the preprocessed skin electrical signal sequence, ω represents the variance, τ represents the mean, and N0 represents the number of elements contained in the preprocessed skin electrical signal sequence;

[0058] The expression of the second feature calculation process is:

[0059]

[0060] Where, μ represents the average rise time, and ν represents the harmonic mean;

[0061] The expression of the third feature calculation process is:

[0062]

[0063] Among them, φ represents the liquidity value and ρ represents the average derivative.

[0064] The beneficial effects of the present invention are:

[0065] The present invention introduces brain-computer interface and electrodermal bracelet into the field of cognitive impairment screening. By organically integrating the signals collected by the brain-computer interface and the electrodermal bracelet, a multimodal cognitive screening method and device are constructed, and fusion is achieved simultaneously at the feature level and the signal level, thereby improving the effect of abnormal signal extraction and ensuring the accuracy of cognitive impairment screening.

[0066] After acquiring the EEG signal and the electrocutaneous signal, the present invention first extracts features from the two types of signals to obtain an EEG feature sequence and an electrocutaneous feature sequence, and then uses the EEG feature sequence, the electrocutaneous feature sequence, the electrocutaneous signal sequence and the EEG signal sequence to perform cognitive impairment assessment processing to obtain a cognitive impairment assessment result. Through this step-by-step approach, the complexity of cognitive impairment assessment is reduced, the assessment process is distributed to two hardware platforms, and the efficiency of the assessment is improved.

[0067] In the process of extracting the EEG feature sequence, the present invention ensures the accuracy of the constructed data matrix through a matrix dimension estimation model, obtains the EEG feature sequence by performing eigenvalue calculation processing on the difference matrix, suppresses EEG noise and interference, and realizes the accurate extraction of cognitive impairment information reflected in the EEG.

[0068] The present invention establishes a set of time-frequency statistical feature quantities for skin electrical signals, performs skin electrical feature calculation processing on the time-frequency statistical feature quantity set, and obtains a skin electrical feature sequence, thereby achieving accurate and efficient extraction of skin electrical cognitive impairment information. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a composition diagram of the device of the present invention;

[0070] Figure 2 It is a flow chart for implementing the method of the present invention. DETAILED DESCRIPTION

[0071] In order to better understand the content of the present invention, an embodiment is given here.

[0072] Figure 1 It is a composition diagram of the device of the present invention; Figure 2 It is a flow chart for implementing the method of the present invention.

[0073] In order to solve the problem of how to organically integrate a brain-computer interface and an electric skin bracelet to construct a multimodal cognitive screening method and device, thereby effectively improving the accuracy and comprehensiveness of cognitive screening, the present invention discloses a multimodal cognitive screening method and device based on a brain-computer interface and an electric skin bracelet.

[0074] In a first aspect of the embodiment of the present application, a multimodal cognitive screening device based on a brain-computer interface and an electrodermal bracelet is disclosed, comprising: a brain-computer interface module, an electrodermal bracelet, a signal feature extraction module, and a cognitive assessment module;

[0075] The brain-computer interface module is used to collect and obtain the EEG signal sequence of the user when performing cognitive testing;

[0076] The electrodermal bracelet is used to collect and obtain the user's electrodermal signal sequence when taking cognitive tests;

[0077] The signal feature extraction module is used to perform feature extraction processing on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence;

[0078] The cognitive assessment module is used to perform cognitive impairment assessment processing on the EEG feature sequence, the electrodermal feature sequence, the electrodermal signal sequence and the EEG signal sequence to obtain a cognitive impairment assessment result;

[0079] The signal feature extraction module is used to perform feature extraction processing on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence, including:

[0080] The signal feature extraction module preprocesses the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence;

[0081] Performing feature calculation processing on the preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence respectively to obtain an EEG feature sequence and a skin electrical feature sequence;

[0082] The signal feature extraction module preprocesses the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence, including:

[0083] The signal feature extraction module performs data cleaning processing on the collected EEG signal sequence and skin electrical signal sequence to obtain a cleaned EEG signal sequence and a cleaned skin electrical signal sequence;

[0084] Performing data normalization processing on the cleaned EEG signal sequence and the cleaned skin electrical signal sequence respectively to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence;

[0085] The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence, including:

[0086] Obtain standard EEG signal sequence;

[0087] Subtracting the preprocessed EEG signal sequence from the standard EEG signal sequence to obtain a difference signal sequence;

[0088] Obtaining a set of statistical characteristic values ​​of the difference signal sequence by statistics; the set of statistical characteristic values ​​includes a median value, a mode value, a range value, a variance value and a range value;

[0089] Using a matrix dimension estimation model, the length and statistical characteristic value set of the difference signal sequence are calculated and processed to obtain a matrix row dimension value and a matrix column dimension value;

[0090] Based on the matrix row dimension value and the matrix column dimension value, using the difference signal sequence, construct a difference matrix;

[0091] Performing eigenvalue calculation processing on the difference matrix to obtain an EEG feature sequence;

[0092] Performing time-frequency feature statistical processing on the preprocessed skin electrical signal sequence to obtain a set of time-frequency statistical feature quantities; the set of time-frequency statistical feature quantities includes mean, variance, average rise time, harmonic mean, fluidity value, and average derivative;

[0093] Performing skin electrical feature calculation processing on the time-frequency statistical feature quantity set to obtain a skin electrical feature sequence;

[0094] The expression of the matrix dimension estimation model is:

[0095]

[0096] Wherein, κ is the median value, λ is the mode value, α is the variance value, β is the range value, N is the length of the difference signal sequence, L1 and L2 are the matrix row dimension value and the matrix column dimension value, respectively. Indicates rounding up. Indicates rounding down;

[0097] The step of performing eigenvalue calculation on the difference matrix to obtain an EEG feature sequence includes:

[0098] The difference matrix is ​​decomposed to obtain a feature matrix; the calculation expression of the decomposition process is:

[0099] Y=UAV,

[0100] Among them, U is the left decomposition matrix, Y is the difference matrix, A is the feature matrix, V is the right decomposition matrix, U and V are both orthogonal matrices, and A is a diagonal matrix;

[0101] Extracting the diagonal elements of the feature matrix to obtain a feature vector;

[0102] Performing linear fitting processing on the elements and element sequence values ​​of the feature vector to obtain a feature estimation polynomial;

[0103] For each row vector of the difference matrix, calculate the average value u0 of the row vector;

[0104] For each element in the row vector, the absolute value of the difference between the element and the average value u0 is calculated, and the absolute value of the difference is determined as the offset of the element;

[0105] For each row vector, find the element with the maximum offset, and determine the sequence number of the element in the row vector, which is the deviation sequence number of the row vector;

[0106] Using the characteristic estimation polynomial, the deviation sequence value of each row vector is calculated and processed to obtain the EEG characteristic value corresponding to each row vector;

[0107] Using the EEG eigenvalues ​​corresponding to all row vectors, an EEG feature sequence is constructed;

[0108] The step of performing skin electrical feature calculation processing on the time-frequency statistical feature quantity set to obtain a skin electrical feature sequence includes:

[0109] Using the mean and variance, performing a first feature calculation process on the preprocessed skin electrical signal sequence to obtain a first skin electrical feature value H1;

[0110] Performing a second characteristic calculation process on the average rise time and the harmonic mean to obtain a second skin electrical characteristic value H2;

[0111] Performing a third characteristic calculation process on the fluidity value, the average derivative, the mean value and the preprocessed skin electrical signal sequence to obtain a third skin electrical characteristic value H3;

[0112] Constructing a skin electrical feature sequence using the first skin electrical feature, the second skin electrical feature and the third skin electrical feature;

[0113] The expression of the first feature calculation process is:

[0114]

[0115] Among them, r i is the i-th element of the preprocessed skin electrical signal sequence, ω represents the variance, τ represents the mean, and N0 represents the number of elements contained in the preprocessed skin electrical signal sequence;

[0116] The expression of the second feature calculation process is:

[0117]

[0118] Among them, μ represents the average rise time, and v represents the harmonic mean;

[0119] The expression of the third feature calculation process is:

[0120]

[0121] Among them, φ represents the liquidity value, ρ represents the average derivative;

[0122] The performing cognitive impairment assessment processing on the EEG feature sequence, the electrodermal feature sequence, the electrodermal signal sequence and the EEG signal sequence to obtain a cognitive impairment assessment result includes:

[0123] The median value a1 and the mode value a2 of the skin electrical signal sequence are obtained by statistics;

[0124] Performing electroencephalogram cognitive impairment assessment calculation processing on the difference matrix of the electroencephalogram signal sequence and the electroencephalogram feature sequence to obtain a first assessment value;

[0125] The expression for the EEG cognitive impairment assessment calculation process is:

[0126]

[0127] Among them, T i () represents the i-th term of the first kind Chebyshev polynomial, p i represents the i-th element of the EEG feature sequence, q i represents the mean of the i-th row vector of the difference matrix, and ne represents the first evaluation value;

[0128] Performing skin electrical cognitive impairment assessment calculation processing on the median value a1 and the mode value a2 of the skin electrical signal sequence and the skin electrical feature sequence to obtain a second assessment value;

[0129] The expression for the calculation process of the electrodermal cognitive impairment assessment is:

[0130]

[0131] Wherein, L1(), L2(), L3() represent the first-order Laguerre polynomial, the second-order Laguerre polynomial, and the third-order Laguerre polynomial, respectively, and np represents the second evaluation value;

[0132] Performing a weighted summation on the first evaluation value and the second evaluation value to obtain a cognitive impairment evaluation result;

[0133] The cognitive impairment assessment result is used to characterize the severity of the user's cognitive impairment. The larger the value, the more severe the user's cognitive impairment.

[0134] The weight value used for weighted summing of the first evaluation value and the second evaluation value may be 0.2 and 0.3, or 0.4 and 0.5;

[0135] The length of the difference signal sequence is the number of elements contained in the difference signal sequence;

[0136] When L1 multiplied by L2 is less than N, taking the first L1 multiplied by L2 elements of the difference signal sequence to construct a difference matrix;

[0137] The average rise time is the average rise time of all amplitudes of the preprocessed skin electrical signal sequence;

[0138] The harmonic mean is the mean of all harmonic frequencies of the FFT sequence of the preprocessed skin electrical signal sequence;

[0139] The fluidity value is the square root of the first-order derivative variance of the preprocessed skin electrical signal sequence divided by the variance of the signal;

[0140] The average derivative is the average value of the derivative of each point of the preprocessed skin electrical signal sequence;

[0141] The decomposition process can be implemented by using a matrix singular value decomposition algorithm.

[0142] The linear fitting process uses the characteristic vector element serial number value Ix as a known independent variable and the characteristic vector element value as a known dependent variable, and uses the known independent variable and the known dependent variable to construct a curve to be approximated, and uses a function approximation method to perform curve fitting on the curve to be approximated to obtain a characteristic estimation polynomial.

[0143] The feature vector is represented as I a , I a =[λ1,λ2,…,λ N1 ], N1 is the number of elements contained in the feature vector; the curve fitting of the curve to be approximated by the function approximation method can adopt the best consistent linear approximation method. The feature estimation polynomial f(Ix) is expressed as:

[0144] f(Ix)=α P1 (Ix) P1 +α P1-1 (Ix) P1-1 +…+α2(Ix) 2 +α1(Ix)+α0,

[0145] Wherein, P1 is the order of the feature estimation polynomial f(Ix), α0, α1, α2, …, α P1 estimating coefficients of a polynomial f(Ix) for the feature;

[0146] The skin electrical bracelet comprises a bracelet body and a skin electrical signal acquisition device arranged on the bracelet; the skin electrical signal acquisition device comprises a skin electrical sensor and a central processor;

[0147] The data cleaning process includes filling missing values, smoothing noise data, and smoothing or deleting outliers; the smoothed noise data is obtained by first determining the noise data, and then smoothing the noise data according to the data before and after the noise data; the noise data is a value whose value is less than the detection sensitivity of the sensor of the observed data, or greater than the measurement upper limit of the sensor of the observed data. The outlier point can be determined by the Kalman filter method. The filling value of the missing value can be determined by averaging the measured values ​​within a certain sampling interval before and after the missing value.

[0148] The data normalization process is to map data of different value ranges into a specified value range, and its mathematical expression is:

[0149]

[0150] Among them, t is a data value in the data set, t max is the maximum value of all data in the data set, t min is the minimum value of all data in the data set, t 归一化 is the value of data t after normalization;

[0151] In a second aspect of the embodiment of the present application, a multimodal cognitive screening method based on a brain-computer interface and an electrodermal bracelet is disclosed, comprising:

[0152] Using the brain-computer interface module, the brain electrical signal sequence of the user when taking a cognitive test is collected;

[0153] Using the electrodermal bracelet, a skin electrical signal sequence of the user when taking a cognitive test is collected;

[0154] Using the signal feature extraction module, feature extraction is performed on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence;

[0155] The cognitive assessment module is used to perform cognitive impairment assessment processing on the EEG feature sequence, the electrodermal feature sequence, the electrodermal signal sequence and the EEG signal sequence to obtain a cognitive impairment assessment result.

[0156] The signal feature extraction module is used to perform feature extraction processing on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence, including:

[0157] Using the signal feature extraction module, preprocessing the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence;

[0158] The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence.

[0159] The method of using the signal feature extraction module to preprocess the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence includes:

[0160] Using the signal feature extraction module, the collected EEG signal sequence and skin electrical signal sequence are respectively subjected to data cleaning processing to obtain a cleaned EEG signal sequence and a cleaned skin electrical signal sequence;

[0161] The cleaned EEG signal sequence and the cleaned skin electrical signal sequence are respectively subjected to data normalization processing to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence.

[0162] The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence, including:

[0163] Obtain standard EEG signal sequence;

[0164] Subtracting the preprocessed EEG signal sequence from the standard EEG signal sequence to obtain a difference signal sequence;

[0165] Obtaining a set of statistical characteristic values ​​of the difference signal sequence by statistics; the set of statistical characteristic values ​​includes a median value, a mode value, a range value and a variance value;

[0166] Using a matrix dimension estimation model, the length and statistical characteristic value set of the difference signal sequence are calculated and processed to obtain a matrix row dimension value and a matrix column dimension value;

[0167] Based on the matrix row dimension value and the matrix column dimension value, using the difference signal sequence, construct a difference matrix;

[0168] Performing eigenvalue calculation processing on the difference matrix to obtain an EEG feature sequence;

[0169] Performing time-frequency feature statistical processing on the preprocessed skin electrical signal sequence to obtain a set of time-frequency statistical feature quantities; the set of time-frequency statistical feature quantities includes mean, variance, average rise time, harmonic mean, fluidity value, and average derivative;

[0170] The electrical skin feature calculation process is performed on the time-frequency statistical feature quantity set to obtain an electrical skin feature sequence.

[0171] The expression of the matrix dimension estimation model is:

[0172]

[0173]

[0174] Wherein, κ is the median value, λ is the mode value, α is the variance value, β is the range value, N is the length of the difference signal sequence, L1 and L2 are the matrix row dimension value and the matrix column dimension value, respectively. Indicates rounding up. Indicates rounding down.

[0175] The step of performing eigenvalue calculation on the difference matrix to obtain an EEG feature sequence includes:

[0176] The difference matrix is ​​decomposed to obtain a feature matrix; the calculation expression of the decomposition process is:

[0177] Y=UAV,

[0178] Among them, U is the left decomposition matrix, Y is the difference matrix, A is the feature matrix, V is the right decomposition matrix, U and V are both orthogonal matrices, and A is a diagonal matrix;

[0179] Extracting the diagonal elements of the feature matrix to obtain a feature vector;

[0180] Performing linear fitting processing on the elements and element sequence values ​​of the feature vector to obtain a feature estimation polynomial;

[0181] For each row vector of the difference matrix, the average value of the row vector is calculated; for each element in the row vector, the absolute value of the difference between the row vector and the average value is calculated, and the absolute value of the difference is determined as the offset of the element;

[0182] For each row vector, find the element with the maximum offset, and determine the sequence number of the element in the row vector, which is the deviation sequence number of the row vector;

[0183] Using the characteristic estimation polynomial, the deviation sequence value of each row vector is calculated and processed to obtain the EEG characteristic value corresponding to each row vector;

[0184] The EEG feature sequence is constructed using the EEG feature values ​​corresponding to all row vectors.

[0185] The step of performing skin electrical feature calculation processing on the time-frequency statistical feature quantity set to obtain a skin electrical feature sequence includes:

[0186] Using the mean and variance, performing a first feature calculation process on the preprocessed skin electrical signal sequence to obtain a first skin electrical feature value H1;

[0187] Performing a second characteristic calculation process on the average rise time and the harmonic mean to obtain a second skin electrical characteristic value H2;

[0188] Performing a third characteristic calculation process on the fluidity value, the average derivative, the mean value and the preprocessed skin electrical signal sequence to obtain a third skin electrical characteristic value H3;

[0189] Constructing a skin electrical feature sequence using the first skin electrical feature, the second skin electrical feature and the third skin electrical feature;

[0190] The expression of the first feature calculation process is:

[0191]

[0192] Among them, r i is the i-th element of the preprocessed skin electrical signal sequence, ω represents the variance, τ represents the mean, and N0 represents the number of elements contained in the preprocessed skin electrical signal sequence;

[0193] The expression of the second feature calculation process is:

[0194]

[0195] Where, μ represents the average rise time, and ν represents the harmonic mean;

[0196] The expression of the third feature calculation process is:

[0197]

[0198] Among them, φ represents the liquidity value and ρ represents the average derivative.

[0199] The performing cognitive impairment assessment processing on the EEG feature sequence, the electrodermal feature sequence, the electrodermal signal sequence and the EEG signal sequence to obtain a cognitive impairment assessment result includes:

[0200] The median value a1 and the mode value a2 of the skin electrical signal sequence are obtained by statistics;

[0201] Performing electroencephalogram cognitive impairment assessment calculation processing on the difference matrix of the electroencephalogram signal sequence and the electroencephalogram feature sequence to obtain a first assessment value;

[0202] The expression for the EEG cognitive impairment assessment calculation process is:

[0203]

[0204] Among them, T i () represents the i-th term of the first kind Chebyshev polynomial, p i represents the i-th element of the EEG feature sequence, qi represents the mean of the i-th row vector of the difference matrix, and ne represents the first evaluation value;

[0205] Performing skin electrical cognitive impairment assessment calculation processing on the median value a1 and the mode value a2 of the skin electrical signal sequence and the skin electrical feature sequence to obtain a second assessment value;

[0206] The expression for the calculation process of the electrodermal cognitive impairment assessment is:

[0207]

[0208] Wherein, L1(), L2(), L3() represent the first-order Laguerre polynomial, the second-order Laguerre polynomial, and the third-order Laguerre polynomial, respectively, and np represents the second evaluation value;

[0209] Performing a weighted summation on the first evaluation value and the second evaluation value to obtain a cognitive impairment evaluation result;

[0210] The cognitive impairment assessment result is used to characterize the severity of the user's cognitive impairment. The larger the value, the more severe the user's cognitive impairment.

[0211] The weight value used for weighted summing of the first evaluation value and the second evaluation value may be 0.2 and 0.3, or 0.4 and 0.5;

[0212] The length of the difference signal sequence is the number of elements contained in the difference signal sequence;

[0213] When L1 multiplied by L2 is less than N, taking the first L1 multiplied by L2 elements of the difference signal sequence to construct a difference matrix;

[0214] The average rise time is the average rise time of all amplitudes of the preprocessed skin electrical signal sequence;

[0215] The harmonic mean is the mean of all harmonic frequencies of the FFT sequence of the preprocessed skin electrical signal sequence;

[0216] The fluidity value is the square root of the first-order derivative variance of the preprocessed skin electrical signal sequence divided by the variance of the signal;

[0217] The average derivative is the average value of the derivative of each point of the preprocessed skin electrical signal sequence;

[0218] The decomposition process can be implemented by using a matrix singular value decomposition algorithm.

[0219] The linear fitting process uses the characteristic vector element serial number value Ix as a known independent variable and the characteristic vector element value as a known dependent variable, and uses the known independent variable and the known dependent variable to construct a curve to be approximated, and uses a function approximation method to perform curve fitting on the curve to be approximated to obtain a characteristic estimation polynomial.

[0220] The feature vector is represented as I a , I a =[λ1,λ2,…,λ N1 ], N1 is the number of elements contained in the feature vector; the curve fitting of the curve to be approximated by the function approximation method can adopt the best consistent linear approximation method. The feature estimation polynomial f(Ix) is expressed as:

[0221] f(Ix)=α P1 (Ix) P1 +α P1-1 (Ix) P1-1 +…+α2(Ix) 2 +α1(Ix)+α0,

[0222] Wherein, P1 is the order of the feature estimation polynomial f(Ix), α0, α1, α2, …, α P1 Estimate the coefficients of the polynomial f(Ix) for the feature.

[0223] The cognitive test may be a memory test, an attention test, a digital memory, a graphic matching test, a logical reasoning test, etc.

[0224] The brain-computer interface module can be implemented using a brain-computer interface device.

[0225] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A multimodal cognitive screening device based on a brain-computer interface and an electrodermal bracelet, characterized in that: include: Brain-computer interface module, electrodermal bracelet, signal feature extraction module, cognitive assessment module; The brain-computer interface module is used to collect and obtain the EEG signal sequence of the user when performing cognitive testing; The electrodermal bracelet is used to collect and obtain the user's electrodermal signal sequence when taking cognitive tests; The signal feature extraction module is connected to the brain-computer interface module, the skin electrode bracelet and the cognitive assessment module respectively, and is used to perform feature extraction processing on the collected EEG signal sequence and skin electrode signal sequence to obtain an EEG feature sequence and a skin electrode feature sequence; The cognitive assessment module is used to perform cognitive impairment assessment processing on the EEG feature sequence, the electrodermal feature sequence, the electrodermal signal sequence and the EEG signal sequence to obtain a cognitive impairment assessment result.

2. The multimodal cognitive screening device based on brain-computer interface and electrodermal bracelet as claimed in claim 1, characterized in that: The signal feature extraction module is used to perform feature extraction processing on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence, including: The signal feature extraction module preprocesses the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence; The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence.

3. The multimodal cognitive screening device based on brain-computer interface and electrodermal bracelet as claimed in claim 2, characterized in that: The signal feature extraction module preprocesses the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence, including: The signal feature extraction module performs data cleaning processing on the collected EEG signal sequence and skin electrical signal sequence to obtain a cleaned EEG signal sequence and a cleaned skin electrical signal sequence; The cleaned EEG signal sequence and the cleaned skin electrical signal sequence are respectively subjected to data normalization processing to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence.

4. A multimodal cognitive screening method based on brain-computer interface and electrodermal bracelet, characterized in that: The method is implemented by using a multimodal cognitive screening device based on a brain-computer interface and an electrodermal bracelet according to any one of claims 1 to 3, comprising: Using the brain-computer interface module, the brain electrical signal sequence of the user when taking a cognitive test is collected; Using the electrodermal bracelet, a skin electrical signal sequence of the user when taking a cognitive test is collected; Using the signal feature extraction module, feature extraction is performed on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence; The cognitive assessment module is used to perform cognitive impairment assessment processing on the EEG feature sequence, the electrodermal feature sequence, the electrodermal signal sequence and the EEG signal sequence to obtain a cognitive impairment assessment result.

5. The multimodal cognitive screening method based on brain-computer interface and electrodermal bracelet as claimed in claim 4, characterized in that: The signal feature extraction module is used to perform feature extraction processing on the collected EEG signal sequence and skin electrical signal sequence to obtain an EEG feature sequence and a skin electrical feature sequence, including: Using the signal feature extraction module, preprocessing the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence; The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence.

6. The multimodal cognitive screening method based on brain-computer interface and electrodermal bracelet as claimed in claim 5, characterized in that: The method of using the signal feature extraction module to preprocess the collected EEG signal sequence and skin electrical signal sequence to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence includes: Using the signal feature extraction module, the collected EEG signal sequence and skin electrical signal sequence are respectively subjected to data cleaning processing to obtain a cleaned EEG signal sequence and a cleaned skin electrical signal sequence; The cleaned EEG signal sequence and the cleaned skin electrical signal sequence are respectively subjected to data normalization processing to obtain a preprocessed EEG signal sequence and a preprocessed skin electrical signal sequence.

7. The multimodal cognitive screening method based on brain-computer interface and electrodermal bracelet as claimed in claim 6, characterized in that: The preprocessed EEG signal sequence and the preprocessed skin electrical signal sequence are respectively subjected to feature calculation processing to obtain an EEG feature sequence and a skin electrical feature sequence, including: Obtain standard EEG signal sequence; Subtracting the preprocessed EEG signal sequence from the standard EEG signal sequence to obtain a difference signal sequence; Obtaining a set of statistical characteristic values ​​of the difference signal sequence by statistics; the set of statistical characteristic values ​​includes a median value, a mode value, a range value and a variance value; Using a matrix dimension estimation model, the length and statistical characteristic value set of the difference signal sequence are calculated and processed to obtain a matrix row dimension value and a matrix column dimension value; Based on the matrix row dimension value and the matrix column dimension value, using the difference signal sequence, construct a difference matrix; Performing eigenvalue calculation processing on the difference matrix to obtain an EEG feature sequence; Performing time-frequency feature statistical processing on the preprocessed skin electrical signal sequence to obtain a set of time-frequency statistical feature quantities; the set of time-frequency statistical feature quantities includes mean, variance, average rise time, harmonic mean, fluidity value, and average derivative; The electrical skin feature calculation process is performed on the time-frequency statistical feature quantity set to obtain an electrical skin feature sequence.

8. The multimodal cognitive screening method based on brain-computer interface and electrodermal bracelet as claimed in claim 7, characterized in that: The expression of the matrix dimension estimation model is: Wherein, κ is the median value, λ is the mode value, α is the variance value, β is the range value, N is the length of the difference signal sequence, L1 and L2 are the matrix row dimension value and the matrix column dimension value, respectively. Indicates rounding up. Indicates rounding down.

9. The multimodal cognitive screening method based on brain-computer interface and electrodermal bracelet as claimed in claim 7, characterized in that: The step of performing eigenvalue calculation on the difference matrix to obtain an EEG feature sequence includes: The difference matrix is ​​decomposed to obtain a feature matrix; the calculation expression of the decomposition process is: Y=UAV, Among them, U is the left decomposition matrix, Y is the difference matrix, A is the feature matrix, V is the right decomposition matrix, U and V are both orthogonal matrices, and A is a diagonal matrix; Extracting the diagonal elements of the feature matrix to obtain a feature vector; Performing linear fitting processing on the elements and element sequence values ​​of the feature vector to obtain a feature estimation polynomial; For each row vector of the difference matrix, the average value of the row vector is calculated; for each element in the row vector, the absolute value of the difference between the row vector and the average value is calculated, and the absolute value of the difference is determined as the offset of the element; For each row vector, find the element with the maximum offset, and determine the sequence number of the element in the row vector, which is the deviation sequence number of the row vector; Using the characteristic estimation polynomial, the deviation sequence value of each row vector is calculated and processed to obtain the EEG characteristic value corresponding to each row vector; The EEG feature sequence is constructed using the EEG feature values ​​corresponding to all row vectors.

10. The multimodal cognitive screening method based on brain-computer interface and electrodermal bracelet as claimed in claim 7, characterized in that: The step of performing skin electrical feature calculation processing on the time-frequency statistical feature quantity set to obtain a skin electrical feature sequence includes: Using the mean and variance, performing a first feature calculation process on the preprocessed skin electrical signal sequence to obtain a first skin electrical feature value H1; Performing a second characteristic calculation process on the average rise time and the harmonic mean to obtain a second skin electrical characteristic value H2; Performing a third characteristic calculation process on the fluidity value, the average derivative, the mean value and the preprocessed skin electrical signal sequence to obtain a third skin electrical characteristic value H3; Constructing a skin electrical feature sequence using the first skin electrical feature, the second skin electrical feature and the third skin electrical feature; The expression of the first feature calculation process is: Among them, r i is the i-th element of the preprocessed skin electrical signal sequence, ω represents the variance, τ represents the mean, and N0 represents the number of elements contained in the preprocessed skin electrical signal sequence; The expression of the second feature calculation process is: Where, μ represents the average rise time, and ν represents the harmonic mean; The expression of the third feature calculation process is: Among them, φ represents the liquidity value and ρ represents the average derivative.

Images