Multi-modal cognitive screening method and apparatus based on brain-computer interface and skin conductance bracelet

Abstract

The application discloses a kind of based on brain-computer interface and multimodal cognitive screening method and device of skin electricity ring, the device includes: brain-computer interface module, skin electricity ring, signal feature extraction module, cognitive assessment module;The brain-computer interface module is used to collect the electroencephalogram sequence of user when carrying out cognitive test;The skin electricity ring is used to collect the electrodermal signal sequence of user when carrying out cognitive test;The signal feature extraction module is used to carry out feature extraction processing to the electroencephalogram sequence and electrodermal signal sequence collected, and obtains electroencephalogram feature sequence and skin electricity feature sequence;The cognitive assessment module is used to carry out cognitive impairment assessment processing to the electroencephalogram feature sequence, skin electricity feature sequence, electrodermal signal sequence and electroencephalogram sequence, and obtains cognitive impairment assessment result.

Description

Technical Field

[0001] This invention relates to the fields of brain-computer interfaces and brain cognition, specifically to a multimodal cognitive screening method and device based on a brain-computer interface and a magnetic field wristband. Background Technology

[0002] Cognitive impairment, including but not limited to dementia and mild cognitive impairment, severely affects an individual's daily life and work abilities. Early and accurate cognitive screening is crucial for intervention and early rehabilitation.

[0003] Currently, common cognitive screening methods mainly rely on questionnaires and neuropsychological tests, which suffer from problems such as high subjectivity and limited accuracy. Brain-computer interface (BCI) technology, as an emerging neurotechnology, can directly acquire neural activity signals from the brain, providing a new approach to assessing cognitive function. However, in practical applications, BCI technology alone may be affected by various environmental factors and interference signals, leading to insufficient stability and reliability of cognitive screening results.

[0004] Skin conductance wristbands can monitor skin conductance activity in real time, reflecting an individual's emotional and physiological arousal state, and also showing some correlation with cognitive function. How to organically integrate brain-computer interfaces and skin conductance wristbands to construct multimodal cognitive screening methods and devices, thereby effectively improving the accuracy and comprehensiveness of cognitive screening, is an urgent problem to be solved. Summary of the Invention

[0005] This invention primarily addresses the problem of how to organically integrate brain-computer interfaces and magnetic resonance imaging (MRI) wristbands to construct multimodal cognitive screening methods and devices, thereby effectively improving the accuracy and comprehensiveness of cognitive screening. This invention discloses a multimodal cognitive screening method and device based on brain-computer interfaces and MRI wristbands.

[0006] In a first aspect, this application discloses a multimodal cognitive screening device based on a brain-computer interface and a magnetic field bracelet, comprising: a brain-computer interface module, a magnetic field bracelet, a signal feature extraction module, and a cognitive assessment module;

[0007] The brain-computer interface module is used to acquire the EEG signal sequence of the user during cognitive testing;

[0008] The skin conductance wristband is used to collect the skin conductance signal sequence of the user during cognitive testing;

[0009] The signal feature extraction module is connected to the brain-computer interface module, the electrodermal wristband and the cognitive assessment module respectively, and is used to perform feature extraction processing on the acquired electroencephalogram (EEG) signal sequence and electrodermal (ED) signal sequence to obtain EEG feature sequence and EEG feature sequence.

[0010] The cognitive assessment module is used to perform cognitive impairment assessment processing on the electroencephalogram (EEG) feature sequence, skin conductance feature sequence, skin conductance signal sequence, and EEG signal sequence to obtain cognitive impairment assessment results.

[0011] The signal feature extraction module is used to perform feature extraction processing on the acquired electroencephalogram (EEG) signal sequence and electrodermal signal sequence to obtain EEG feature sequences and electrodermal signal feature sequences, including:

[0012] The signal feature extraction module preprocesses the acquired EEG signal sequence and EDS signal sequence to obtain preprocessed EEG signal sequence and preprocessed EDS signal sequence.

[0013] The preprocessed EEG signal sequence and the preprocessed ESC signal sequence are subjected to feature calculation processing to obtain EEG feature sequence and ESC feature sequence, respectively.

[0014] The signal feature extraction module preprocesses the acquired EEG signal sequence and ESC signal sequence to obtain preprocessed EEG signal sequence and preprocessed ESC signal sequence, including:

[0015] The signal feature extraction module performs data cleaning processing on the acquired EEG signal sequence and EDS signal sequence to obtain cleaned EEG signal sequence and cleaned EDS signal sequence.

[0016] Data normalization was performed on the cleaned EEG signal sequence and the cleaned EDS signal sequence to obtain the preprocessed EEG signal sequence and the preprocessed EDS signal sequence.

[0017] A second aspect of this invention discloses a multimodal cognitive screening method based on a brain-computer interface and a magnetic resonance imaging (MRI) wristband, implemented using the aforementioned multimodal cognitive screening device based on a brain-computer interface and a MRI wristband, comprising:

[0018] Using the brain-computer interface module, the EEG signal sequence of the user during cognitive testing is acquired;

[0019] The skin conductance bracelet was used to collect the skin conductance signal sequence of the user during a cognitive test;

[0020] The signal feature extraction module is used to perform feature extraction processing on the acquired EEG signal sequence and ESC signal sequence to obtain EEG feature sequence and ESC feature sequence;

[0021] Using the cognitive assessment module, the electroencephalogram (EEG) feature sequences, skin conductance feature sequences, skin conductance signal sequences, and EEG signal sequences are processed to assess cognitive impairment, and cognitive impairment assessment results are obtained.

[0022] The signal feature extraction module is used to perform feature extraction processing on the acquired EEG signal sequence and skin conductance signal sequence to obtain EEG feature sequence and skin conductance feature sequence, including:

[0023] The signal feature extraction module is used to preprocess the acquired EEG signal sequence and EDS signal sequence to obtain preprocessed EEG signal sequence and preprocessed EDS signal sequence.

[0024] The preprocessed EEG signal sequence and the preprocessed ESC signal sequence are subjected to feature calculation processing to obtain EEG feature sequence and ESC feature sequence, respectively.

[0025] The process of using the signal feature extraction module to preprocess the acquired EEG signal sequence and ESC signal sequence to obtain preprocessed EEG signal sequence and preprocessed ESC signal sequence includes:

[0026] Using the signal feature extraction module, the acquired EEG signal sequence and EDS signal sequence are cleaned to obtain cleaned EEG signal sequence and cleaned EDS signal sequence, respectively.

[0027] Data normalization was performed on the cleaned EEG signal sequence and the cleaned EDS signal sequence to obtain the preprocessed EEG signal sequence and the preprocessed EDS signal sequence.

[0028] The step involves performing feature calculations on the preprocessed EEG signal sequence and the preprocessed ESC signal sequence to obtain EEG feature sequences and ESC feature sequences, respectively, including:

[0029] Obtain standard EEG signal sequences;

[0030] The preprocessed EEG signal sequence is subtracted from the standard EEG signal sequence to obtain the difference signal sequence;

[0031] The statistical feature set of the differential signal sequence is obtained by statistical analysis; the statistical feature set includes the median, mode, range, and variance.

[0032] Using a matrix dimension estimation model, the length and statistical feature set of the differential signal sequence are calculated to obtain the matrix row dimension and matrix column dimension.

[0033] Based on the matrix row dimension values ​​and matrix column dimension values, a difference matrix is ​​constructed using the difference signal sequence;

[0034] The difference matrix is ​​processed by eigenvalue calculation to obtain the EEG feature sequence;

[0035] The preprocessed electrodermal signal sequence is subjected to time-frequency statistical processing to obtain a set of time-frequency statistical features; the set of time-frequency statistical features includes mean, variance, average rise time, harmonic mean, fluidity value, and average derivative;

[0036] The skin conductance feature is calculated by performing skin conductance feature calculation on the set of time-frequency statistical features to obtain the skin conductance feature sequence.

[0037] The expression for the matrix dimension estimation model is:

[0038]

[0039] Where κ is the median, λ is the mode, α is the variance, β is the range, N is the length of the difference signal sequence, and L1 and L2 are the row and column dimensions of the matrix, respectively. Indicates rounding up. This indicates rounding down to the nearest integer.

[0040] The step of performing eigenvalue calculation on the difference matrix to obtain the EEG feature sequence includes:

[0041] The difference matrix is ​​decomposed to obtain the feature matrix; the calculation expression for the decomposition is as follows:

[0042] Y = UAV,

[0043] Where U is the left decomposition matrix, Y is the difference matrix, A is the characteristic matrix, V is the right decomposition matrix, U and V are both orthogonal matrices, and A is a diagonal matrix;

[0044] Extract the diagonal elements of the feature matrix to obtain the feature vector;

[0045] The elements and element index values ​​of the feature vector are subjected to linear fitting to obtain the feature estimation polynomial;

[0046] For each row vector of the difference matrix, the average value of the row vector is calculated; for each element in the row vector, the absolute value of the difference between it and the average value is calculated, and the absolute value of the difference is determined as the offset of the element.

[0047] For each row vector, find the element with the largest offset, and determine the index value of the element in the row vector as the offset index value of the row vector;

[0048] By using the feature estimation polynomial, the deviation index value of each row vector is calculated and processed to obtain the EEG feature value corresponding to each row vector;

[0049] By utilizing the EEG feature values ​​corresponding to all row vectors, an EEG feature sequence is constructed.

[0050] The step of performing skin conductance feature calculation on the time-frequency statistical feature set to obtain a skin conductance feature sequence includes:

[0051] Using the mean and variance, the preprocessed electrodermal signal sequence is subjected to a first feature calculation to obtain the first electrodermal feature quantity H1;

[0052] The average rise time and harmonic mean are processed by a second feature calculation to obtain the second skin conductance characteristic quantity H2.

[0053] The fluidity value, average derivative, mean, and preprocessed electrodermal signal sequence are subjected to a third feature calculation to obtain the third electrodermal feature quantity H3;

[0054] Using the first, second, and third skin conductance features, a skin conductance feature sequence is constructed.

[0055] The expression for the first feature calculation is:

[0056]

[0057] Where, r i Let ω be the i-th element of the preprocessed EDS signal sequence, τ be the variance, and N0 be the mean.

[0058] The expression for the second feature calculation is:

[0059]

[0060] Where μ represents the average rise time and ν represents the harmonic mean;

[0061] The expression for the third feature calculation process is:

[0062]

[0063] Where φ represents the liquidity value and ρ represents the average derivative.

[0064] The beneficial effects of this invention are as follows:

[0065] This invention introduces brain-computer interfaces and electrophysiological wristbands into the field of cognitive impairment screening. By organically fusing the signals acquired by the brain-computer interface and the electrophysiological wristband, a multimodal cognitive screening method and device are constructed. This achieves fusion at both the feature level and the signal level, improving the effectiveness of abnormal signal extraction and ensuring the accuracy of cognitive impairment screening.

[0066] This invention, after acquiring electroencephalogram (EEG) and electrodermal signal (EDS) signals, first extracts features from both types of signals to obtain EEG feature sequences and EDS feature sequences. Then, it uses these EEG feature sequences, EDS feature sequences, EDS signal sequences, and EEG signal sequences to perform cognitive impairment assessment processing, obtaining the cognitive impairment assessment result. This step-by-step approach reduces the complexity of cognitive impairment assessment, distributes the assessment process across two hardware platforms, and improves assessment efficiency.

[0067] In the process of extracting EEG feature sequences, this invention ensures the accuracy of the constructed data matrix by using a matrix dimension estimation model. By performing eigenvalue calculation on the difference matrix, the EEG feature sequences are obtained, thereby suppressing EEG noise and interference and achieving accurate extraction of cognitive impairment information from EEG.

[0068] This invention establishes a set of time-frequency statistical features for the electrodermal signal, performs electrodermal feature calculation on the set of time-frequency statistical features, and obtains an electrodermal feature sequence, thereby achieving accurate and efficient extraction of information on electrodermal cognitive impairment. Attached Figure Description

[0069] Figure 1 This is a diagram showing the composition of the device of the present invention;

[0070] Figure 2 This is a flowchart illustrating the implementation of the method of the present invention. Detailed Implementation

[0071] To better understand the content of this invention, an embodiment is provided here.

[0072] Figure 1 This is a diagram showing the composition of the device of the present invention; Figure 2 This is a flowchart illustrating the implementation of the method of the present invention.

[0073] To address the problem of how to organically integrate brain-computer interfaces and magnetic resonance imaging (MRI) wristbands to construct multimodal cognitive screening methods and devices, thereby effectively improving the accuracy and comprehensiveness of cognitive screening, this invention discloses a multimodal cognitive screening method and device based on brain-computer interfaces and MRI wristbands.

[0074] In a first aspect, this application discloses a multimodal cognitive screening device based on a brain-computer interface and a magnetic field bracelet, comprising: a brain-computer interface module, a magnetic field bracelet, a signal feature extraction module, and a cognitive assessment module;

[0075] The brain-computer interface module is used to acquire the EEG signal sequence of the user during cognitive testing;

[0076] The skin conductance wristband is used to collect the skin conductance signal sequence of the user during cognitive testing;

[0077] The signal feature extraction module is used to perform feature extraction processing on the acquired EEG signal sequence and ESC signal sequence to obtain EEG feature sequence and ESC feature sequence.

[0078] The cognitive assessment module is used to perform cognitive impairment assessment processing on the electroencephalogram (EEG) feature sequence, skin conductance feature sequence, skin conductance signal sequence, and EEG signal sequence to obtain cognitive impairment assessment results.

[0079] The signal feature extraction module is used to perform feature extraction processing on the acquired electroencephalogram (EEG) signal sequence and electrodermal signal sequence to obtain EEG feature sequences and electrodermal signal feature sequences, including:

[0080] The signal feature extraction module preprocesses the acquired EEG signal sequence and EDS signal sequence to obtain preprocessed EEG signal sequence and preprocessed EDS signal sequence.

[0081] The preprocessed EEG signal sequence and the preprocessed ESC signal sequence are subjected to feature calculation processing to obtain EEG feature sequence and ESC feature sequence, respectively;

[0082] The signal feature extraction module preprocesses the acquired EEG signal sequence and ESC signal sequence to obtain preprocessed EEG signal sequence and preprocessed ESC signal sequence, including:

[0083] The signal feature extraction module performs data cleaning processing on the acquired EEG signal sequence and EDS signal sequence to obtain cleaned EEG signal sequence and cleaned EDS signal sequence.

[0084] The cleaned EEG signal sequence and the cleaned EDS signal sequence were respectively subjected to data normalization to obtain the preprocessed EEG signal sequence and the preprocessed EDS signal sequence.

[0085] The step involves performing feature calculations on the preprocessed EEG signal sequence and the preprocessed ESC signal sequence to obtain EEG feature sequences and ESC feature sequences, respectively, including:

[0086] Obtain standard EEG signal sequences;

[0087] The preprocessed EEG signal sequence is subtracted from the standard EEG signal sequence to obtain the difference signal sequence;

[0088] The statistical feature set of the differential signal sequence is obtained by statistical analysis; the statistical feature set includes the median, mode, range, variance, and range.

[0089] Using a matrix dimension estimation model, the length and statistical feature set of the differential signal sequence are calculated to obtain the matrix row dimension and matrix column dimension.

[0090] Based on the matrix row dimension values ​​and matrix column dimension values, a difference matrix is ​​constructed using the difference signal sequence;

[0091] The difference matrix is ​​processed by eigenvalue calculation to obtain the EEG feature sequence;

[0092] The preprocessed electrodermal signal sequence is subjected to time-frequency statistical processing to obtain a set of time-frequency statistical features; the set of time-frequency statistical features includes mean, variance, average rise time, harmonic mean, fluidity value, and average derivative;

[0093] The skin conductance feature is calculated by performing skin conductance feature calculation on the set of time-frequency statistical features to obtain the skin conductance feature sequence;

[0094] The expression for the matrix dimension estimation model is:

[0095]

[0096] Where κ is the median, λ is the mode, α is the variance, β is the range, N is the length of the difference signal sequence, and L1 and L2 are the row and column dimensions of the matrix, respectively. Indicates rounding up. Indicates rounding down;

[0097] The step of performing eigenvalue calculation on the difference matrix to obtain the EEG feature sequence includes:

[0098] The difference matrix is ​​decomposed to obtain the feature matrix; the calculation expression for the decomposition is as follows:

[0099] Y = UAV,

[0100] Where U is the left decomposition matrix, Y is the difference matrix, A is the characteristic matrix, V is the right decomposition matrix, U and V are both orthogonal matrices, and A is a diagonal matrix;

[0101] Extract the diagonal elements of the feature matrix to obtain the feature vector;

[0102] The elements and element index values ​​of the feature vector are subjected to linear fitting to obtain the feature estimation polynomial;

[0103] For each row vector of the difference matrix, the average value u0 of the row vector is calculated;

[0104] For each element in the row vector, calculate the absolute value of the difference between it and the average value u0, and determine the absolute value of the difference as the offset of the element;

[0105] For each row vector, find the element with the largest offset, and determine the index value of the element in the row vector as the offset index value of the row vector;

[0106] By using the feature estimation polynomial, the deviation index value of each row vector is calculated and processed to obtain the EEG feature value corresponding to each row vector;

[0107] Using the EEG feature values ​​corresponding to all row vectors, an EEG feature sequence is constructed.

[0108] The step of performing skin conductance feature calculation on the time-frequency statistical feature set to obtain a skin conductance feature sequence includes:

[0109] Using the mean and variance, the preprocessed electrodermal signal sequence is subjected to a first feature calculation to obtain the first electrodermal feature quantity H1;

[0110] The average rise time and harmonic mean are processed by a second feature calculation to obtain the second skin conductance characteristic quantity H2.

[0111] The fluidity value, average derivative, mean, and preprocessed electrodermal signal sequence are subjected to a third feature calculation to obtain the third electrodermal feature quantity H3;

[0112] Using the first, second, and third skin conductance features, a skin conductance feature sequence is constructed.

[0113] The expression for the first feature calculation is:

[0114]

[0115] Where, r i Let ω be the i-th element of the preprocessed EDS signal sequence, τ be the variance, and N0 be the mean.

[0116] The expression for the second feature calculation is:

[0117]

[0118] Where μ represents the average rise time and v represents the harmonic mean;

[0119] The expression for the third feature calculation process is:

[0120]

[0121] Where φ represents the liquidity value and ρ represents the average derivative;

[0122] The cognitive impairment assessment is performed on the electroencephalogram (EEG) feature sequences, skin conductance feature sequences, skin conductance signal sequences, and EEG signal sequences to obtain cognitive impairment assessment results, including:

[0123] The median value a1 and mode value a2 of the electrodermal signal sequence were obtained statistically.

[0124] The difference matrix and EEG feature sequence of the EEG signal sequence are processed for EEG cognitive impairment assessment to obtain a first assessment value;

[0125] The expression for the calculation and processing of the EEG cognitive impairment assessment is as follows:

[0126]

[0127] Among them, T i () represents the i-th term of a Chebyshev polynomial of the first kind, p i q represents the i-th element of the EEG feature sequence. i Let represent the mean of the vector in the i-th row of the difference matrix, and ne represent the first evaluation value;

[0128] The median value a1 and mode value a2 of the electrodermal signal sequence and the electrodermal characteristic sequence are processed to calculate and calculate the electrodermal cognitive impairment assessment to obtain the second assessment value;

[0129] The expression for the calculation and processing of the skin conductance cognitive impairment assessment is as follows:

[0130]

[0131] Where L1(), L2(), and L3() represent first-order Laguerre polynomials, second-order Laguerre polynomials, and third-order Laguerre polynomials, respectively, and np represents the second evaluation value;

[0132] The first and second assessment values ​​are weighted and summed to obtain the cognitive impairment assessment result.

[0133] The cognitive impairment assessment results are used to characterize the severity of the user's cognitive impairment; the higher the value, the more severe the user's cognitive impairment.

[0134] The weight values ​​used to perform a weighted summation of the first and second evaluation values ​​can be 0.2 and 0.3, or 0.4 and 0.5.

[0135] The length of the difference signal sequence is the number of elements contained in the difference signal sequence;

[0136] When L1 multiplied by L2 is less than N, the first L1 multiplied by L2 elements of the difference signal sequence are taken to construct the difference matrix;

[0137] The average rise time is the average rise time of all amplitudes in the preprocessed electrodermal signal sequence.

[0138] The harmonic mean is the average of all harmonic frequencies of the FFT sequence of the preprocessed electrodermal signal sequence;

[0139] The fluidity value is the square root of the variance of the first derivative of the preprocessed electrodermal signal sequence divided by the variance of the signal.

[0140] The average derivative is the average value of the derivatives at each point in the preprocessed electrodermal signal sequence;

[0141] The decomposition process can be implemented using the singular value decomposition algorithm.

[0142] The linear fitting process involves using the characteristic vector element index value Ix as the known independent variable and the characteristic vector element value as the known dependent variable. The curve to be approximated is constructed using the known independent and dependent variables, and the curve to be approximated is fitted using the function approximation method to obtain the characteristic estimation polynomial.

[0143] The feature vector is represented as I a I a =[λ1,λ2,…,λ N1 N1 is the number of elements contained in the feature vector; the curve fitting of the curve to be approximated using the function approximation method can employ the best uniform linear approximation method. The feature estimation polynomial f(Ix) is expressed as:

[0144] f(Ix)=α P1 (Ix) P1 +α P1-1 (Ix) P1-1 +…+α2(Ix) 2 +α1(Ix)+α0,

[0145] Where P1 is the order of the feature estimation polynomial f(Ix), α0, α1, α2, ..., α P1 Estimate the coefficients of the polynomial f(Ix) for the aforementioned feature;

[0146] The skin conductance bracelet includes a bracelet body and a skin conductance signal acquisition device disposed on the bracelet; the skin conductance signal acquisition device includes a skin conductance sensor and a central processing unit;

[0147] The data cleaning process includes filling in missing values, smoothing noisy data, and smoothing or deleting outliers. Smoothing noisy data involves first identifying the noisy data, and then smoothing it based on the data preceding and following it. The noisy data refers to values ​​whose values ​​are less than the sensor's detection sensitivity for the observed data, or greater than the sensor's measurement limit for the observed data. Outlier identification can be performed using Kalman filtering. The filling values ​​for missing values ​​can be determined by averaging the measurements within a certain sampling interval before and after the missing value.

[0148] The data normalization process maps data from different value ranges to a specified value range. Its mathematical expression is:

[0149]

[0150] Where t is a specific data value in the dataset, t max t represents the maximum value of all data in the dataset. min t is the minimum value of all data in the dataset. 归一化 The value of data t after normalization;

[0151] A second aspect of this application discloses a multimodal cognitive screening method based on a brain-computer interface and a magnetic resonance imaging (MRI) wristband, comprising:

[0152] Using the brain-computer interface module, the EEG signal sequence of the user during cognitive testing is acquired;

[0153] The skin conductance bracelet was used to collect the skin conductance signal sequence of the user during a cognitive test;

[0154] The signal feature extraction module is used to perform feature extraction processing on the acquired EEG signal sequence and ESC signal sequence to obtain EEG feature sequence and ESC feature sequence;

[0155] Using the cognitive assessment module, the electroencephalogram (EEG) feature sequences, skin conductance feature sequences, skin conductance signal sequences, and EEG signal sequences are processed to assess cognitive impairment, and cognitive impairment assessment results are obtained.

[0156] The signal feature extraction module is used to perform feature extraction processing on the acquired EEG signal sequence and skin conductance signal sequence to obtain EEG feature sequence and skin conductance feature sequence, including:

[0157] The signal feature extraction module is used to preprocess the acquired EEG signal sequence and EDS signal sequence to obtain preprocessed EEG signal sequence and preprocessed EDS signal sequence.

[0158] The preprocessed EEG signal sequence and the preprocessed ESC signal sequence are subjected to feature calculation processing to obtain EEG feature sequence and ESC feature sequence, respectively.

[0159] The process of using the signal feature extraction module to preprocess the acquired EEG signal sequence and ESC signal sequence to obtain preprocessed EEG signal sequence and preprocessed ESC signal sequence includes:

[0160] Using the signal feature extraction module, the acquired EEG signal sequence and EDS signal sequence are cleaned to obtain cleaned EEG signal sequence and cleaned EDS signal sequence, respectively.

[0161] Data normalization was performed on the cleaned EEG signal sequence and the cleaned EDS signal sequence to obtain the preprocessed EEG signal sequence and the preprocessed EDS signal sequence.

[0162] The step involves performing feature calculations on the preprocessed EEG signal sequence and the preprocessed ESC signal sequence to obtain EEG feature sequences and ESC feature sequences, respectively, including:

[0163] Obtain standard EEG signal sequences;

[0164] The preprocessed EEG signal sequence is subtracted from the standard EEG signal sequence to obtain the difference signal sequence;

[0165] The statistical feature set of the differential signal sequence is obtained by statistical analysis; the statistical feature set includes the median, mode, range, and variance.

[0166] Using a matrix dimension estimation model, the length and statistical feature set of the differential signal sequence are calculated to obtain the matrix row dimension and matrix column dimension.

[0167] Based on the matrix row dimension values ​​and matrix column dimension values, a difference matrix is ​​constructed using the difference signal sequence;

[0168] The difference matrix is ​​processed by eigenvalue calculation to obtain the EEG feature sequence;

[0169] The preprocessed electrodermal signal sequence is subjected to time-frequency statistical processing to obtain a set of time-frequency statistical features; the set of time-frequency statistical features includes mean, variance, average rise time, harmonic mean, fluidity value, and average derivative;

[0170] The skin conductance feature is calculated by performing skin conductance feature calculation on the set of time-frequency statistical features to obtain the skin conductance feature sequence.

[0171] The expression for the matrix dimension estimation model is:

[0172]

[0173]

[0174] Where κ is the median, λ is the mode, α is the variance, β is the range, N is the length of the difference signal sequence, and L1 and L2 are the row and column dimensions of the matrix, respectively. Indicates rounding up. This indicates rounding down to the nearest integer.

[0175] The step of performing eigenvalue calculation on the difference matrix to obtain the EEG feature sequence includes:

[0176] The difference matrix is ​​decomposed to obtain the feature matrix; the calculation expression for the decomposition is as follows:

[0177] Y = UAV,

[0178] Where U is the left decomposition matrix, Y is the difference matrix, A is the characteristic matrix, V is the right decomposition matrix, U and V are both orthogonal matrices, and A is a diagonal matrix;

[0179] Extract the diagonal elements of the feature matrix to obtain the feature vector;

[0180] The elements and element index values ​​of the feature vector are subjected to linear fitting to obtain the feature estimation polynomial;

[0181] For each row vector of the difference matrix, the average value of the row vector is calculated; for each element in the row vector, the absolute value of the difference between it and the average value is calculated, and the absolute value of the difference is determined as the offset of the element.

[0182] For each row vector, find the element with the largest offset, and determine the index value of the element in the row vector as the offset index value of the row vector;

[0183] By using the feature estimation polynomial, the deviation index value of each row vector is calculated and processed to obtain the EEG feature value corresponding to each row vector;

[0184] By utilizing the EEG feature values ​​corresponding to all row vectors, an EEG feature sequence is constructed.

[0185] The step of performing skin conductance feature calculation on the time-frequency statistical feature set to obtain a skin conductance feature sequence includes:

[0186] Using the mean and variance, the preprocessed electrodermal signal sequence is subjected to a first feature calculation to obtain the first electrodermal feature quantity H1;

[0187] The average rise time and harmonic mean are processed by a second feature calculation to obtain the second skin conductance characteristic quantity H2.

[0188] The fluidity value, average derivative, mean, and preprocessed electrodermal signal sequence are subjected to a third feature calculation to obtain the third electrodermal feature quantity H3;

[0189] Using the first, second, and third skin conductance features, a skin conductance feature sequence is constructed.

[0190] The expression for the first feature calculation is:

[0191]

[0192] Where, r i Let ω be the i-th element of the preprocessed EDS signal sequence, τ be the variance, and N0 be the mean.

[0193] The expression for the second feature calculation is:

[0194]

[0195] Where μ represents the average rise time and ν represents the harmonic mean;

[0196] The expression for the third feature calculation process is:

[0197]

[0198] Where φ represents the liquidity value and ρ represents the average derivative.

[0199] The cognitive impairment assessment is performed on the electroencephalogram (EEG) feature sequences, skin conductance feature sequences, skin conductance signal sequences, and EEG signal sequences to obtain cognitive impairment assessment results, including:

[0200] The median value a1 and mode value a2 of the electrodermal signal sequence were obtained statistically.

[0201] The difference matrix and EEG feature sequence of the EEG signal sequence are processed for EEG cognitive impairment assessment to obtain a first assessment value;

[0202] The expression for the calculation and processing of the EEG cognitive impairment assessment is as follows:

[0203]

[0204] Among them, T i () represents the i-th term of a Chebyshev polynomial of the first kind, p i q represents the i-th element of the EEG feature sequence.i Let represent the mean of the vector in the i-th row of the difference matrix, and ne represent the first evaluation value;

[0205] The median value a1 and mode value a2 of the electrodermal signal sequence and the electrodermal characteristic sequence are processed to calculate and calculate the electrodermal cognitive impairment assessment to obtain the second assessment value;

[0206] The expression for the calculation and processing of the skin conductance cognitive impairment assessment is as follows:

[0207]

[0208] Where L1(), L2(), and L3() represent first-order Laguerre polynomials, second-order Laguerre polynomials, and third-order Laguerre polynomials, respectively, and np represents the second evaluation value;

[0209] The first and second assessment values ​​are weighted and summed to obtain the cognitive impairment assessment result.

[0210] The cognitive impairment assessment results are used to characterize the severity of the user's cognitive impairment; the higher the value, the more severe the user's cognitive impairment.

[0211] The weight values ​​used to perform a weighted summation of the first and second evaluation values ​​can be 0.2 and 0.3, or 0.4 and 0.5.

[0212] The length of the difference signal sequence is the number of elements contained in the difference signal sequence;

[0213] When L1 multiplied by L2 is less than N, the first L1 multiplied by L2 elements of the difference signal sequence are taken to construct the difference matrix;

[0214] The average rise time is the average rise time of all amplitudes in the preprocessed electrodermal signal sequence.

[0215] The harmonic mean is the average of all harmonic frequencies of the FFT sequence of the preprocessed electrodermal signal sequence;

[0216] The fluidity value is the square root of the variance of the first derivative of the preprocessed electrodermal signal sequence divided by the variance of the signal.

[0217] The average derivative is the average value of the derivatives at each point in the preprocessed electrodermal signal sequence;

[0218] The decomposition process can be implemented using the singular value decomposition algorithm.

[0219] The linear fitting process involves using the characteristic vector element index value Ix as the known independent variable and the characteristic vector element value as the known dependent variable. The curve to be approximated is constructed using the known independent and dependent variables, and the curve to be approximated is fitted using the function approximation method to obtain the characteristic estimation polynomial.

[0220] The feature vector is represented as I a I a =[λ1,λ2,…,λ N1 N1 is the number of elements contained in the feature vector; the curve fitting of the curve to be approximated using the function approximation method can employ the best uniform linear approximation method. The feature estimation polynomial f(Ix) is expressed as:

[0221] f(Ix)=α P1 (Ix) P1 +α P1-1 (Ix) P1-1 +…+α2(Ix) 2 +α1(Ix)+α0,

[0222] Where P1 is the order of the feature estimation polynomial f(Ix), α0, α1, α2, ..., α P1 Estimate the coefficients of the polynomial f(Ix) for the aforementioned feature.

[0223] The cognitive test may be a memory test, attention test, number memory test, image matching test, logical reasoning test, etc.

[0224] The brain-computer interface module can be implemented using a brain-computer interface device.

[0225] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A multimodal cognitive screening device based on brain-computer interface and magnetic resonance imaging (MRI) wristband, characterized in that, include: Brain-computer interface module, electrodermal wristband, signal feature extraction module, cognitive assessment module; The brain-computer interface module is used to acquire the EEG signal sequence of the user during cognitive testing; The skin conductance wristband is used to collect the skin conductance signal sequence of the user during cognitive testing; The signal feature extraction module is connected to the brain-computer interface module, the electrodermal wristband and the cognitive assessment module respectively, and is used to perform feature extraction processing on the acquired electroencephalogram (EEG) signal sequence and electrodermal (ED) signal sequence to obtain EEG feature sequence and EEG feature sequence. The cognitive assessment module is used to perform cognitive impairment assessment processing on the electroencephalogram (EEG) feature sequence, skin conductance feature sequence, skin conductance signal sequence, and EEG signal sequence to obtain cognitive impairment assessment results. The signal feature extraction module is used to perform feature extraction processing on the acquired electroencephalogram (EEG) signal sequence and electrodermal signal sequence to obtain EEG feature sequences and electrodermal signal feature sequences, including: The signal feature extraction module preprocesses the acquired EEG signal sequence and EDS signal sequence to obtain preprocessed EEG signal sequence and preprocessed EDS signal sequence. The preprocessed EEG signal sequence and the preprocessed ESC signal sequence are subjected to feature calculation processing to obtain EEG feature sequences and ESC feature sequences, including: Obtain standard EEG signal sequences; The preprocessed EEG signal sequence is subtracted from the standard EEG signal sequence to obtain the difference signal sequence; The statistical feature set of the differential signal sequence is obtained by statistical analysis; the statistical feature set includes the median, mode, range, and variance. Using a matrix dimension estimation model, the length and statistical feature set of the differential signal sequence are calculated to obtain the matrix row dimension and matrix column dimension. Based on the matrix row dimension values ​​and matrix column dimension values, a difference matrix is ​​constructed using the difference signal sequence; The difference matrix is ​​processed by eigenvalue calculation to obtain the EEG feature sequence; The preprocessed electrodermal signal sequence is subjected to time-frequency statistical processing to obtain a set of time-frequency statistical features; the set of time-frequency statistical features includes mean, variance, average rise time, harmonic mean, fluidity value, and average derivative; The skin conductance feature is calculated by performing skin conductance feature calculation on the set of time-frequency statistical features to obtain the skin conductance feature sequence; The expression for the matrix dimension estimation model is: ， ， in, The median value For the number of people, For variance, Where is the range value, and N is the length of the difference signal sequence. and These are the row dimension value and column dimension value of the matrix, respectively. Indicates rounding up. This indicates rounding down to the nearest integer.

2. The multimodal cognitive screening device based on brain-computer interface and magnetic resonance imaging (MRI) wristband as described in claim 1, characterized in that, The signal feature extraction module preprocesses the acquired EEG signal sequence and ESC signal sequence to obtain preprocessed EEG signal sequence and preprocessed ESC signal sequence, including: The signal feature extraction module performs data cleaning processing on the acquired EEG signal sequence and EDS signal sequence to obtain cleaned EEG signal sequence and cleaned EDS signal sequence. Data normalization was performed on the cleaned EEG signal sequence and the cleaned EDS signal sequence to obtain the preprocessed EEG signal sequence and the preprocessed EDS signal sequence.

3. A multimodal cognitive screening method based on brain-computer interface and magnetic resonance imaging (MRI) wristband, characterized in that, The multimodal cognitive screening device based on a brain-computer interface and a magnetic resonance imaging (MRI) wristband, as described in any one of claims 1 to 2, includes: Using the brain-computer interface module, the EEG signal sequence of the user during cognitive testing is acquired; The skin conductance bracelet was used to collect the skin conductance signal sequence of the user during a cognitive test; The signal feature extraction module is used to perform feature extraction processing on the acquired EEG signal sequence and ESC signal sequence to obtain EEG feature sequence and ESC feature sequence; Using the cognitive assessment module, the electroencephalogram (EEG) feature sequences, skin conductance feature sequences, skin conductance signal sequences, and EEG signal sequences are processed to assess cognitive impairment, and cognitive impairment assessment results are obtained.

4. The multimodal cognitive screening method based on brain-computer interface and magnetic resonance imaging (MRI) wristband as described in claim 3, characterized in that, The signal feature extraction module is used to perform feature extraction processing on the acquired EEG signal sequence and skin conductance signal sequence to obtain EEG feature sequence and skin conductance feature sequence, including: The signal feature extraction module is used to preprocess the acquired EEG signal sequence and EDS signal sequence to obtain preprocessed EEG signal sequence and preprocessed EDS signal sequence. The preprocessed EEG signal sequence and the preprocessed ESC signal sequence are subjected to feature calculation processing to obtain EEG feature sequence and ESC feature sequence, respectively.

5. The multimodal cognitive screening method based on brain-computer interface and magnetic resonance imaging (MRI) wristband as described in claim 4, characterized in that, The process of using the signal feature extraction module to preprocess the acquired EEG signal sequence and ESC signal sequence to obtain preprocessed EEG signal sequence and preprocessed ESC signal sequence includes: Using the signal feature extraction module, the acquired EEG signal sequence and EDS signal sequence are cleaned to obtain cleaned EEG signal sequence and cleaned EDS signal sequence, respectively. Data normalization was performed on the cleaned EEG signal sequence and the cleaned EDS signal sequence to obtain the preprocessed EEG signal sequence and the preprocessed EDS signal sequence.

6. The multimodal cognitive screening method based on brain-computer interface and magnetic resonance imaging (MRI) wristband as described in claim 3, characterized in that, The step of performing eigenvalue calculation on the difference matrix to obtain the EEG feature sequence includes: The difference matrix is ​​decomposed to obtain the feature matrix; the calculation expression for the decomposition is as follows: ， in, Y is the left decomposition matrix, and Y is the difference matrix. For the characteristic matrix, It is a right decomposition matrix. and All are orthogonal matrices. It is a diagonal matrix; Extract the diagonal elements of the feature matrix to obtain the feature vector; The elements and element index values ​​of the feature vector are subjected to linear fitting to obtain the feature estimation polynomial; For each row vector of the difference matrix, the average value of the row vector is calculated; for each element in the row vector, the absolute value of the difference between it and the average value is calculated, and the absolute value of the difference is determined as the offset of the element. For each row vector, find the element with the largest offset, and determine the index value of the element in the row vector as the offset index value of the row vector; By using the feature estimation polynomial, the deviation index value of each row vector is calculated and processed to obtain the EEG feature value corresponding to each row vector; By utilizing the EEG feature values ​​corresponding to all row vectors, an EEG feature sequence is constructed.

7. The multimodal cognitive screening method based on brain-computer interface and magnetic resonance imaging (MRI) wristband as described in claim 3, characterized in that, The step of performing skin conductance feature calculation on the time-frequency statistical feature set to obtain a skin conductance feature sequence includes: Using the mean and variance, the preprocessed electrodermal signal sequence is subjected to a first feature calculation to obtain the first electrodermal feature quantity. ; The average rise time and harmonic mean are processed by a second feature calculation to obtain the second skin conductance characteristic. ; The fluidity value, average derivative, mean, and preprocessed electrodermal signal sequence are subjected to third feature calculation to obtain the third electrodermal feature quantity. ; Using the first, second, and third skin conductance features, a skin conductance feature sequence is constructed. The expression for the first feature calculation is: ， in, For the i-th element of the preprocessed electrodermal signal sequence, Represents variance. This represents the mean. This indicates the number of elements contained in the preprocessed electrodermal signal sequence; The expression for the second feature calculation is: ， in, Indicates the average rise time. Indicates the mean value of harmonics; The expression for the third feature calculation process is: ， in, Indicates liquidity value, It represents the average derivative.

Images