A consciousness level detection system integrating ASSR and P300

Abstract

The present invention discloses a consciousness level detection system integrating ASSR and P300, including: a data transmission module for receiving and marking EEG signal data; a data processing module for importing EEG signal data and performing preprocessing operations; an experimental paradigm module for presenting a stimulation paradigm to a subject; a model training module for importing EEG signal data, extracting features and training a P300 model and an ASSR model as well as a hybrid model integrating the two components of P300 and ASSR; an online test module for processing the EEG signal data received from the EEG signal acquisition software while the subject is undergoing the experiment, and inputting the processed data into the model saved by the model training module, and feeding back the prediction results to the subject. The present invention makes full use of the superiority of brain-computer interface technology to improve the diagnosis level of the consciousness state of patients with consciousness disorders, and at the same time, provides a certain degree of explainability for the diagnosis results, thereby enhancing the credibility of the diagnosis results.

Description

Technical Field

[0001] The present invention relates to the technical field of brain-computer interface, and in particular to a consciousness level detection system integrating ASSR and P300. Background Art

[0002] Brain Computer Interface (BCI) refers to a direct communication and control channel established between the human or animal brain and a computer or other electronic device. Current brain-computer interface technology mainly achieves the purpose of control by collecting and analyzing EEG signals in different states, and effectively extracting and classifying them. Disorder of consciousness (DoC) refers to a state of loss of consciousness caused by various severe brain injuries, such as coma, vegetative state (VS) and minimally conscious state (MCS). Generally speaking, patients with different degrees of disorder of consciousness require different treatment plans, and accurate assessment of the state of consciousness of patients with disorder of consciousness is crucial to the formulation of diagnosis and treatment plans.

[0003] Existing brain-computer interface patent technologies are rarely used in the field of consciousness disorders. Most of the existing patent technologies are used in motion assistance and rehabilitation training, such as robotic arms, human exoskeletons and wheelchairs. Although there are relevant patents for detecting the consciousness level of patients with consciousness disorders, most of them induce ERP potentials through visual stimulation, which is not friendly to patients with severely impaired visual abilities.

[0004] At present, the clinical assessment of the degree of consciousness disorder in patients with consciousness disorder mainly relies on behavioral scales (such as CRS-R behavioral scale and Glasgow Coma Scale) and clinical experience, and a comprehensive evaluation is performed by examining the reactions caused by stimulation of eyes, speech and movement. In some cases, this method is simple and easy. However, literature studies have shown that the misdiagnosis rate of this method is as high as 37%-43%. Therefore, it is necessary to design an accurate and efficient method and system to help patients with consciousness disorder receive accurate diagnosis and treatment. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and proposes a consciousness level detection system that integrates ASSR and P300, breaking through the subjectivity and high misdiagnosis rate of traditional clinical diagnosis methods, making full use of the superiority of brain-computer interface technology, and improving the diagnosis level of the consciousness state of patients with consciousness disorders. At the same time, it provides a certain degree of explainability for the diagnosis results, thereby enhancing the credibility of the diagnosis results.

[0006] To achieve the above object, the technical solution provided by the present invention is: a consciousness level detection system integrating ASSR and P300, comprising:

[0007] The data transmission module is used to receive the EEG signal data transmitted by the EEG data acquisition software, mark the EEG signal data, and store them inside the system;

[0008] The data processing module is used to import the EEG signal data stored in the data transmission module and perform preprocessing operations on the EEG signal data, including channel selection, downsampling, baseline removal and filtering operations, and store them inside the system;

[0009] The experimental paradigm module is used to present the stimulus paradigm to the subject, i.e., the person who is undergoing the experiment, including the instruction broadcasting function and the paradigm running function;

[0010] The model training module is used to import the EEG signal data stored in the data processing module, extract the features of the EEG signal data and use them to train the P300 model and the ASSR model, as well as the hybrid model integrating the two components of P300 and ASSR, which are stored inside the system;

[0011] The online test module is used to process the EEG signal data received from the EEG signal acquisition software through the data transmission module and the data processing module while the subject is undergoing the experiment, and input the processed data into the model saved by the model training module to feed back the prediction results to the subject.

[0012] Furthermore, the experimental paradigm module includes a paradigm operation interface module, an instruction broadcasting module and a stimulus generation module, wherein:

[0013] The paradigm operation interface module is used to display the experimental paradigm, instructions and prediction result information;

[0014] The instruction broadcasting module randomly selects a position on the left and right of the subject as the target position, and the paradigm operation interface displays instructions to prompt the subject to pay attention to the stimulation occurring in the target position, while playing the instructions through headphones; wherein the stimulation occurring includes visual stimulation for inducing P300 potential and auditory stimulation for inducing ASSR potential;

[0015] The stimulation generation module displays a button on the left and right halves of the paradigm operation interface, respectively displaying the Chinese characters "left" and "right", and the two buttons flash alternately. While the left and right buttons flash, stimulation sounds are played in the left and right ears of the subject through headphones; wherein, the left button and the right button flash once each, and the left stimulation sound and the right stimulation sound are played once each, which is regarded as a round of stimulation, and an experiment is repeated at least eight rounds.

[0016] Further, the model training module includes an ASSR model training module, a P300 model training module and a hybrid model training module, wherein:

[0017] The ASSR model training module extracts data features and trains the ASSR model based on the typical association analysis method CCA, including the following steps:

[0018] a1. Select q / 2 sine-cosine signal pairs with the same frequency corresponding to the ASSR stimulation frequency to form a reference signal matrix:

[0019]

[0020] Where q is twice the number of sine-cosine signal pairs contained in the reference signal, and y q (t) represents the sine or cosine signal in the q / 2th sine-cosine signal pair; s represents the sequence number of the stimulation frequency, s = 1, 2; f s represents the stimulation frequency corresponding to sequence number s, Y s represents the reference signal matrix corresponding to the stimulation frequency with serial number s; t represents time in seconds, f c represents the sampling frequency, T c Represents the number of sampling points;

[0021] a2. Load the EEG data signal stored in the data processing module and represent the EEG signal data as a two-dimensional matrix X = (x1, x2, ..., x p ) T , where p is the number of signal acquisition electrodes, x ζ represents the vector composed of the EEG signal data collected by the ζ-th signal collection electrode, 1≤ζ≤p, T represents transposition;

[0022] a3. In order to simplify the formula, y τ (t) is expressed as y τ ,Right now:

[0023] Y s =(y1(t),y2(t),...,y q (t)) T =(y1,y2,...,y q ) T

[0024] Among them, y τ is a vector, 1≤τ≤q;

[0025] a4. Substitute matrices X and Y s The vectors in are expressed as linear combinations:

[0026]

[0027]

[0028] In the formula, up represents the pth linear combination of the vectors in the matrix X, v p Represents the matrix Y s The pth linear combination of the vectors in a 11 ,...,a pp and b 11 ,...,b pq are all constants, representing the coefficients of each element in the linear combination, so u p and v p is also a vector;

[0029] a5. Solve linear combinations and Pearson correlation coefficient

[0030]

[0031] In the formula, is the subscript of the linear combination, represent and The covariance of represent The variance of represent variance; r is an integer, 1≤r≤min(p,q), min(p,q) represents the smaller value between p and q;

[0032] a6. Typical related questions are: and The Pearson correlation coefficient is the largest, that is, and In the case of The largest

[0033] a7. For the stimulation frequency with serial number s, use the Euclidean norm to combine the stimulation frequencies to solve the r Pearson correlation coefficients:

[0034]

[0035] In the formula, c s For all Pearson correlation coefficients corresponding to the stimulus frequency with sequence number s, find the value after Euclidean norm;

[0036] a8. The target stimulus frequency is identified as the maximum value of the combined values ​​of all Pearson correlation coefficients:

[0037] f target =max(c s )

[0038] In the formula, ftarget is the calculated target stimulation frequency;

[0039] The P300 model training module extracts data features based on the support vector machine method SVM and trains the P300 model, including the following steps:

[0040] b1. Load the EEG signal data stored in the data processing module, reorganize the EEG signal data into several data packets epochs according to the annotations in the data, and label the data of each epoch. The data of the epoch is recorded as (f k ,l k ), each epoch represents a sample point, where k is the subscript of the epoch and also the subscript of the sample point; f k represents the eigenvalue of the kth sample point, l k Represents the label of the kth sample point;

[0041] b2. Define the eigenvector f=(f1,f2,...,f θ ), θ is the number of sample points in a set of sample points, f θ is the θth sample point; for a linearly separable data set, there are many separating hyperplanes that can correctly separate the data set, but the separating hyperplane with the largest geometric distance is unique. The separating hyperplane equation with the largest geometric distance is defined as: wt*φ(f)+bi=0, where wt represents the weight matrix, φ(f) represents the eigenvector after the eigenvector f is mapped, and bi represents the offset vector;

[0042] b3. The distance formula for calculating each set of sample points to the separating hyperplane is:

[0043]

[0044] In the formula, |wt T +bi|means (wt T +bi), ||wt|| represents the 2-norm of the weight matrix wt;

[0045] b4. The training set in the system contains two types of samples. The two separation hyperplane equations that pass through the support vectors of these two types of samples are defined as:

[0046]

[0047] In the formula, φ(f k ) represents the eigenvector after the eigenvalue mapping of the kth sample point;

[0048] Combining the above two equations, we get l k (wt T ·φ(f k)+bi)≥1, substitute it into the distance formula: The calculation formula for the distance d from each support vector to the separating hyperplane with the largest geometric distance is:

[0049]

[0050] b5. Convert the maximization distance d into a minimization problem:

[0051]

[0052] In the formula, st refers to the constraints of the following conditional formula;

[0053] b6. Use the Lagrange multiplier method to obtain the "dual problem" of the above minimization problem, whose Lagrangian function L(wt,bi,α) is:

[0054]

[0055] In the formula, α k is the Lagrange multiplier corresponding to the kth sample point; using the duality conclusion, we can obtain the “dual problem”:

[0056]

[0057]

[0058] In the formula, k and v are the subscripts representing the sample points in the group, α v is the Lagrange multiplier corresponding to the vth sample point, α is the set of Lagrange multipliers, f v represents the eigenvalue of the vth sample point, l v Represents the label of the vth sample point;

[0059] b7. After solving for α, substitute it back into the formula: Find the weight vector wt and the offset vector bi;

[0060] b8. When classifying, input the unlabeled data into the discriminant hyperplane equation: wt*φ(f)+bi=0, and output its category prediction value;

[0061] The hybrid model training module extracts data features based on the regularized linear discriminant method RLDA and trains the model, including the following steps:

[0062] c1. Load the EEG signal data stored in the data processing module, input it into the ASSR model and P300 model trained by the ASSR model training module and P300 model training module respectively, calculate the patient's ASSR score and P300 score, respectively recorded as Score ASSRand Score P300 , Score ASSR and Score P300 are two-dimensional matrices with N rows. ASSR and Score P300 The concatenation results in the N·Q dimension feature matrix F=(fr1, fr2, ..., fr Q ), where Q is the number of columns in F, fr Q is the Qth eigenvector in F;

[0063] c2. Construct a label vector L with a dimension of N*1;

[0064] c3. Calculate the unbiased estimated covariance matrix of the feature matrix F

[0065]

[0066] Where ε is the subscript of the eigenvector, 1≤ε≤Q; is an unbiased estimate of the mean of all elements in the eigenvector:

[0067]

[0068] c4, covariance matrix The average value υ of the trace is:

[0069]

[0070] In the formula, di is The number of columns;

[0071] c5. Calculate the identification parameter γ * :

[0072]

[0073] In the formula, ∑S F T S F Indicates the matrix S F T S F The sum of the elements in , var(z) represents the variance calculated along the first dimension of the matrix z; S F represents the covariance matrix of the feature matrix F, diag(S F ) represents the matrix S F The vector composed of the elements on the diagonal; z = (z1, z2, ..., z N ), where z i represents the i-th eigenvector in the feature matrix F minus the unbiased estimate The inner product after:

[0074]

[0075] In the formula, fr i is the i-th eigenvector in F;

[0076] c6. Define the separation hyperplane equation: H = w*F + b, then:

[0077]

[0078]

[0079] Where w is the weight matrix, b is the offset vector, Represents the mean of the characteristic values ​​corresponding to the s-th stimulation frequency, s represents the sequence number of the stimulation frequency, s=1,2.

[0080] Furthermore, the online testing module includes a model prediction module and a result feedback module, wherein:

[0081] The model prediction module is used to load the EEG signal data stored in the data processing module, input the EEG signal data into the model trained and saved by the model training module, and calculate and save the prediction results;

[0082] The result feedback module is used to display the prediction results of the model prediction module on the paradigm operation interface.

[0083] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0084] 1. Apply visual and auditory brain-computer interface technology to the detection of consciousness level of patients with consciousness disorders. Through visual and auditory stimulation, event-related potentials P300 and ASSR are induced respectively. For patients with consciousness disorders with impaired visual or auditory functions, this system can be used to detect the consciousness level, which can meet the needs of most patients with consciousness disorders.

[0085] 2. Explore the application of ASSR event-related potentials in the field of consciousness disorders and lay the foundation for subsequent in-depth research.

[0086] 3. In response to the common problem of small sample size in the field of consciousness disorders, this system combines machine learning algorithms, especially SVM algorithms, which perform better in small sample conditions and improve the accuracy of classification and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 It is a relationship diagram between various modules of the system of the present invention.

[0088] Figure 2This is a diagram of the EEG signal data channels selected when the system of the present invention is used to train the P300 model.

[0089] Figure 3 This is a diagram of EEG signal data channels selected when the system of the present invention is used to train the ASSR model.

[0090] Figure 4 Detailed diagram of binaural stimulation sounds used in the system of the present invention.

[0091] Figure 5 The figure is a flow chart of the detection algorithm of the system of the present invention. DETAILED DESCRIPTION

[0092] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0093] This embodiment provides a consciousness level detection system integrating ASSR and P300. It is a consciousness level detection system developed in C++ language and can be run on Windows devices. The relationship between the modules of the system is as follows: Figure 1 As shown in the figure, the system detection algorithm flow is as follows Figure 5 It includes:

[0094] The data transmission module is used to receive the EEG signal data transmitted by the EEG data acquisition software, mark the EEG signal data, and store them inside the system;

[0095] The data processing module is used to import the EEG signal data stored in the data transmission module, and perform preprocessing operations on the EEG signal data, including but not limited to channel selection, downsampling, baseline removal and filtering operations, and finally store them inside the system;

[0096] The experimental paradigm module is used to present the stimulation paradigm to the subject, i.e. the person who is undergoing the experiment, including the instruction broadcasting function and the paradigm running function;

[0097] The model training module is used to import the EEG signal data stored in the data processing module, extract the features of the EEG signal data and use them to train the P300 model and the ASSR model, as well as the hybrid model integrating the two components of P300 and ASSR, and finally store them inside the system;

[0098] The online test module is used to process the EEG signal data received from the EEG signal acquisition software through the data transmission module and the data processing module while the subject is undergoing the experiment, and input the processed data into the model saved by the model training module to feed back the prediction results to the subject;

[0099] Furthermore, the data processing module selects data from different electrode channels of the head-mounted electrode cap for training the ASSR model and the P300 model respectively. Figure 2 As shown in Figure 2, the EEG signal data of FZ, FCZ, CZ, CPZ, P7, P3, PZ, P4, P8, O1, OZ, O2 and other channels are used as the data source for training the P300 model; as shown in Figure 2 Figure 3 As shown in the figure, the EEG signal data of FZ, FCZ, T7, CZ, T8, O1, OZ, O2 and other channels are used as the data source for training the ASSR model.

[0100] Furthermore, the experimental paradigm module includes a paradigm operation interface module, an instruction broadcasting module and a stimulus generation module, wherein:

[0101] The paradigm operation interface module is used to display information such as the experimental paradigm, instructions and prediction results.

[0102] The instruction broadcasting module randomly selects a position on the left and right of the subject as the target position, and the paradigm operation interface displays instructions to prompt the subject to pay attention to the stimulation occurring in the target position, while playing the instructions through headphones. The stimulation occurring includes visual stimulation for inducing P300 potential and auditory stimulation for inducing ASSR potential.

[0103] The stimulation generation module displays a button on the left and right half of the paradigm operation interface, respectively displaying the Chinese characters "left" and "right". The two buttons flash alternately. While the left and right buttons flash, the stimulation sound is played in the left and right ears of the subject through the headphones. The stimulation sounds for the left and right ears are short pure tones generated and amplitude modulated using Matlab. Figure 4 The amplitude and frequency of the two stimuli are displayed. The left and right buttons flash once, and the left and right stimuli are played once, which is considered a round of stimulation. One experiment is repeated at least eight times.

[0104] Furthermore, the model training module includes an ASSR model training module, a P300 model training module and a hybrid model training module, wherein:

[0105] The ASSR model training module extracts data features and trains the ASSR model based on the Canonical Correlation Analysis (CCA) method, including the following steps:

[0106] a1. Select q / 2 sine-cosine signal pairs with the same frequency corresponding to the ASSR stimulation frequency to form a reference signal matrix:

[0107]

[0108] Where q is twice the number of sine-cosine signal pairs contained in the reference signal, and y q (t) represents the sine or cosine signal in the q / 2th sine-cosine signal pair; s represents the sequence number of the stimulation frequency, s = 1, 2; f s represents the stimulation frequency corresponding to sequence number s, Y s represents the reference signal matrix corresponding to the stimulation frequency with serial number s; t represents time in seconds, f c represents the sampling frequency, T c Represents the number of sampling points.

[0109] a2. Load the EEG data signals stored in the data processing module and represent these EEG data as a two-dimensional matrix X: X = (x1, x2, ..., x p ) T . Where p is the number of signal collection electrodes, x ζ Represents the vector composed of the EEG signal data collected by the ζ-th signal collection electrode, 1≤ζ≤p; T represents transpose.

[0110] a3. In order to simplify the formula, y τ (t) is expressed as y τ ,Right now:

[0111] Y s =(y1(t),y2(t),...,y q (t)) T =(y1,y2,...,y q ) T

[0112] Among them, y τ is a vector, 1≤τ≤q.

[0113] a4. Substitute matrices X and Y s The vectors in are expressed as linear combinations:

[0114]

[0115]

[0116] Among them, u p represents the pth linear combination of the vectors in the matrix X, v p Represents the matrix Y s The pth linear combination of the vectors in a 11 ,...,a pp and b 11 ,...,b pq are all constants, representing the coefficients of each element in the linear combination, so up and v p is also a vector.

[0117] a5. Solve linear combinations and Pearson correlation coefficient

[0118]

[0119] in, is the subscript of the linear combination, represent and The covariance of represent The variance of represent variance; r is an integer, 1≤r≤min(p,q), min(p,q) represents the smaller value between p and q.

[0120] a6. Typical related questions are: and The Pearson correlation coefficient is the largest, that is, and In the case of The largest

[0121] a7. For the stimulation frequency with serial number s, use the Euclidean norm to combine the stimulation frequencies to solve the r Pearson correlation coefficients:

[0122]

[0123] Among them, c s For all Pearson correlation coefficients corresponding to the stimulus frequency with sequence number s, find the value after Euclidean norm.

[0124] a8. The target stimulus frequency is identified as the maximum value of the combination of all Pearson correlation coefficients: f target =max(c s ).

[0125] Among them, f target is the target stimulation frequency calculated by the algorithm.

[0126] The P300 model training module extracts data features and trains the P300 model based on a support vector machine (SVM) method, including the following steps:

[0127] b1. Load the EEG signal data stored in the data processing module, reorganize the EEG signal data into several data packets (epochs) according to the annotations in the data, and label the data of each epoch. The data of the epoch is recorded as (f k ,l k ), each epoch represents a sample point. Where k is the subscript of the epoch and also the subscript of the sample point; f k represents the eigenvalue of the kth sample point, l k Represents the label of the kth sample point.

[0128] b2. Define the eigenvector f: f = (f1, f2, ..., f θ ), θ is the number of sample points in a set of sample points, f θ is the θth sample point. For a linearly separable data set, there are many separating hyperplanes that can correctly separate the data set, but the separating hyperplane with the largest geometric distance is unique. The equation for defining the separating hyperplane with the largest geometric distance is: wt*φ(f)+bi=0, where wt represents the weight matrix, φ(f) represents the eigenvector after the eigenvector f is mapped, and bi represents the offset vector.

[0129] b3. The distance formula for calculating each set of sample points to the separating hyperplane is:

[0130]

[0131] Among them, |wt T +bi|means (wt T +bi), and ||wt|| represents the 2-norm of the weight matrix wt.

[0132] b4. The training set in this system contains two types of samples. The two separating hyperplane equations that pass through the support vectors of these two types of samples are defined as:

[0133]

[0134] In the formula, φ(f k ) represents the eigenvector after the eigenvalue mapping of the kth sample point;

[0135] Combining the above two equations, we get l k (wt T φ(f k )+bi)≥1, substitute it into the distance formula: The calculation formula for the distance (d) from each support vector to the separating hyperplane with the largest geometric distance is:

[0136]

[0137] b5. Convert the maximization distance d into a minimization problem:

[0138]

[0139] Among them, st refers to the constraints of the following conditional formula.

[0140] b6. Use the Lagrange multiplier method to obtain the "dual problem" of the above minimization problem, whose Lagrangian function L(wt,bi,α) is:

[0141]

[0142] Among them, α k is the Lagrange multiplier corresponding to the kth sample point. Using the duality conclusion, we can obtain the “dual problem”:

[0143]

[0144]

[0145] Among them, k and v are the subscripts representing the sample point in the group, α v is the Lagrange multiplier corresponding to the vth sample point, α is the set of Lagrange multipliers, f v represents the eigenvalue of the vth sample point, l v Represents the label of the vth sample point;

[0146] b7. After solving for α, substitute it back into the formula: Find the weight vector wt and the offset vector bi.

[0147] b8. When classifying, input the unlabeled data into the discriminant hyperplane equation: wt*φ(f)+bs=0, and output its category prediction value.

[0148] The hybrid model training module extracts data features and trains the model based on the regularized linear discriminant method (RLDA), including the following steps:

[0149] c1. Load the EEG signal data stored in the data processing module, input it into the ASSR model and P300 model trained by the ASSR model training module and P300 model training module respectively, calculate the patient's ASSR score and P300 score, respectively recorded as Score ASSR and Score P300 . Score ASSR and Score P300 are both two-dimensional matrices with N rows. ASSR and Score P300The concatenation results in the N·Q-dimensional feature matrix F: F = (fr1, fr2, ..., fr Q ). Where Q is the number of columns of F, fr Q is the Qth eigenvector in F.

[0150] c2. Construct a label vector L with a dimension of N*1.

[0151] c3. Calculate the unbiased estimated covariance matrix of the feature matrix F

[0152]

[0153] Among them, ε is the subscript of the eigenvector, 1≤ε≤Q; is an unbiased estimate of the mean of all elements in the eigenvector:

[0154]

[0155] c4, covariance matrix The average value υ of the trace is:

[0156]

[0157] Among them, di is The number of columns.

[0158] c5. Calculate the identification parameter γ*:

[0159]

[0160] Among them, ∑S F T S F Indicates the matrix S F T S F The sum of the elements in , var(z) represents the variance calculated along the first dimension of the matrix z; S F represents the covariance matrix of the feature matrix F, diag(S F ) represents the matrix S F The vector composed of the elements on the diagonal; z = (z1, z2, ..., z N ), where z i represents the i-th eigenvector in the feature matrix F minus the unbiased estimate The inner product after:

[0161]

[0162] In the formula, fr i is the i-th eigenvector in F;

[0163] c6. Define the separation hyperplane equation: H = w*F + b, then

[0164]

[0165]

[0166] Among them, w is the weight matrix, b is the offset vector, Represents the mean of the eigenvalues ​​corresponding to the sth stimulation frequency.

[0167] The online testing module includes a model prediction module and a result feedback module, wherein:

[0168] The model prediction module is used to load the EEG signal data stored in the data processing module, input the EEG signal data into the model trained and saved by the model training module, and calculate and save the prediction results.

[0169] The result feedback module is used to display the prediction result of the model prediction module on the paradigm operation interface. If the prediction result is the same as the target direction prompted by the experimental paradigm module, the system will play a cheering sound, otherwise the prompt "Don't be discouraged, come on!" will be displayed on the paradigm operation interface.

[0170] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A consciousness level detection system integrating ASSR and P300, characterized in that: include: The data transmission module is used to receive the EEG signal data transmitted by the EEG data acquisition software, mark the EEG signal data, and store them inside the system; The data processing module is used to import the EEG signal data stored in the data transmission module and perform preprocessing operations on the EEG signal data, including channel selection, downsampling, baseline removal and filtering operations, and store them inside the system; The experimental paradigm module is used to present the stimulus paradigm to the subject, i.e., the person who is undergoing the experiment, including the instruction broadcasting function and the paradigm running function; The model training module is used to import the EEG signal data stored in the data processing module, extract the features of the EEG signal data and use them to train the P300 model and the ASSR model, as well as the hybrid model integrating the two components of P300 and ASSR, which are stored inside the system; The online test module is used to process the EEG signal data received from the EEG signal acquisition software through the data transmission module and the data processing module while the subject is undergoing the experiment, and input the processed data into the model saved by the model training module to feed back the prediction results to the subject; The model training module includes an ASSR model training module, a P300 model training module and a hybrid model training module, wherein: The ASSR model training module extracts data features and trains the ASSR model based on the typical association analysis method CCA, including the following steps: a1. Select q / 2 sine-cosine signal pairs with the same frequency corresponding to the ASSR stimulation frequency to form a reference signal matrix: Where q is twice the number of sine-cosine signal pairs contained in the reference signal, and y q (t) represents the sine or cosine signal in the q / 2th sine-cosine signal pair; s represents the sequence number of the stimulation frequency, s = 1, 2; f s represents the stimulation frequency corresponding to sequence number s, Y s represents the reference signal matrix corresponding to the stimulation frequency with serial number s; t represents time in seconds, f c represents the sampling frequency, T c Represents the number of sampling points; a2. Load the EEG data signal stored in the data processing module and represent the EEG signal data as a two-dimensional matrix X = (x1, x2, ..., x p ) T , where p is the number of signal acquisition electrodes, x ζ represents the vector composed of the EEG signal data collected by the ζ-th signal collection electrode, 1≤ζ≤p, T represents transposition; a3. In order to simplify the formula, y τ (t) is expressed as y τ ,Right now: Y s =(y1(t),y2(t),...,y q (t)) T =(y1,y2,...,y q ) T Among them, y τ is a vector, 1≤τ≤q; a4. Substitute matrices X and Y s The vectors in are expressed as linear combinations: In the formula, u p represents the pth linear combination of the vectors in the matrix X, v p Represents the matrix Y s The pth linear combination of the vectors in a 11 ,...,a pp and b 11 ,...,b pq are all constants, representing the coefficients of each element in the linear combination, so u p and v p is also a vector; a5. Solve linear combinations and Pearson correlation coefficient In the formula, is the subscript of the linear combination, represent and The covariance of represent The variance of represent variance; r is an integer, 1≤r≤min(p,q), min(p,q) represents the smaller value between p and q; a6. Typical related questions are: and The Pearson correlation coefficient is the largest, that is, and In the case of The largest a7. For the stimulation frequency with serial number s, use the Euclidean norm to combine the stimulation frequencies to solve the r Pearson correlation coefficients: In the formula, c s For all Pearson correlation coefficients corresponding to the stimulus frequency with sequence number s, find the value after Euclidean norm; a8. The target stimulus frequency is identified as the maximum value of the combined values ​​of all Pearson correlation coefficients: f target =max(c s ) In the formula, f target is the calculated target stimulation frequency; The P300 model training module extracts data features based on the support vector machine method SVM and trains the P300 model, including the following steps: b1. Load the EEG signal data stored in the data processing module, reorganize the EEG signal data into several data packets epochs according to the annotations in the data, and label the data of each epoch. The data of the epoch is recorded as (f k ,l k ), each epoch represents a sample point, where k is the subscript of the epoch and also the subscript of the sample point; f k represents the eigenvalue of the kth sample point, l k Represents the label of the kth sample point; b2. Define the eigenvector f=(f1,f2,...,f θ ), θ is the number of sample points in a set of sample points, f θ is the θth sample point; for a linearly separable data set, there are many separating hyperplanes that can correctly separate the data set, but the separating hyperplane with the largest geometric distance is unique. The separating hyperplane equation with the largest geometric distance is defined as: wt*φ(f)+bi=0, where wt represents the weight matrix, φ(f) represents the eigenvector after the eigenvector f is mapped, and bi represents the offset vector; b3. The distance formula for calculating each set of sample points to the separating hyperplane is: In the formula, |wt T +bi|means (wt T +bi), ||wt|| represents the 2-norm of the weight matrix wt; b4. The training set in the system contains two types of samples. The two separation hyperplane equations that pass through the support vectors of these two types of samples are defined as: In the formula, φ(f k ) represents the eigenvector after the eigenvalue mapping of the kth sample point; Combining the above two equations, we get l k (wt T ·φ(f k )+bi)≥1, substitute it into the distance formula: The calculation formula for the distance d from each support vector to the separating hyperplane with the largest geometric distance is: b5. Convert the maximization distance d into a minimization problem: In the formula, st refers to the constraints of the following conditional formula; b6. Use the Lagrange multiplier method to obtain the "dual problem" of the above minimization problem, whose Lagrangian function L(wt,bi,α) is: In the formula, α k is the Lagrange multiplier corresponding to the kth sample point; using the duality conclusion, we can obtain the "dual problem": In the formula, k and v are the subscripts representing the sample points in the group, α v is the Lagrange multiplier corresponding to the vth sample point, α is the set of Lagrange multipliers, f v represents the eigenvalue of the vth sample point, l v Represents the label of the vth sample point; b7. After solving for α, substitute it back into the formula: Find the weight vector wt and the offset vector bi; b8. When classifying, input the unlabeled data into the discriminant hyperplane equation: wt*φ(f)+bi=0, and output its category prediction value; The hybrid model training module extracts data features based on the regularized linear discriminant method RLDA and trains the model, including the following steps: c1. Load the EEG signal data stored in the data processing module, input it into the ASSR model and P300 model trained by the ASSR model training module and P300 model training module respectively, calculate the patient's ASSR score and P300 score, respectively recorded as Score ASSR and Score P300 , Score ASSR and Score P300 are two-dimensional matrices with N rows. ASSR and Score P300 The concatenation results in the N·Q dimension feature matrix F=(fr1, fr2, ..., fr Q ), where Q is the number of columns in F, fr Q is the Qth eigenvector in F; c2. Construct a label vector L with a dimension of N*1; c3. Calculate the unbiased estimated covariance matrix of the feature matrix F Where ε is the subscript of the eigenvector, 1≤ε≤Q; is an unbiased estimate of the mean of all elements in the eigenvector: c4, covariance matrix The average value υ of the trace is: In the formula, di is The number of columns; c5. Calculate the identification parameter γ * : Where, ΣS F T S F Indicates the matrix S F T S F The sum of the elements in , var(z) represents the variance calculated along the first dimension of the matrix z; S F represents the covariance matrix of the feature matrix F, diag(S F ) represents the matrix S F The vector composed of the elements on the diagonal; z = (z1, z2, ..., z N ), where z i represents the i-th eigenvector in the feature matrix F minus the unbiased estimate The inner product after: In the formula, fr i is the i-th eigenvector in F; c6. Define the separation hyperplane equation: H = w*F + b, then: Where w is the weight matrix, b is the offset vector, Represents the mean of the characteristic values ​​corresponding to the s-th stimulation frequency, s represents the sequence number of the stimulation frequency, s=1,2.

2. A consciousness level detection system integrating ASSR and P300 according to claim 1, characterized in that: The experimental paradigm module includes a paradigm operation interface module, an instruction broadcast module and a stimulus generation module, wherein: The paradigm operation interface module is used to display the experimental paradigm, instructions and prediction result information; The instruction broadcasting module randomly selects a position on the left and right of the subject as the target position, and the paradigm operation interface displays instructions to prompt the subject to pay attention to the stimulation occurring in the target position, while playing the instructions through headphones; wherein the stimulation occurring includes visual stimulation for inducing P300 potential and auditory stimulation for inducing ASSR potential; The stimulation generation module displays a button on the left and right halves of the paradigm operation interface, respectively displaying the Chinese characters "left" and "right", and the two buttons flash alternately. While the left and right buttons flash, stimulation sounds are played in the left and right ears of the subject through headphones; wherein, the left button and the right button flash once each, and the left stimulation sound and the right stimulation sound are played once each, which is regarded as a round of stimulation, and an experiment is repeated at least eight rounds.

3. A consciousness level detection system integrating ASSR and P300 according to claim 1, characterized in that: The online testing module includes a model prediction module and a result feedback module, wherein: The model prediction module is used to load the EEG signal data stored in the data processing module, input the EEG signal data into the model trained and saved by the model training module, and calculate and save the prediction results; The result feedback module is used to display the prediction results of the model prediction module on the paradigm operation interface.

Images