A method for EEG speech imagery recognition based on dynamic hypergraph learning

Abstract

The present invention discloses a method for recognizing EEG speech imagery based on dynamic hypergraph learning. The method is as follows: 1. EEG data acquisition; 2. Preprocessing and feature extraction; 3. Establishing an objective function; 4. Solving the variables in the objective function to obtain a feature projection matrix and a speech imagery category corresponding to the measured EEG data. 5. Analyzing the feature projection matrix to obtain the importance of each channel and each frequency band, and reconstructing the sample features based on the importance as the model input to obtain the final recognition result. The present invention uses a hypergraph to construct an EEG data model, mines high-order information between data, better characterizes the correlation of data, and dynamically and adaptively updates the vertex weights, hyperedge weights and hypergraph structure in the hypergraph during the label learning process, thereby improving the learning ability and decoding accuracy of the model. In addition, the dynamically updated feature projection matrix projects the original features into a low-dimensional subspace, thereby improving the robustness of the model.

Description

Technical Field

[0001] The invention belongs to the technical field of electroencephalogram signal processing, and specifically designs an electroencephalogram speech imagery recognition method based on dynamic hypergraph learning. Background Art

[0002] Brain-Computer Interface (BCI) technology, as an innovative way of communication, is an interdisciplinary technology involving cognitive science, neuroscience, biotechnology, biomedical engineering, information technology, computer science and applied mathematics. It does not rely on traditional neuromuscular pathways, allowing direct communication between the brain and electronic devices, analyzing the user's brain activity signals and converting them into instructions. This technology has shown great potential in helping people with limited speech or limb function communicate with the outside world. Brain-computer interfaces have been developed and tested to control communication equipment, which can help patients with neurological diseases such as speech disorders to interact with the outside world, so that they can express their intentions without speaking, just by imagining pronunciation in their brains. This type of brain-computer interface technology is called speech imagination brain-computer interface.

[0003] Speech imagination is the process of simulating language production in the brain. It does not involve the output of actual language. During the speech imagination process, the physiological signals generated by the brain, namely electroencephalography (EEG), can be detected. By preprocessing and analyzing the EEG signals, the individual's intentions can be decoded and conveyed to the outside world. The advantage of the brain-computer interface based on speech imagination is that it is the form closest to verbal communication in daily life. The user experience is more natural. It can help patients with language disorders, locked-in syndrome, amyotrophic lateral sclerosis and other diseases communicate with the outside world more directly and effectively, and can also play an auxiliary role in their rehabilitation process.

[0004] Constructing a relational model for the extracted EEG data is an important prerequisite for analyzing and decoding the data. In the field of traditional machine learning, graphs are widely studied and applied as an efficient relational expression structure. However, in the field of brain physiological signal analysis, the non-paired relational structure between different brain regions, different frequency bands and different features is difficult to construct and generate simply using graphs. As a special graph structured data, hypergraphs have the characteristic that a hyperedge can contain any number of nodes, which enables them to better mine high-order relationships between data and have higher correlation modeling capabilities. Therefore, the construction of EEG data models based on hypergraphs has relatively better advantages and better performance in practical applications.

[0005] However, in the traditional static hypergraph learning process, the structure of the hypergraph itself does not change, and only the decoding results are updated based on the hypergraph constructed based on the initial data, which makes the model susceptible to noise and outliers in the original data, and cannot effectively utilize new information that appears during the learning process, thereby reducing the accuracy of the model. Therefore, a model that can dynamically and adaptively change the hypergraph parameters during the learning process will have better performance, and the present invention will anchor this direction to build the model. Summary of the invention

[0006] The present invention aims at the problem of how to dynamically update the hypergraph in the process of hypergraph learning in the field of speech imagination brain-computer interface, and proposes a method for EEG speech imagination recognition based on dynamic hypergraph learning. The model proposed by the present invention can dynamically and adaptively update the weights of the vertices, the hyperedge weights and the structure of the hypergraph itself in the hypergraph learning process, so that the model can also be dynamically updated to learn effective information during the learning process. In addition, in order to enhance the robustness of the model, the present invention also introduces a feature projection matrix that is dynamically updated with the hypergraph, and projects the original feature data from the high-dimensional feature space to a low-dimensional feature subspace, so as to reduce the influence of noise and abnormal points in the original data space. After obtaining the updated projection matrix, the feature projection matrix is ​​analyzed to obtain the importance of each channel and each frequency band and quantify them, and the analysis results are fed back to the decoding model to improve the model efficiency and accuracy. Considering the data collection annotation, time cost and recognition accuracy in the actual application process, the algorithm used in the present invention belongs to semi-supervised learning, and it is necessary to input labeled data and unlabeled data at the same time for model training.

[0007] In a first aspect, the present invention provides an EEG speech imagery recognition method based on dynamic hypergraph learning, which comprises the following steps:

[0008] Step 1: Collecting labeled EEG data of multiple subjects under different speech imagery tasks; collecting EEG data of subjects who are performing EEG speech imagery;

[0009] Step 2: Preprocess and extract features of the measured EEG data and the labeled EEG data to obtain a sample feature matrix; the sample feature matrix is ​​divided into a labeled sample feature matrix X l And the unlabeled sample feature matrix X u ;

[0010] Step 3: Use a hypergraph to describe the correlation between samples, dynamically update the hypergraph vertex weights, hyperedge weights and hypergraph structure, and introduce a feature projection matrix to reduce the dimension of sample features to construct the objective function of the machine learning model based on dynamic hypergraph learning;

[0011] Step 4: Solve the variables in the objective function to obtain the feature projection matrix and the EEG intention decoding result matrix. Based on the EEG intention decoding result matrix, the speech imagination category corresponding to the measured EEG data is obtained.

[0012] Preferably, the objective function constructed in step 3 is as follows:

[0013]

[0014] in, is the Laplacian matrix of the hypergraph, which is used to describe the high-order relationship between samples. is the sample’s EEG intention decoding result matrix, H is the hypergraph’s association matrix, U is the sample’s vertex weight diagonal matrix, and D v is a vertex degree diagonal matrix, whose diagonal elements are is a hyper-degree diagonal matrix, whose diagonal elements are is the feature projection matrix, where f is the number of features after projection, w = (w1, w2, ..., w m ) is a vector representing the hyperedge weight, where w i (i=1, 2, ..., m) is the weight of each hyperedge, X l is the labeled sample feature matrix, Y l is the corresponding sample EEG intention category matrix, I is the unit matrix, and α and β are regularization coefficients. The first term of the objective function is a regularization term that makes the hypergraph smooth in the label space, and also makes the label matrix smooth in the hypergraph structure. It enables samples with strong connections in the hypergraph to share similar decoding results and achieve EEG intention category prediction; the second term is a regularization term that makes the hypergraph smooth in the feature space. Combined with the constraint M T XX T M=I, which means learning the projection matrix from the original feature space to the low-dimensional feature space. According to the sample features after projection, the vertex weights and hypergraph structure can be updated. The third term is the 2-norm regularization term of w, which combines the first two terms and the constraint w T 1=1 and w i >0(i=1, 2, …, m), the weight of each hyperedge can be learned.

[0015] Preferably, in step 4, after obtaining the feature projection matrix and the EEG intention decoding result matrix, a feature importance vector m describing the importance of all dimensions in the feature vector is established according to the feature projection matrix; the relative importance of each channel and each frequency band of EEG acquisition is calculated according to the feature importance vector m; according to the importance, some feature vectors with high importance are extracted to construct a new sample feature matrix; the new sample feature matrix is ​​input into the objective function for solution to obtain an updated EEG intention decoding result matrix.

[0016] Preferably, the method for calculating the relative importance of each frequency band and each channel according to the feature importance vector m is as follows:

[0017] The expression for calculating the importance of each frequency band, frequent(i), is as follows:

[0018]

[0019] Where i is the frequency band number; p is the number of extracted feature categories, ch is the number of channels; m((i-1)×p×ch+k) is the ((i-1)×p×ch+k)th element in the feature importance vector m;

[0020] The expression for calculating the importance of each channel channel(j) is as follows:

[0021]

[0022] Among them, fre is the number of frequency bands.

[0023] Preferably, there are three types of extracted features, namely, statistical, time domain, and frequency domain features. There are five extracted frequency bands, namely, Delta band, Theta band, Alpha band, Beta band, and Gamma band. There are 64 extracted channels.

[0024] As a preference,

[0025] The process of preprocessing and feature extraction of EEG data in step 2 is as follows: first, notch filtering is performed to remove 50Hz power frequency noise, and to remove eye movement and electromyographic artifacts, and then a fifth-order Butterworth filter is used to extract the data of each target frequency band; then, statistical, time domain, and frequency domain feature extraction are performed on the data of each target frequency band; the statistical feature is one or more of skewness and kurtosis, the time domain feature is Shannon entropy, and the frequency domain feature is power spectrum entropy. For a single sample, all the features extracted above are concatenated into a feature vector. Assuming the number of features is d, a d×1 feature vector is obtained, and for all samples, a d×n sample feature matrix X is obtained.

[0026] Preferably, in step 1, visual stimulation and auditory stimulation of speech imagination are applied to the subjects simultaneously to obtain EEG data under different speech imagination tasks.

[0027] Preferably, the process of solving the objective function in step 4 is: after initializing the variables in the objective function, the EEG intention decoding result matrix F, the feature projection matrix M, the hypergraph association matrix H and the hyperedge weight matrix W are solved by using the joint iterative optimization method.

[0028] The initialization method of each variable is as follows: Let F in the EEG intention decoding result matrix F l =Yl , F u All elements in each row are initialized to 1 / c, let H be constructed based on the sample distance, let W be the unit matrix, let U be initialized according to the distance between each sample point, and let M be an all-one matrix.

[0029] Each entry in the initialized incidence matrix H represents whether the vertex v is included in the hyperedge w, and its expression is as follows:

[0030]

[0031] The distance-based method is used to construct the hypergraph structure of EEG feature data. Select its k nearest neighbor vertices, build a hyperedge connecting the current vertex and these k nearest neighbor vertices, and complete the initialization.

[0032] The specific initialization method of U is as follows: Let d(v i , v j ) represents the distance between any two vertices, Represents vertex v i The average distance to all other vertices, and the vertex weight is given by Determine, let U=diag([u(v1),u(v2),…,u(v n )]) to complete the initialization.

[0033] Preferably, the process of solving the objective function by using the joint iterative optimization method in step 4 is as follows:

[0034] Step 4.1, fix the feature projection matrix M, the hypergraph association matrix H, and the hyperedge weight diagonal matrix W, and update the sample EEG intention decoding result matrix F. After fixing the remaining variables, the original objective function is transformed into the following formula:

[0035]

[0036] Solving the above formula, we can get F.

[0037] Step 4.2, fix the sample EEG intention decoding result matrix F, the hypergraph association matrix H, the hyperedge weight diagonal matrix W, and update the feature projection matrix M. After fixing the remaining variables, the original objective function is transformed into the following formula:

[0038]

[0039] Solving the above formula, we can get M.

[0040] Step 4.3, fix the sample EEG intention decoding result matrix F, feature projection matrix M, hyperedge weight diagonal matrix W, and update the hypergraph association matrix H. The initialization of H is based on the sample distance of X. Here, H is updated based on the sample distance in the feature space after projection, that is, based on M T In addition, based on the projected sample M T X updates U.

[0041] Step 4.4, fix the sample EEG intention decoding result matrix F, feature projection matrix M, hypergraph association matrix H, and update the hyperedge weight diagonal matrix W. After fixing the remaining variables, the original objective function is transformed into the following formula:

[0042]

[0043] Solving the above formula, we can get W.

[0044] Preferably, in step 4.1, step 4.2, and step 4.4, the Lagrangian function is used to solve the sample EEG intention decoding result matrix F, the feature projection matrix M, and the hyperedge weight diagonal matrix W.

[0045] Preferably, in step 4.3, the initialization method for H and U in step 4 is used to update both.

[0046] Preferably, in step 6, the feature projection matrix is ​​processed and analyzed as follows: for each row M(i,:) (i=1, 2, ..., d) of M, the 2-norm is calculated to obtain a d×1-dimensional vector m, and after probabilistic normalization, the relative importance of d features can be obtained. The calculation method is as follows:

[0047]

[0048] In a second aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory stores the computer program; the processor executes the aforementioned EEG speech imagery recognition method based on dynamic hypergraph learning.

[0049] In a third aspect, the present invention provides a readable storage medium storing a computer program; when the computer program is executed by a processor, it is used to implement the aforementioned EEG speech imagery recognition method based on dynamic hypergraph learning.

[0050] The present invention has the following beneficial effects:

[0051] 1. The present invention constructs a model based on a hypergraph. Unlike the traditional graph-based model, the hypergraph is a special graph structure in which a hyperedge can connect multiple vertices, which enables it to better mine information between data, expand the model to a high-dimensional, more complete nonlinear space, and have better correlation modeling capabilities. At the same time, due to the particularity of EEG data, the unpaired relationships between different brain regions, different frequency bands, and different features are difficult to describe using traditional graphs. Therefore, the hypergraph-based model used in the present invention has better information mining capabilities and data representation capabilities in speech imagery EEG decoding than the traditional graph-based model.

[0052] 2. The dynamic hypergraph learning algorithm proposed in the present invention is different from the ordinary hypergraph learning algorithm. The ordinary hypergraph learning algorithm only updates the sample prediction category matrix during the training process, and the hypergraph structure is only constructed when the data is input, and it will not change during the learning process, which makes it impossible to learn the newly mined information during the update process. The dynamic hypergraph learning algorithm proposed in the present invention iteratively updates the vertex weights, hyperedge weights and hypergraph structure in the hypergraph while updating the sample prediction category matrix, which enables it to dynamically learn the information contained in the data, and adaptively adjust the importance of each sample and feature according to the model, thereby improving the accuracy of the learning algorithm in the present invention.

[0053] 3. The present invention introduces a feature projection matrix, which is used to further project the preprocessed sample features into a low-dimensional feature subspace. This feature projection matrix will also be dynamically updated during the training process. The existence of the feature projection matrix can project the high-dimensional feature space into a low-dimensional space, thereby reducing the scale of operations and improving operational efficiency. In addition, it can also reduce the impact of noise and outliers in the original feature space and improve the robustness of the model. The present invention also analyzes the projection matrix to obtain a quantitative index of the importance of each feature in decoding, and constructs a sample feature matrix to improve the model decoding efficiency and its physiological interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a machine learning model framework diagram for dynamic hypergraph learning constructed in the present invention.

[0055] Figure 2 Flow chart of the experimental paradigm of the present invention. DETAILED DESCRIPTION

[0056] The present invention is further described below in conjunction with the accompanying drawings.

[0057] Due to the particularity of brain signals, the present invention believes that using traditional graphs to describe the correlation in EEG data will cause the loss of high-order information, which is not conducive to the decoding of EEG signals. Therefore, it is proposed to use hypergraphs to construct EEG signal models to mine implicit information between data and improve decoding accuracy. At the same time, the present invention dynamically and adaptively updates the vertex weights, hyperedge weights and hypergraph structure in the hypergraph during the learning process of the model, so as to timely capture new information that appears in the updating process and improve the model learning ability. In addition, we introduce a dynamically updated feature projection matrix to project the original sample features into a low-dimensional subspace to enhance the robustness of the model. After training, the feature projection matrix is ​​analyzed to obtain features with higher importance and use them to construct a new sample feature matrix as the input of the decoding model to improve efficiency. The model framework diagram of the present invention is shown in the figure below. Figure 1 shown.

[0058] A method for EEG speech imagery recognition based on dynamic hypergraph learning comprises the following steps:

[0059] Step 1: Collection of EEG data.

[0060] The present invention uses a 64-lead Neuroscan system with a 10-20 international configuration to collect EEG data, and designs a verbal imagery paradigm that includes four categories, namely "eating", "drinking water", "sleeping" and "going to the toilet". The subjects have a 4-second resting time at the beginning of each round of experiments, and then the stimulation is given simultaneously in the form of images and audio. The stimulation ends after 2 seconds. After a 1-second reminder time, the subjects start a 2-second verbal imagery task, repeat the imagination task 4 times, and then enter the next category of experiments. The process is the same as above. In order to ensure the accuracy and effectiveness of the experimental data of the subjects, the number and time of experiments need to be controlled, so 50 rounds of experiments are conducted for each category, and a total of 200 rounds of experimental data are collected for each subject. The time taken for each experiment is controlled to be about 60 minutes, and finally the EEG data of n subjects are obtained. See the experimental paradigm process. Figure 2 .

[0061] Step 2: Preprocess and extract features of the original EEG data.

[0062] After obtaining the raw EEG data collected in step 1, in order to improve the utilization rate and recognition accuracy of the data, the data needs to be preprocessed and feature extracted. First, the notch filter is used to remove the 50Hz power frequency noise, and the eye movement and electromyography artifacts are removed. Then, the EEG data of the five frequency bands are extracted separately using the fifth-order Butterworth filter. The five frequency bands are Delta (0.5-4Hz), Theta (4-8Hz), Alpha (8-13Hz), Beta (13-30Hz) and Gamma (30-100Hz). After that, the features are extracted on the five frequency bands respectively. The extracted features are mainly divided into three categories: statistical features such as skewness and kurtosis, time domain features such as Shannon entropy, and frequency domain features such as power spectrum entropy. For a single sample, all the features extracted above are concatenated into a feature vector. If the number of features is d, a d×1 feature vector is obtained. For all samples, a d×n sample feature matrix X is obtained.

[0063] The n subjects are divided into l labeled subjects and u unlabeled subjects, and the sample matrix is ​​divided into the labeled sample feature matrix And the unlabeled sample feature matrix Get the labeled sample EEG intention category matrix And the unlabeled sample EEG intention category matrix

[0064] Step 3: Construct the objective function.

[0065] To construct the objective function based on the dynamic hypergraph learning algorithm, we first need to use the hypergraph to describe the correlation between samples, and dynamically update the hypergraph vertex weights, hyperedge weights and hypergraph structure. We also need to introduce the feature projection matrix to reduce the dimensionality of sample features.

[0066] For the hypergraph constructed by the model, Indicates that is the sample vertex set, the number of which is n; ε is the hyperedge set, the number of which is m, is the hyperedge weight diagonal matrix. Combining the above requirements, the objective function is constructed as follows:

[0067]

[0068] in is the Laplacian matrix of the hypergraph, which is used to describe the high-order relationship between samples. is the sample’s EEG intention decoding result matrix, is the incidence matrix of the hypergraph, is the diagonal matrix of vertex weights of the sample, is a vertex degree diagonal matrix, whose diagonal elements are is a hyper-degree diagonal matrix, whose diagonal elements are is the feature projection matrix, where f is the number of features after projection, w = (w1, w2, ..., w m ) is a vector representing the hyperedge weight, where w i (i=1, 2, ..., m) is the weight of each hyperedge, I is the unit matrix, and α and β are the regularization coefficients.

[0069] The first term of the objective function is a regularization term that makes the hypergraph smooth in the label space, and also makes the label matrix smooth in the hypergraph structure. It enables samples with strong connections in the hypergraph to share similar decoding results and realize EEG intention category prediction. The second term is a regularization term that makes the hypergraph smooth in the feature space. Combined with the constraint M T XX T M=I, which means learning the projection matrix from the original feature space to the low-dimensional feature space. According to the sample features after projection, the vertex weights and hypergraph structure can be updated. The third term is the 2-norm regularization term of w, which combines the first two terms and the constraint w T 1=1 and w i >0(i=1, 2, ..., m), the weight of each hyperedge can be learned.

[0070] Step 4: Initialize variables and iteratively optimize to solve the objective function.

[0071] Before solving, we need to initialize all variables. Specifically, F l =Y l , F u All elements in each row are initialized to 1 / c, let H be constructed based on the sample distance, let W be the unit matrix, let U be initialized according to the distance between each sample point, and let M be an all-one matrix.

[0072] Now we give the H initialization method. Each item in the association matrix H represents whether the vertex v is included in the hyperedge w. It is constructed by the following formula:

[0073]

[0074] There are three main methods for constructing the hypergraph structure H: distance-based, representation-based, attribute-based, and network-based. We use the distance-based method to construct the hypergraph structure of EEG feature data. Select its k nearest neighbor vertices, build a hyperedge connecting the current vertex and these k nearest neighbor vertices, and complete the initialization.

[0075] At the same time, the initialization method of U is given, let d(v i , v j ) represents the distance between any two vertices, Represents vertex v iThe average distance to all other vertices, and the vertex weight is given by Determine, let U=diag([u(v1),u(v2),…,u(v n )]) to complete the initialization.

[0076] The objective function is solved by using the joint iterative optimization method.

[0077] Step 4.1, fix the feature projection matrix M, the hypergraph association matrix H, and the hyperedge weight diagonal matrix W, and update the sample EEG intention decoding result matrix F. After fixing the remaining variables, the original objective function is transformed into the following formula:

[0078]

[0079] where F l is the known EEG intention category matrix Y of the labeled sample l , no need to update, just update the EEG intention decoding result matrix F of the unlabeled samples u Taking the derivative of the above equation and setting it to 0, we get

[0080] LF=0, (12)

[0081] Re-order have

[0082]

[0083] So there is F u =-L uu -1 L ul F l , and because F l =Y l , and get F u Update formula:

[0084] F u =-L uu -1 L ul Y l . (14)

[0085] Step 4.2, fix the sample EEG intention decoding result matrix F, the hypergraph association matrix H, the hyperedge weight diagonal matrix W, and update the feature projection matrix M. After fixing the remaining variables, the original objective function is transformed into the following formula:

[0086]

[0087] Get the Lagrangian function about M:

[0088]

[0089] Taking the derivative and setting it to 0, we get

[0090]

[0091] Observing formula (17), we can find that each column in M ​​is a matrix (XX T ) -1 αXLXX T Regarding the eigenvector of λ, we know that the update of the projection matrix M can be solved by solving (XX T ) -1 αXLX T to solve for the eigenvector of .

[0092] Step 4.3, fix the sample EEG intention decoding result matrix F, feature projection matrix M, hyperedge weight diagonal matrix W, and update the hypergraph association matrix H. The initialization of H is based on the sample distance of X. Here, H is updated based on the sample distance in the feature space after projection, that is, based on M T The distance between samples of X is constructed by referring to the k nearest neighbor method used in initialization. T X updates U using the same method used during initialization.

[0093] Step 4.4, fix the sample EEG intention decoding result matrix F, feature projection matrix M, hypergraph association matrix H, and update the hyperedge weight diagonal matrix W. After fixing the remaining variables, the original objective function is transformed into the following formula:

[0094]

[0095] make a=diag(A), b=diag(B), equation (18) can be rewritten as:

[0096]

[0097] Further rewriting gives the Lagrangian function about w:

[0098]

[0099] Among them, η≥0, γ≥0 are Lagrange multipliers. Combined with the KKT conditions, the following solution for w can be obtained:

[0100]

[0101] Here (·) + represents that the value in the brackets is positive. Let W = diag(w1, w2, ..., w m ) then we get the updated formula of W.

[0102] Repeat steps 4.1 to 4.4 above, continuously iterating and updating each variable until the objective function value stabilizes within a certain threshold or the number of iterations reaches the set maximum value, and the solution is completed.

[0103] Step 5: Preprocess the measured EEG data in step 2 and extract features to obtain the EEG sample feature matrix X; input the EEG sample feature matrix X into the model to obtain the preliminary predicted EEG intention decoding result matrix F of the unlabeled sample u And the feature projection matrix M.

[0104] Step 6: After the feature projection matrix M is obtained through the training in the above steps, the importance of each channel and each frequency band in the decoding of speech imagery EEG intention can be obtained by processing and analyzing M. The processing and analysis method is as follows:

[0105] For each row M(i,:) (i=1,2,…,d) of M, find its L2 norm, the formula is as follows:

[0106]

[0107] Then the L2 norm of each row is normalized as a measure of importance, and a d×1-dimensional vector m representing the importance of each feature is obtained. The calculation formula is as follows:

[0108]

[0109] The above method can be used to obtain quantitative indicators of the relative importance of d features. Based on this, the importance of each channel and each frequency band can also be obtained, which helps to improve the physiological interpretability of this model. The method is as follows:

[0110] Assume that each sample feature vector is organized in the following form: 64-channel features in the Delta band, 64-channel features in the Theta band, 64-channel features in the Alpha band, 64-channel features in the Beta band, and 64-channel features in the Gamma band, and the total number of extracted statistical, time domain, and frequency domain features is p.

[0111] The importance of each frequency band is calculated as follows, where i = 1, 2, ..., 5:

[0112] fre(i)=m((i-1)×p×64+1)+m((i-1)×p×64+2)+…+m(i×p×64) (24)

[0113] The importance of each channel is calculated as follows, where j = 1, 2, ..., 64:

[0114]

[0115] Thus, the relative importance of each frequency band and each channel in the EEG intention decoding of speech imagery is obtained. Finally, the features with importance higher than the specified value are selected to construct the sample feature matrix, which is then input into the decoding model for training to obtain the predicted EEG intention decoding result matrix F of the unlabeled sample. u , which is the prediction result of the sample, and the known EEG intention category matrix Y of the unlabeled sample u The prediction accuracy can be obtained by comparison, thus achieving high-precision and high-efficiency EEG intention decoding based on dynamic hypergraph learning.

Claims

1. A method for EEG speech imagery recognition based on dynamic hypergraph learning, characterized in that: The steps include: Step 1: Collect labeled EEG data of multiple subjects under different speech imagery tasks; Step 2: preprocessing and feature extraction of the measured EEG data and the labeled EEG data to obtain a sample feature matrix; sample The feature matrix is ​​divided into the labeled sample feature matrix X l And the unlabeled sample feature matrix X u ; Step 3: Use a hypergraph to describe the correlation between samples, dynamically update the hypergraph vertex weights, hyperedge weights and hypergraph structure, and introduce a feature projection matrix to reduce the dimension of sample features to construct the objective function of the machine learning model based on dynamic hypergraph learning; The constructed objective function is as follows: in, is the Laplacian matrix of the hypergraph, which is used to describe the high-order relationship between samples; is the sample’s EEG intention decoding result matrix; W is the hyperedge weight diagonal matrix; H is the hypergraph’s association matrix, U is the sample’s vertex weight diagonal matrix, and D v is a vertex degree diagonal matrix, whose diagonal elements are is a hyper-degree diagonal matrix, whose diagonal elements are is the feature projection matrix, where f is the number of features after projection, is a vector representing the hyperedge weight, where is the weight of each hyperedge, X l is the labeled sample feature matrix, Y l is the corresponding sample EEG intention category matrix, I is the identity matrix, α and β are the regularization coefficients; Step 4: Solve the variables in the objective function to obtain the feature projection matrix and the EEG intention decoding result matrix; based on the EEG intention decoding result matrix, obtain the speech imagination category corresponding to the measured EEG data.

2. The method for EEG speech imagery recognition based on dynamic hypergraph learning according to claim 1, characterized in that: In step 4, after obtaining the feature projection matrix and the EEG intention decoding result matrix, a feature importance vector m describing the importance of all dimensions in the feature vector is established according to the feature projection matrix; According to the feature importance vector m, the relative importance of each channel and frequency band of EEG acquisition is calculated; according to the importance, some feature vectors are extracted to construct a new sample feature matrix; the new sample feature matrix is ​​input into the objective function for solution to obtain the updated EEG intention decoding result matrix.

3. The method for EEG speech imagery recognition based on dynamic hypergraph learning according to claim 2, characterized in that: The method for calculating the relative importance of each frequency band and each channel according to the feature importance vector m is as follows: The expression for calculating the importance of each frequency band, frequent(i), is as follows: Among them, i is the frequency band number; p is the number of feature categories extracted, and ch is the number of channels; The expression for calculating the importance of each channel channel(j) is as follows: Among them, fre is the number of frequency bands.

4. The method for EEG speech imagery recognition based on dynamic hypergraph learning according to claim 1, characterized in that: There are three types of extracted features, namely statistical, time domain, and frequency domain features; there are five extracted frequency bands, namely Delta band, Theta band, Alpha band, Beta band, and Gamma band; there are 64 extracted channels.

5. The method for EEG speech imagery recognition based on dynamic hypergraph learning according to claim 1, characterized in that: The process of preprocessing and feature extraction of EEG data in step 2 is as follows: first, notch filtering is performed to remove 50 Hz power frequency noise, and to remove eye movement and electromyographic artifacts, and then a fifth-order Butterworth filter is used to extract the data of each target frequency band; then, statistical, time domain, and frequency domain feature extraction are performed on the data of each target frequency band.

6. The method for EEG speech imagery recognition based on dynamic hypergraph learning according to claim 1, characterized in that: In step 1, visual stimulation and auditory stimulation of speech imagination are applied to the subjects at the same time to obtain EEG data under different speech imagination tasks.

7. The method for EEG speech imagery recognition based on dynamic hypergraph learning according to claim 1, characterized in that: The process of solving the objective function in step 4 is: after initializing the variables in the objective function, the EEG intention decoding result matrix F, feature projection matrix M, hypergraph association matrix H and hyperedge weight matrix W are solved by using the joint iterative optimization method.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The memory stores computer programs; the processor executes the EEG speech imagery recognition method based on dynamic hypergraph learning as described in any one of claims 1-7.

9. A readable storage medium storing a computer program; characterized in that: When the computer program is executed by a processor, it is used to implement the EEG speech imagery recognition method based on dynamic hypergraph learning as described in any one of claims 1 to 7.

Images