Cross-subject speech imagination electroencephalogram intention decoding domain migration method based on collaborative clustering

Abstract

The invention discloses a collaborative clustering-based cross-subject speech imagination electroencephalogram intention decoding domain migration method, which comprises the following steps of: 1, acquiring existing speech imagination electroencephalogram data so as to construct a source domain data set and pre-process the source domain data set; 2, extracting speech imagination-related electroencephalogram features in the preprocessed source domain data set; 3, initializing a clustering indication matrix of the samples and a clustering indication matrix of the features, and solving and optimizing; 4, training a semi-supervised speech imagination intention decoding model based on electroencephalogram samples and feature collaborative clustering; step 5, collecting target domain data, and performing preprocessing; step 6, determining optimal source domain matching of the target domain; step 7, obtaining an adjusted coefficient matrix; and step 8, obtaining a final optimized sample feature coupling clustering electroencephalogram speech imagination intention decoding matrix decomposition model on the target domain, and then converting an identification tag of intention decoding into a clustering indication matrix of a target domain sample and solving the clustering indication matrix.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal processing, and in particular to a method for cross-subject speech imagery EEG intention decoding domain migration based on collaborative clustering. Background Art

[0002] Speech imagery intention decoding is a cutting-edge research topic. Its significance and value lie in its ability to deeply explore the mechanism of human brain processing and generating language, and to open up a new way of brain-computer interface interaction. Speech imagery, that is, the inner pronunciation in a silent state, represents the individual's internal language experience. By decoding these internal sounds, researchers have the opportunity to convert the imagined speech in people's brains into external recognizable signals or sounds, which has far-reaching implications in both basic research and practical applications. At the basic research level, speech imagery intention decoding provides us with a unique window to understand the language preprocessing mechanism of the brain. This technology can reveal how the brain processes, stores and generates language in a silent state, thereby deepening our understanding of cognitive processes. At the application level, the potential of speech imagery intention decoding technology is even broader. For patients who have lost their ability to speak due to diseases such as stroke, Parkinson's disease, and brain trauma, this technology provides new hope for restoring the ability to communicate. By directly converting the speech intention in the brain into sound or text, patients can communicate with others again without relying on traditional vocal organs. In addition, this technology can also be applied to fields such as encrypted communication and brain pattern recognition, providing people with a way of communication that does not rely on language production. In general, speech imagery intention decoding technology not only promotes our understanding of the brain's cognitive mechanisms, but also brings revolutionary application prospects to medical rehabilitation, silent communication and other fields.

[0003] Among them, speech imagery intention decoding based on EEG signals is a technology that identifies and decodes an individual's internal speech imagery by analyzing electroencephalogram (EEG) signals. It has attracted much attention due to its advantages of low cost, ease of use, convenient collection and high detection accuracy. The core of this technology is to use the language processing and generation information contained in EEG signals to infer an individual's speech intention, and to achieve effective communication even without making a sound, which broadens the application scenarios of the brain-computer interface (BCI) system and demonstrates broad application prospects and promotion value.

[0004] However, although EEG signals are cheap and easy to obtain, they are typical non-stationary signals with low signal-to-noise ratio and unclear data features. In addition, there are significant differences in EEG signals across time periods or subjects. Models trained on existing speech imagery EEG datasets often have difficulty showing the same performance on new subjects or new time periods. In this case, directly applying the trained model to the newly collected EEG data usually leads to a significant reduction in the accuracy of speech imagery intention decoding, which in turn affects the practical application value of the model.

[0005] In response to the above technical problems, there are mainly two coping strategies in the currently known existing technologies.

[0006] The first is to retrain a speech imagery intention decoding model for new subjects and new time periods. Although this method can be customized and optimized for new data to improve decoding accuracy, it also has significant disadvantages: retraining the model is not only time-consuming and labor-intensive, but the sample collection process often consumes a lot of time and human resources, resulting in high time costs. In addition, this method requires repeated training for each new subject or time period, and cannot be effectively promoted and applied.

[0007] Another approach is to perform transfer learning based on existing data sets and trained models. That is, based on the newly collected data, the previous models and data are used to adjust and optimize them to adapt to the new data.

[0008] Among them, in transfer learning, the source domain refers to the domain where we already have a large amount of labeled data and trained models, providing knowledge and experience for training preliminary models; the target domain is the domain to which we hope to migrate the source domain model, which may have limited labeled data or even no labeled data. The task of the target domain is to apply the knowledge already acquired from the source domain to a new environment in order to achieve similar learning effects or improve the performance of the target task. The feature distribution and tasks of the target domain may be different from those of the source domain, which is also the challenge of transfer learning. Domain Adaptation is an important process in transfer learning, which aims to reduce the distribution difference between the source domain and the target domain. Specifically, domain adaptation attempts to adjust or optimize the model so that it can generalize better on the target domain.

[0009] Existing domain adaptation technologies are mainly divided into the following two categories:

[0010] Model Adjustment: Based on the model trained in the source domain, adjustments are made to the target domain. Specifically, fine-tuning is performed using a small amount of labeled data from the target domain to optimize the performance of the model in the target domain. The disadvantage of this technology is that it still relies on a large amount of labeled data for calibration, which still requires high costs and efforts in the actual application of EEG data.

[0011] Distribution Alignment: Through optimization techniques, the original feature distribution of the source domain and the target domain or the distribution in the feature space after encoding dimensionality reduction is adjusted to align them as much as possible. The probability distribution distance between the source domain and the target domain is minimized to make the distribution of the source domain and the target domain as similar as possible. Commonly used tools to measure the probability distribution distance include KL divergence, JS distance, and maximum mean difference (also known as MMD distance). Representatives of this type of technology include transfer component analysis (TCA), joint distribution adaption (JDA), transfer joint matching (TJM), etc. However, the problem with this technology is that the source domain and the target domain may not be compatible. Blind distribution alignment will cause the "negative transfer" phenomenon, resulting in unstable transfer learning effects.

[0012] In addition, the domain adaptation process in traditional transfer learning usually involves complex algorithm adjustments. How to improve the interpretability and complexity of the migration process is also a technical pain point that still needs to be solved.

[0013] Based on the above defects, it brings great challenges to EEG speech imagery decoding. The non-stationarity of EEG signals means that their statistical properties (such as mean and variance) change over time, which makes the analysis and decoding of signals more complicated. Brain activity may have different electrical activity patterns at different time points, resulting in changes in signal characteristics, making stable and reliable decoding difficult. Therefore, the characteristics of EEG signals may change between different experimental periods, which may be due to changes in experimental conditions, changes in subject status, or other environmental factors. This period change increases the complexity of model training and decoding, making real-time and long-term stable decoding a major challenge. At the same time, there are natural differences in brain electrical activity patterns of different individuals. This difference between individuals makes it difficult to generalize the decoding model across subjects. Each person's EEG signal may differ in structure and dynamic characteristics, making the generalization effect of the speech imagery intention decoding model based on EEG signals poor.

[0014] Therefore, it is particularly important to construct a universal intention decoding model that can handle the individual differences and time period differences of these speech imagery EEG data and has good transferability among different subjects and different time periods. Summary of the invention

[0015] According to actual needs and in view of the shortcomings of existing speech imagery intention decoding methods based on EEG signals, the present invention proposes a cross-subject speech imagery EEG intention decoding domain transfer method based on collaborative clustering, aiming to construct a transfer learning pre-adaptation method to simply and effectively deal with the differences in EEG signals across time periods and across subjects. Based on the semi-supervised speech imagery intention decoding algorithm based on EEG samples and feature collaborative clustering, the present invention studies the feature and sample coupling analysis of cross-subject EEG data distribution alignment, solves the problem of EEG signal differences between different individuals, and improves the universality of the decoding method. Through the development of the above content, the present invention hopes to provide technical support for a new brain-computer interface interaction mode based on EEG signals and speech imagery, and further improve the application potential and practical value of the brain-computer interface.

[0016] In order to solve the above technical problems, the technical solution of the present invention is:

[0017] A method for domain transfer of speech imagery EEG intention decoding across subjects based on collaborative clustering includes the following steps:

[0018] Step 1: Obtain existing speech imagery EEG data to construct a source domain dataset and preprocess it. The source domain dataset X s Contains m source domain data subsets;

[0019] Step 2, extract the EEG features related to speech imagination in the preprocessed source domain data set, and then define the clustering indicator matrix of the features and the clustering indicator matrix of the samples as and coefficient matrix according to the label type;

[0020] Step 3, using the prior speech imagination intention label information to initialize the clustering indicator matrix of the sample corresponding to the labeled sample, and initialize the clustering indicator matrix of the sample corresponding to the unlabeled sample and the clustering indicator matrix of the feature by random initialization; after completing the initialization, use the co-clustering algorithm to solve the clustering indicator matrix of the feature, the clustering indicator matrix of the sample, and the coefficient matrix in an alternating manner, so as to obtain the optimized clustering indicator matrix of the feature, the clustering indicator matrix of the sample, and the coefficient matrix;

[0021] Step 4: Use the optimized feature clustering indicator matrix, sample clustering indicator matrix, and coefficient matrix to train a semi-supervised speech imagery intention decoding model based on EEG sample and feature co-clustering. For each data subset X si Get a corresponding independent decoding model;

[0022] Step 5: Collect the labeled EEG data of the subjects with speech imagination intention labels in real time at any time period, that is, the target domain data, and preprocess them to obtain the target domain data set X t ;

[0023] Step 6: Traverse all source domain data subsets X si and its corresponding cluster indicator matrix and coefficient matrix of optimized features to determine the best source domain match for the target domain;

[0024] Step 7: For the best matching source domain data subset X s* The clustering indicator matrix and coefficient matrix of the features are adjusted to obtain the adjusted coefficient matrix;

[0025] Step 8: In the target domain, according to the clustering indicator matrix P of the features s* And the adjusted coefficient matrix S s* The final optimized sample feature coupled clustering EEG speech imagery intention decoding matrix decomposition model is obtained, and then the recognition label of intention decoding can be converted into the clustering indicator matrix of the target domain sample;

[0026] Step 9: Solve the cluster indicator matrix of the target domain samples to obtain the decoding results.

[0027] Preferably, in step 1, the preprocessing method of the source domain data set includes denoising and filtering, so as to obtain EEG data with artifacts and noise removed to ensure data quality and consistency.

[0028] Preferably, in step 2, it is assumed that the data matrix X consists of labeled samples X l and unlabeled samples X u Composition, which contains labeled samples X l The number of unlabeled samples is l, u The number of is u, and the original data matrix is ​​rearranged into The corresponding sample cluster indicator matrix Q can be expressed as Among them, Q l ∈Φ l×c , Q u ∈Φ (n-l)×c , where c is the number of categories.

[0029] In this co-clustering algorithm:

[0030] The matrix P represents the feature clustering indicator matrix, which is d×c in size, where d is the total number of features and c is the number of clustering categories. Each row of the matrix P corresponds to a feature, and each column corresponds to a clustering category. The element P ij It represents the indicator value of the second feature in the jth category, which is 1 if the feature belongs to this category and 0 otherwise.

[0031] The matrix Q represents the feature cluster indicator matrix, which is n×c in size, where n is the total number of samples and c is the number of cluster categories (consistent with the number of categories in the feature cluster indicator matrix PPP). Each row of the matrix Q corresponds to a sample and each column corresponds to a cluster category. j It represents the indicator value of the i-th sample in the j-th category, which is 1 if the feature belongs to this category, otherwise it is 0.

[0032] The matrix S represents the coefficient matrix, usually a diagonal matrix of size c×c. The diagonal elements S j Represents the weight or importance of each cluster category. Since S∈diag, it means that S is a diagonal matrix with zero non-diagonal elements, and only the weights corresponding to each cluster category on the diagonal are retained.

[0033] The parameter c refers to the number of cluster categories. In speech imagery tasks, c is usually equal to the number of intent categories. For example, if the speech imagery categories include four intents: "up", "down", "left", and "right", then c = 4.

[0034] The matrix Q works together with the feature clustering indicator matrix P and the coefficient matrix S to find the best clustering method for features and samples by optimizing the reconstruction error, so that the model can effectively distinguish different categories of speech imagination intentions.

[0035] Preferably, the sample clustering indicator matrix Q′ of the labeled samples is initialized as follows:

[0036] For the i-th labeled sample (i∈{1, 2, ..., l}), its corresponding indicator vector satisfies:

[0037]

[0038] Preferably, the sample clustering indicator matrix Q of the unlabeled samples is u The initialization method is: for the i-th labeled sample (i∈{l+1, l+2, ..., n}), the corresponding indicator vector is randomly assigned to satisfy:

[0039]

[0040] Each unlabeled sample takes the value of 1 only in one randomly selected category and the values ​​of the other categories are 0.

[0041] Preferably, the cluster indicator matrix of the feature is initialized by: also using a random initialization method, taking a value of 1 only in a randomly selected category, and taking values ​​of 0 in the remaining categories.

[0042] Preferably, the method for solving the clustering indicator matrix of the features, the clustering indicator matrix of the samples, and the coefficient matrix is:

[0043] First, the coefficient matrix is ​​solved by fixing the clustering indicator matrix of the features and the clustering indicator matrix of the samples. The formula is as follows:

[0044] s=H -1 r

[0045] Where H = P T PQ T Q, r = P T XQ, where P is the cluster indicator matrix of features, Q is the cluster indicator matrix of samples, and S is the coefficient matrix;

[0046] Then, only the sample clustering indicator matrix Q corresponding to the unlabeled samples is u Iterate and calculate, while the labeled sample part Ql remains unchanged, and the cluster indicator matrix and system matrix of the fixed feature are calculated. u ,

[0047] Thanks to Q u Each vector q in i Only one element in Q is equal to 1, and the rest are 0, so Q u The solution can be determined by the following formula:

[0048]

[0049] where r k is the kth column of R, R = PS;

[0050] Finally, when the clustering indicator matrix and coefficient matrix of the sample are fixed, the clustering indicator matrix of the feature is solved. The solution formula is as follows:

[0051]

[0052] Among them l k is the kth row of L, L = SQ T ;

[0053] Repeat the above solution process, alternately solve the optimization, and iterate until convergence, so as to obtain the cluster indicator matrix P of the optimized feature i , the cluster indicator matrix Q of the sample i , coefficient matrix S i .

[0054] Preferably, in step 4, the method for training a semi-supervised speech imagery intention decoding model based on EEG samples and feature co-clustering is:

[0055] Based on a semi-supervised sparse neighbor-constrained co-clustering algorithm based on matrix decomposition, the original data matrix X, where the number of samples is n and the number of features is d, is subjected to feature clustering and sample clustering. The optimization objective formula of the co-clustering is as follows:

[0056]

[0057] Among them, P represents the clustering indicator matrix of features, Q represents the clustering indicator matrix of samples, and S is the coefficient matrix used to connect P and Q. For each data subset X si (i∈{12,...,N s}) to obtain a corresponding independent decoding model. The symbol T represents the transpose operation of the matrix, the symbol F represents the Frobenius norm, d represents the number of features, c represents the number of clusters, n represents the number of samples, diag represents a diagonal matrix, and S∈diag represents that S is a diagonal matrix.

[0058] Preferably, in step 5, the target domain data is preprocessed by:

[0059] First, the collected EEG data is downsampled, and the signals corresponding to the bad electrodes are removed and regenerated by interpolating adjacent electrodes; the EEG is then filtered to 0.1-50Hz through a bandpass filter to remove noise and eliminate the interference of noise on the classification accuracy; then the time-frequency-space multi-dimensional features are extracted, connected, sorted and standardized.

[0060] Preferably, in step 6, the subset of speech imagery EEG data collected in the target domain is X t , from the labeled sample X tl and unlabeled samples X tu The intent decoding label corresponding to the labeled sample is Y tl , then let the cluster indicator matrix Q of the target domain samples tl =Y tl .

[0061] Preferably, in step 6, all source domain data subsets X are traversed. si and its corresponding matrix P i and S i , by calculating the target domain data X t The approximate matrix PSQ output by the decoding model corresponding to the subset of source domain data T The F norm between Select the source domain subset X that minimizes the norm s* , thus obtaining the source domain data subset X s* The corresponding feature cluster indicator matrix P s* and the system matrix S s*, and takes it as the best source domain match for the target domain, and then as the object of domain adaptation.

[0062] Preferably, the source domain data subset X s* The corresponding feature cluster indicator matrix P s* and the system matrix S s* The adjustment method is to fix the cluster indicator matrix P of the feature s* , only adjust the coefficient matrix S s* , the adjustment formula is as follows:

[0063] S s* ′=H -1 r

[0064] in

[0065] Preferably, the final optimized sample feature coupled clustering EEG speech imagery intention decoding matrix decomposition model is expressed as:

[0066] X t ≈P s *S′ s *Q t .

[0067] Preferably, in step 8, the identification label intended to be decoded can be converted into a clustering indicator matrix Q of the target domain sample: t Solve, the cluster indicator matrix Q of the target domain samples t The solution is: Since Q t Each vector q in i Only one element in Q is equal to 1, and the rest are 0, so Q t The solution can be determined by the following formula:

[0068]

[0069] where r k R = P s *S′ s* The kth column of .

[0070] Preferably, the method further includes step 10, wherein the clustering indicator matrix Q of the target domain samples is t Visualize the solution results.

[0071] The present invention has the following characteristics and beneficial effects:

[0072] According to actual needs, the present invention establishes a domain migration system for speech imagery EEG intention decoding across subjects and time periods based on collaborative clustering to address the problems of poor universality and insufficient generalization ability of existing speech imagery intention decoding models.

[0073] Compared with directly applying the trained model or retraining on the speech imagery EEG data collected from new subjects or new time periods, the present invention adopts the idea of ​​transfer learning, fully utilizes the source domain information of existing data sets, effectively transfers knowledge between different domains, improves the accuracy and robustness of the model, and saves time and resources for model construction and training.

[0074] Compared with the existing transfer learning algorithms, the domain transfer system based on collaborative clustering proposed in the present invention selects the most matching source domain from the perspective of minimizing the loss of collaborative clustering matrix decomposition. While improving the transfer effect, it has lower algorithm complexity and higher interpretability.

[0075] Compared with the existing domain adaptation algorithms, the cross-subject and cross-time period speech imagery EEG intention decoding domain transfer system based on collaborative clustering proposed in the present invention does not involve complex multi-round learning and parameter adjustment processes in the process of optimizing the source domain decoding model for target domain data, and requires a smaller number of labeled samples, thereby improving the response and optimization speed of domain adaptation. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0077] Figure 1 It is a structural schematic diagram of a cross-subject speech imagery EEG intention decoding domain migration system based on collaborative clustering according to an embodiment of the present invention.

[0078] Figure 2 The present invention is a flowchart of the domain selection and domain adaptation modules in the cross-subject speech imagery EEG intention decoding domain transfer method based on collaborative clustering in an embodiment of the present invention. DETAILED DESCRIPTION

[0079] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0080] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings.

[0081] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0082] The present invention provides a method for transferring the EEG intention decoding domain of speech imagery across subjects based on collaborative clustering. Figure 1 and Figure 2 As shown in the figure, firstly, a source domain data preprocessing module is applied to process and manage the existing speech imagery EEG dataset, which constitutes the source domain of the whole system. The specific functions include data preprocessing, feature extraction and model training.

[0083] The specific steps are as follows:

[0084] Step 1: Obtain existing speech imagery EEG data to construct a source domain dataset and preprocess it. The source domain dataset X s Contains m source domain data subsets;

[0085] In preprocessing, EEG data undergoes denoising, filtering and other processing steps to remove artifacts and noise and ensure the quality and consistency of the data.

[0086] Step 2: Then, through feature extraction, the EEG features related to speech imagination are extracted, and then the clustering indicator matrix of the features and the clustering indicator matrix of the samples are defined as and coefficient matrix according to the label type.

[0087] In this embodiment, it is assumed that the data matrix X consists of labeled samples X l and unlabeled samples X u Composition, which contains labeled samples X l The number of unlabeled samples is l, u The number of is u, and the original data matrix is ​​rearranged into The corresponding sample cluster indicator matrix Q can be expressed as Among them, Q l ∈Φ l×c , Q u ∈Φ (n-l)×c , where c is the number of categories.

[0088] Specifically, in this embodiment, in this co-clustering algorithm:

[0089] The matrix P represents the feature clustering indicator matrix, which is d×c in size, where d is the total number of features and c is the number of clustering categories. Each row of the matrix P corresponds to a feature, and each column corresponds to a clustering category. The element P ijIt represents the indicator value of the second feature in the jth category, which is 1 if the feature belongs to this category and 0 otherwise.

[0090] The matrix Q represents the feature cluster indicator matrix, which is of size n×c, where n is the total number of samples and c is the number of cluster categories (consistent with the number of categories in the feature cluster indicator matrix PPP). Each row of the matrix Q corresponds to a sample and each column corresponds to a cluster category. j It represents the indicator value of the i-th sample in the j-th category, which is 1 if the feature belongs to this category, otherwise it is 0.

[0091] The matrix S represents the coefficient matrix, which is usually a diagonal matrix with a size of c×c. The diagonal elements S j Represents the weight or importance of each cluster category. Since S∈diag, it means that S is a diagonal matrix with zero non-diagonal elements, and only the weights corresponding to each cluster category on the diagonal are retained.

[0092] The parameter c refers to the number of cluster categories. In speech imagery tasks, c is usually equal to the number of intent categories. For example, if the speech imagery categories include four intents: "up", "down", "left", and "right", then c = 4.

[0093] The matrix Q works together with the feature clustering indicator matrix P and the coefficient matrix S to find the best clustering method for features and samples by optimizing the reconstruction error, so that the model can effectively distinguish different categories of speech imagination intentions.

[0094] Step 3: In order to make full use of the label information of speech imagery EEG data and further improve the quality and interpretability of clustering results, the optimization solution of P, Q, and S is completed under a semi-supervised framework.

[0095] First, the clustering indicator matrix P of the features and the clustering indicator matrix Q of the samples need to be initialized.

[0096] Specifically, unlike the traditional method of initializing P and Q as arbitrary category indicator matrices, this module will make full use of the prior speech imagination intention label information when initializing the Q matrix. First, the cluster category is fixed to the speech imagination intention label category. In this embodiment, the speech imagination intention label is defined as the words "up", "down", "left", and "right", and the number of fixed cluster categories c=4.

[0097] The clustering indicator matrix Q of the samples corresponding to the labeled samples is initialized using the prior speech imagination intention label information l ,

[0098] For the i-th labeled sample (i∈{1, 2, ..., l}), its corresponding indicator vector satisfies:

[0099]

[0100] This initialization method ensures that the category affiliation of labeled samples during the clustering process is consistent with their prior labels.

[0101] Furthermore, the clustering indicator matrix of the sample and the clustering indicator matrix of the feature corresponding to the unlabeled sample are initialized by random initialization.

[0102] For the i-th labeled sample (i∈{l+1, l+2, ..., n}), its corresponding indicator vector is randomly assigned to satisfy:

[0103]

[0104] Among them, each unlabeled sample takes the value of 1 only in one randomly selected category, and the values ​​of the other categories are 0. Such random initialization provides a starting point for subsequent iterative optimization.

[0105] The cluster indicator matrix of the feature is initialized by random initialization, with the value of 1 being taken only in one randomly selected category and the values ​​of the other categories being taken as 0.

[0106] In this way, the label information of the sample is directly embedded in the co-clustering algorithm, providing a guide for the algorithm, thereby maintaining sensitivity to the sample intent during the clustering process. This initialization method can better combine the label information of the data with the clustering process, so that the final clustering result not only reflects the intrinsic structure of the data, but also retains and highlights the intent information of the sample.

[0107] After initialization, the co-clustering algorithm is used to solve the clustering indicator matrix of the features, the clustering indicator matrix of the samples, and the coefficient matrix in an alternating manner, thereby obtaining the optimized clustering indicator matrix of the features, the clustering indicator matrix of the samples, and the coefficient matrix.

[0108] Specifically, first, the coefficient matrix is ​​solved when the clustering indicator matrix of the features and the clustering indicator matrix of the samples are fixed. The formula is as follows:

[0109] s=H -1 r

[0110] Where H = P T PQ T Q, r = P T XQ, where P is the cluster indicator matrix of features, Q is the cluster indicator matrix of samples, and S is the coefficient matrix;

[0111] Then, only the sample clustering indicator matrix Q corresponding to the unlabeled samples is u Iterate and calculate, and there are labeled sample parts Ql Keeping the cluster indicator matrix and system matrix unchanged, calculate Q u ,

[0112] Thanks to Q u Each vector q in i Only one element in Q is equal to 1, and the rest are 0, so Q u The solution can be determined by the following formula:

[0113]

[0114] where r k is the kth column of R, R = PS; in this process, x i represents the feature vector of the i-th unlabeled sample, and we determine the category of the sample by minimizing the Euclidean distance between the sample and all category centers. u The iterative calculation can effectively utilize the potential structural information of unlabeled samples while keeping the category indication of labeled samples unchanged, thereby further optimizing the clustering results of unlabeled samples in the co-clustering process.

[0115] Finally, when the clustering indicator matrix and coefficient matrix of the sample are fixed, the clustering indicator matrix of the feature is solved. The solution formula is as follows:

[0116]

[0117] Among them l k is the kth row of L, L = SQ T ;

[0118] Repeat the above solution process, alternately solve the optimization, and iterate until convergence. The algorithm gradually optimizes the clustering results of features and samples, so that feature clustering and sample clustering can promote each other and ultimately achieve better clustering effects.

[0119] After the above steps, the data subset X s The corresponding decoding model is optimized to obtain the final feature clustering indicator matrix P i , the cluster indicator matrix Q of the sample i , coefficient matrix S i These matrices not only reflect the intrinsic structure of the data subset, but also retain the speech imagery intention information of the samples.

[0120] Step 4: Specifically, a semi-supervised speech imagery intention decoding model based on EEG sample and feature co-clustering is trained using the optimized feature clustering indicator matrix, sample clustering indicator matrix, and coefficient matrix. For each data subset X si Get a corresponding independent decoding model for each data subset Xsi (i∈{1,2,...,N s}) corresponds to an independent decoding model.

[0121] Specifically, based on a semi-supervised sparse neighbor-constrained co-clustering algorithm based on matrix decomposition, the original data matrix X (with n samples and d features) is subjected to feature clustering and sample clustering, and the optimization objective formula of the co-clustering is as follows:

[0122]

[0123] Among them, P represents the clustering indicator matrix of features, Q represents the clustering indicator matrix of samples, and S is the coefficient matrix used to connect P and Q.

[0124] In a further configuration of this embodiment, the target domain data acquisition module is then used to collect new EEG data from the target domain, which data is usually collected from different subjects or in different time periods from the source domain. The functions of the target domain data acquisition module include real-time collection of EEG signals and data preprocessing.

[0125] The specific steps are as follows:

[0126] Step 5: Collect the labeled EEG data of the subjects with speech imagination intention labels in real time at any time period, that is, the target domain data, and preprocess them to obtain the target domain data set X t

[0127] Specifically, the collected EEG signals are labeled EEG data with speech imagination intention labels. The preprocessing process includes downsampling the collected EEG data or existing EEG data, extracting the signals corresponding to the bad electrodes, and regenerating them through adjacent interpolation; then filtering the EEG to 0.1-50Hz through a bandpass filter to remove noise and remove the interference of noise on the classification accuracy as much as possible; then extracting time-frequency-space multi-dimensional features for connection, organization and standardization. This module ensures that high-quality EEG data is obtained in the target domain as the basis for subsequent transfer learning and model optimization. At the same time, these data also provide an important reference for the domain adaptation process to help adjust and optimize the performance of the model in the target domain.

[0128] In a further configuration of this embodiment, the best source domain match is selected through the domain selection module, and then used as the object of domain adaptation.

[0129] The specific steps are as follows:

[0130] Step 6: Let the subset of speech imagery EEG data collected in the target domain be X t , from the labeled sample X tl and unlabeled samples X tuThe intent decoding label corresponding to the labeled sample is Y tl , then let the cluster indicator matrix Q of the target domain samples tl =Y tl . It is known that for each data subset X si (i∈{1,2,…,N s}) corresponds to an independent decoding model, and the corresponding P i , Q i and S i matrix.

[0131] Then, the most appropriate source domain is selected by minimizing the reconstruction loss. Traverse all source domain data subsets X si and its corresponding matrix P i and S i , calculate the target domain data X t The approximate matrix PSQ output by the decoding model corresponding to the subset of source domain data T The F norm between Select the source domain subset X that minimizes the norm s*， Thus, we get the source domain data subset X s* The corresponding feature cluster indicator matrix P s* and the system matrix S s* , and takes it as the best source domain match for the target domain, and then as the object of domain adaptation.

[0132] A further configuration of this embodiment is to use a pre-adaptation module to determine the best matching source domain X s* After that, we need to s* Co-clustering model matrix P s* and S s* Make certain adjustments and perform domain adaptation.

[0133] Specifically, the steps are as follows:

[0134] Step 7: In order to simplify the algorithm complexity of the domain adaptation process, we fix the clustering indicator matrix P of the feature s* , only adjust the coefficient matrix S s* , the adjustment formula is as follows:

[0135] S s* ′=H -1 r

[0136] in

[0137] Step 8. Therefore, in the target domain, the sample feature coupling clustering EEG speech imagination intention decoding matrix decomposition model finally optimized can be expressed as:

[0138] X t ≈Ps *S′ s *Q t

[0139] Step 9: Then, the identification label of the intended decoding can be converted into the clustering indicator matrix Q of the target domain sample t The solution process is as follows:

[0140]

[0141] where r k R = P s *S′ s* The kth column of .

[0142] Finally, the application results are displayed and the decoding application module is used. This module plays a core role in the final output, evaluation and application in the cross-subject speech imagery EEG intention decoding domain transfer and distribution alignment system based on collaborative clustering. This module will combine the decoding results with the actual application scenarios, and provide users with intuitive system performance feedback and practical application interfaces through visualization, performance evaluation and decoding solution deployment. First, this module includes a decoding result visualization function. In the cross-subject speech imagery task, the decoding results usually include accurate predictions of intent, the EEG signal feature clustering results of each subject and their distribution. In order to better understand and analyze these data, this module will provide multi-dimensional visualization tools, including:

[0143] Time series graph: shows the changing trend of decoded EEG signals in different time periods, helping researchers analyze the timing characteristics of different intentions.

[0144] Clustering result display: The clustering results of subject samples and features are displayed through two-dimensional or three-dimensional projection diagrams, which intuitively reflect the distribution patterns of different subjects and intentions.

[0145] Label comparison chart: compares the matching degree between the decoded labels and the true labels, showing the decoding accuracy and error distribution of each subject.

[0146] This module provides a variety of evaluation indicators to quantify the model's migration ability and decoding accuracy across subjects and time periods, including accuracy, confusion matrix, domain adaptation time, model response speed, etc.

[0147] The module provides an interface for use in brain-computer interface (BCI) applications to decode the user's verbal imagery intentions in real time for controlling external devices (such as wheelchairs, prostheses, etc.) or conducting neurorehabilitation training.

[0148] According to the records of this embodiment, it can be understood that the domain transfer system for cross-subject and cross-time period speech imagery EEG intention decoding established in this embodiment can select the most suitable source domain for transfer from multiple source domains under the condition that only a small number of target domains have labeled samples, effectively avoiding the negative transfer phenomenon;

[0149] Establishing a domain transfer system for speech imagery EEG intention decoding across subjects and time periods based on collaborative clustering can transfer domain knowledge and adapt the collaborative clustering matrix decomposition model based on a small number of labeled samples in the target domain, thereby improving the robustness of the speech imagery EEG intention decoding algorithm across subjects and time periods.

[0150] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering, characterized in that: The steps include: Step 1: Obtain existing speech imagery EEG data to construct a source domain dataset and preprocess it. The source domain dataset X s Contains m source domain data subsets; Step 2, extract the EEG features related to speech imagination in the preprocessed source domain data set, and then define the clustering indicator matrix of the features and the clustering indicator matrix of the samples as and coefficient matrix according to the label type; Step 3: Under the semi-supervised framework, the clustering indicator matrix of the features, the clustering indicator matrix of the samples, and the optimization solution of the coefficient matrix are completed; Step 4: Use the optimized feature clustering indicator matrix, sample clustering indicator matrix, and coefficient matrix to train a semi-supervised speech imagery intention decoding model based on EEG sample and feature co-clustering. For each data subset X si Get a corresponding independent decoding model; Step 5: Collect the labeled EEG data of the subjects with speech imagination intention labels in real time at any time period, that is, the target domain data, and preprocess them to obtain the target domain data set X t ; Step 6: Traverse all source domain data subsets X si and its corresponding cluster indicator matrix and coefficient matrix of optimized features to determine the best source domain match for the target domain; Step 7: For the best matching source domain data subset X s* The clustering indicator matrix and coefficient matrix of the features are adjusted to obtain the adjusted coefficient matrix; Step 8: In the target domain, according to the clustering indicator matrix P of the features s* And the adjusted coefficient matrix S s* The final optimized sample feature coupled clustering EEG speech imagery intention decoding matrix decomposition model is obtained, and then the recognition label of intention decoding can be converted into the clustering indicator matrix of the target domain sample; Step 9: Solve the cluster indicator matrix of the target domain samples to obtain the decoding results.

2. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 1 is characterized in that: In step 1, the preprocessing method of the source domain data set includes denoising and filtering, so as to obtain EEG data with artifacts and noise removed to ensure the quality and consistency of the data.

3. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 1 is characterized in that: In step 2, it is assumed that the data matrix X consists of labeled samples X l and unlabeled samples X u Composition, which contains labeled samples X l The number of unlabeled samples is l, u The number of is u, and the original data matrix is ​​rearranged into The corresponding sample cluster indicator matrix Q can be expressed as Among them, Q l ∈Φ l×c , Q u ∈Φ (n-l)×c , where c is the number of categories.

4. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 3 is characterized in that: The sample clustering indicator matrix Q of the labeled samples l The initialization method is: For the i-th labeled sample (i∈{1, 2, ..., l}), its corresponding indicator vector satisfies:

5. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 3 is characterized in that: The sample clustering indicator matrix Q of the unlabeled samples u The initialization method is: for the i-th labeled sample (i∈{l+1, l+2, ..., n}), the corresponding indicator vector is randomly assigned to satisfy: Each unlabeled sample takes the value of 1 only in one randomly selected category and the values ​​of the other categories are 0.

6. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 5 is characterized in that: The cluster indicator matrix of the feature is initialized by randomly initializing the matrix so that only one randomly selected category takes a value of 1 and the other categories take values ​​of 0.

7. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 1, characterized in that: The method for solving the clustering indicator matrix of the features, the clustering indicator matrix of the samples, and the coefficient matrix is ​​as follows: First, the coefficient matrix is ​​solved by fixing the clustering indicator matrix of the features and the clustering indicator matrix of the samples. The formula is as follows: s=H -1 r Where H = P T PQ T Q, r = P T XQ, where P is the cluster indicator matrix of features, Q is the cluster indicator matrix of samples, and S is the coefficient matrix; Then, only the sample clustering indicator matrix Q corresponding to the unlabeled samples is u Iterate and calculate, and there are labeled sample parts Q l Keeping the cluster indicator matrix and system matrix unchanged, calculate Q u , Thanks to Q u Each vector q in i Only one element in Q is equal to 1, and the rest are 0, so Q u The solution can be determined by the following formula: where r k is the kth column of R, R = PS; Finally, when the clustering indicator matrix and coefficient matrix of the sample are fixed, the clustering indicator matrix of the feature is solved. The solution formula is as follows: Among them l k is the kth row of L, L = SQ T ; Repeat the above solution process, alternately solve the optimization, and iterate until convergence, so as to obtain the cluster indicator matrix P of the optimized feature i , the cluster indicator matrix Q of the sample i , coefficient matrix S i .

8. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 7, characterized in that: In step 4, the method for training a semi-supervised speech imagery intention decoding model based on EEG samples and feature co-clustering is: Based on a semi-supervised sparse neighbor-constrained co-clustering algorithm based on matrix decomposition, the original data matrix X, where the number of samples is n and the number of features is d, is subjected to feature clustering and sample clustering. The optimization objective formula of the co-clustering is as follows: Among them, P represents the clustering indicator matrix of features, Q represents the clustering indicator matrix of samples, and S is the coefficient matrix used to connect P and Q. For each data subset X si (i∈{1,2,...,N s }) obtains a corresponding independent decoding model, the symbol T represents the transpose operation of the matrix, the symbol F represents the Frobenius norm, d represents the number of features, c represents the number of clusters, n represents the number of samples, diag represents a diagonal matrix, and S∈diag represents that S is a diagonal matrix.

9. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 1, characterized in that: In step 5, the target domain data is preprocessed as follows: First, the collected EEG data is downsampled, and the signals corresponding to the bad electrodes are removed and regenerated by interpolating adjacent electrodes; the EEG is then filtered to 0.1-50Hz through a bandpass filter to remove noise and eliminate the interference of noise on the classification accuracy; then the time-frequency-space multi-dimensional features are extracted, connected, sorted and standardized.

10. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 8, characterized in that: In step 6, let the subset of speech imagery EEG data collected in the target domain be X t , from the labeled sample X tl and unlabeled samples X tu The intent decoding label corresponding to the labeled sample is Y tl , then let the cluster indicator matrix Q of the target domain samples tl =Y tl .

11. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 10, characterized in that: In step 6, all source domain data subsets X are traversed. si and its corresponding matrix P i and S i , by calculating the target domain data X t The approximate matrix PSQ output by the decoding model corresponding to the subset of source domain data T The F norm between Select the source domain subset X that minimizes the norm s* , thus obtaining the source domain data subset X s* The corresponding feature cluster indicator matrix P s* and the system matrix S s* , and takes it as the best source domain match for the target domain, and then as the object of domain adaptation.

12. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 11, characterized in that: The source domain data subset X s* The corresponding feature cluster indicator matrix P s* and the system matrix S s* The adjustment method is to fix the cluster indicator matrix P of the feature s* , only adjust the coefficient matrix S s* , the adjustment formula is as follows: S s* ′=H -1 r in 13. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 12, characterized in that: The final optimized sample feature coupling clustering EEG speech imagery intention decoding matrix decomposition model is expressed as:

14. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 13, characterized in that: In step 8, the identification label intended for decoding can be converted into a cluster indicator matrix Q of the target domain sample: t Solve, the cluster indicator matrix Q of the target domain samples t The solution is: Since Q t Each vector q in i Only one element in Q is equal to 1, and the rest are 0, so Q t The solution can be determined by the following formula: where r k yes The kth column of .

15. The method for cross-subject speech imagery EEG intention decoding domain transfer based on collaborative clustering according to claim 13 or 14, characterized in that: The step 10 is also included, in which the clustering indicator matrix Q of the target domain samples is t Visualize the solution results.

Images