A semi-supervised cross-subject motor imagery electroencephalogram classification method and device
Patent Information
- Application Number
- CN202611126784.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-28
- Publication Date
- 2026-10-09
AI Technical Summary
这种离散化形式无法精细刻画样本对不同运动意图类别的隶属程度，尤其对处于感觉运动皮层响应模糊区域的样本，易造成误判
本申请提供了一种半监督跨被试运动想象脑电分类方法及装置，通过伪标签校准，取代传统的单一硬标签，能够更精细地刻画样本对各类别的隶属程度，提高了决策边界附近样本的识别稳定性，另外，对伪标签进行等角紧框架（Equiangular Tight Frame，ETF）几何约束，迫使不同类别的特征分布逼近理想的等角紧框架结构，从而在跨被试场景下更好地保留了脑电信号的拓扑边界，兼顾了运动想象脑电分类的精度与模型鲁棒性。
Smart Images
Figure CN122885635A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of EEG signal recognition technology, and in particular to a semi-supervised method and device for classifying EEG signals across subjects’ motor imagery. Background Technology
[0002] Brain-computer interfaces (BCIs) based on motor imagery are of significant value in neurorehabilitation and intelligent control because they can directly reflect an individual's motor intentions. However, existing cross-subject motor imagery EEG classification techniques have the following main shortcomings: First, in terms of label utilization, most methods employ "hard label" judgment, which completely categorizes samples into a specific category. This discretization method cannot accurately characterize the degree to which a sample belongs to different motor intention categories, especially for samples in the ambiguous region of sensorimotor cortex response, which can easily lead to misjudgment.
[0003] Second, when processing unlabeled target domain data, existing solutions typically generate pseudo-labels directly using a single classifier, lacking effective reliability screening and correction mechanisms. When the signal-to-noise ratio of EEG signals is low or there are significant individual rhythm differences, the high error rate of pseudo-labels can be amplified during iterative training, leading to negative transfer.
[0004] Third, existing transfer learning-based methods tend to focus on global covariance distribution alignment, failing to effectively distinguish local patterns and subdomain differences among different motor imagery categories. Furthermore, they do not fully utilize geometric prior information such as "intra-class aggregation and inter-class maximum angle separation," which makes the alignment process prone to disrupting the boundaries of brain topology and limiting classification accuracy and robustness.
[0005] Therefore, there is an urgent need for a method that can achieve high accuracy and robustness in cross-subject motor imagery EEG classification under conditions of scarce annotations and significant individual differences. Summary of the Invention
[0006] The purpose of this application is to provide a semi-supervised EEG classification method and device for cross-subject motor imagery, which can suppress the influence of pseudo-label noise and individual differences, and improve the accuracy and robustness of cross-subject motor imagery recognition.
[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a semi-supervised cross-subject motor imagery EEG classification method, including: Acquire a motor imagery EEG dataset; the motor imagery EEG dataset includes EEG signal data from multiple subjects; The target learning model is trained using the motor imagery EEG classification dataset to obtain the motor imagery EEG classification model. During the training process, the target learning model introduces hybrid semi-supervised learning guided by isoangular tight frame and pseudo-label calibration, generates a pseudo-label sample set, and applies isoangular tight frame geometric constraints to the feature matrix formed by the pseudo-label samples of each motor imagery category in the pseudo-label sample set. The motor imagery EEG classification model described above was used for motor imagery EEG classification.
[0008] Optionally, the EEG signal data of each subject in the motor imagery EEG dataset is divided into a source domain and a target domain. A set proportion of samples in the source domain are labeled, while all samples in the target domain are unlabeled. The set ratio is 10%.
[0009] Optionally, the target learning model includes a feature extraction module, a hybrid semi-supervised module, a semantic consistency contrastive learning module, and an isoangular tight frame geometric constraint module; The feature extraction module is used to extract spatiotemporal feature vectors from the input EEG signal data based on the augmented covariance matrix; The hybrid semi-supervised module is used to initialize a geometric prototype with maximum angular separation characteristics for each motor imagery category in the motor imagery EEG dataset using an isoangular tight frame; apply weak enhancement to the spatiotemporal feature vector corresponding to any unlabeled sample i in the motor imagery EEG dataset; calculate the predicted category distribution output by the classifier head, as well as the semantic similarity distribution between each weakly enhanced feature and each geometric prototype; fuse the predicted category distribution and the semantic similarity distribution to obtain pseudo-labels for unlabeled samples; and use pseudo-labels with a maximum confidence exceeding a preset threshold as supervision signals for strongly enhanced unlabeled samples i; the samples corresponding to pseudo-labels with a maximum confidence exceeding the preset threshold constitute a pseudo-label sample set; and calculate the consistency loss of the spatiotemporal feature vector corresponding to any unlabeled sample i after weak and strong enhancement. The semantic consistency contrastive learning module is used to apply various enhancement methods to the spatiotemporal feature vectors corresponding to unlabeled samples in the source domain, calculate the semantic consistency of the features after different enhancement methods, and count the proportion of samples whose semantic consistency exceeds a preset similarity threshold under different enhancement combinations. A reliable enhancement combination sample set is selected, and the reliable enhancement combination sample set and the pseudo-label sample set are used for contrastive learning, and the contrastive learning loss is calculated. The isoangular tight frame geometric constraint module is used to determine the feature matrix of samples of each motion imagination category in the pseudo-label sample set, and calculate the isoangular tight frame geometric constraint loss based on the feature matrix of the samples.
[0010] Optionally, the feature extraction module includes an augmented covariance matrix calculation unit, a diagonal loading regularization unit, a tangent space feature calculation unit, and a source adaptive normalization unit; The augmented covariance matrix calculation unit is used to construct an augmented state vector at time t based on the input EEG signal data through time delay embedding, and to calculate the sample covariance matrix based on the augmented state vectors at all times. The diagonal loading regularization unit applies diagonal loading regularization to the sample covariance matrix; The tangent space feature calculation unit is used to calculate the tangent space projection of the matrix after diagonal loading regularization with the Riemann mean of all samples in the motor imagery EEG dataset as the reference point, and then vectorize it to obtain the tangent space features. The source adaptive normalization unit is used to adaptively determine whether to normalize the tangent space features based on the embedding order and the delay step size of the time delay, and outputs a spatiotemporal feature vector; the embedding order is the embedding order used to calculate the sample covariance matrix.
[0011] Optionally, the hybrid semi-supervised module includes a source domain labeled sample cross-entropy loss unit, an isoangular tight frame prototype initialization and update unit, a weakly enhanced semantic similarity unit, a pseudo-label calibration unit, a strongly enhanced consistency constraint unit, and an unlabeled loss unit. The source domain labeled sample cross-entropy loss unit is used to calculate the cross-entropy loss for labeled samples in the source domain. The isoangular tight frame prototype initialization and update unit is used to initialize a geometric prototype with maximum angular separation characteristics for the category prototype of each motion imagination category using an isoangular tight frame. The initial prototype matrix formed by the geometric prototypes satisfies the following formula: ; in, To initialize the prototype matrix, for transpose, The number of categories for the "motion imagination" category. It is the identity matrix. It is a vector of all ones. for transpose, The geometric prototype for the category k of motion imagination; The weakly enhanced semantic similarity unit is used to apply weak enhancement to the spatiotemporal feature vector corresponding to any unlabeled sample i in the motor imagery EEG dataset, and calculate the cosine similarity distribution between the feature vector after weak enhancement and each geometric prototype as the semantic similarity distribution. The pseudo-label calibration unit is used to fuse the predicted category distribution and the semantic similarity distribution according to the following formula to obtain pseudo-labels for unlabeled samples: ; in, The pseudo-label for unlabeled sample i. This is the predicted class probability of the unlabeled sample i output by the classification head. Let be the semantic similarity distribution of unlabeled sample i; The strongly enhanced consistency constraint unit is used to take the pseudo-label with the maximum confidence exceeding the preset threshold as the strongly enhanced supervision signal of the corresponding unlabeled sample i. The unlabeled loss unit is used to calculate the consistency loss of the spatiotemporal feature vector corresponding to any unlabeled sample i after weak and strong enhancement according to the following formula: ; in, The consistency loss is represented by σ, which is a preset threshold. For indicator functions, Represents the predicted probability distribution. Represents cross-entropy loss, For the batch size of labeled samples, This indicates the ratio of unlabeled samples to labeled samples.
[0012] Optionally, the semantic consistency comparison learning module includes multiple strong and weak enhancement units, semantic consistency calculation units, semantic consistency filtering units, and positive sample comparison learning units; The various strong and weak enhancement units are used to apply weak noise enhancement, strong noise enhancement, and weak sign flipping enhancement to the spatiotemporal feature vectors corresponding to unlabeled samples in the source domain, respectively. The semantic consistency calculation unit is used to calculate the cosine similarity of the feature representations after enhancement by each of the two enhancement methods, as the semantic consistency of the feature representations after enhancement by the corresponding two enhancement methods. The semantic consistency screening unit is used to calculate the proportion of samples whose semantic consistency exceeds a preset similarity threshold under different enhancement combinations using the following formula: ; in, This indicates a preset similarity threshold. Let represent the cosine similarity between the feature representations of sample i after enhancement by method a and method b. This represents the number of unlabeled samples in the source domain. This represents the proportion of samples whose semantic consistency exceeds a preset similarity threshold under the enhancement combination of enhancement method a and enhancement method b. When sample i is stored in the reliable enhanced combined sample set; The contrastive learning loss of the positive sample contrastive learning unit is expressed as: ; in, Indicates the contrast learning loss. The sum of the samples in the reliable enhanced combined sample set and the pseudo-label sample set, The set of reliable enhanced combined samples and the set of pseudo-label samples, where T represents the temperature parameter. and Let i and p represent the enhanced feature representations of sample i and sample p, respectively.
[0013] Optionally, the geometric constraint loss of the isoangular tight frame is expressed as: ; in, For the geometric constraint loss of the equiangular tight frame, This is the feature matrix of samples from each motion imagery category in the pseudo-label sample set. for transpose, It is the identity matrix. This indicates the batch size of the labeled samples.
[0014] Optionally, the target learning model further includes a domain adaptation module for calculating entropy loss on the predicted probability distribution of the target domain samples.
[0015] Optionally, during the training process of the target learning model, the supervised loss of the source domain samples, the consistency loss, the contrastive learning loss, the isoangular tight frame geometric constraint loss, and the entropy loss are weighted and summed to form the total loss.
[0016] Secondly, this application provides a semi-supervised cross-subject motor imagery EEG classification device, wherein the semi-supervised cross-subject motor imagery EEG classification device applies the aforementioned semi-supervised cross-subject motor imagery EEG classification method, and the semi-supervised cross-subject motor imagery EEG classification device comprises: The dataset acquisition module is used to acquire a motor imagery EEG dataset; the motor imagery EEG dataset includes EEG signal data from multiple subjects; The model training module is used to train the target learning model using the motor imagery EEG classification dataset to obtain the motor imagery EEG classification model. During the training process, the target learning model introduces hybrid semi-supervised learning guided by isoangular tight frame and pseudo-label calibration, generates a pseudo-label sample set, and applies isoangular tight frame geometric constraints to the feature matrix formed by the pseudo-label samples of each motor imagery category in the pseudo-label sample set. The model application module is used to perform motor imagery EEG classification using the motor imagery EEG classification model.
[0017] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a semi-supervised cross-subject motor imagery EEG classification method and device. By using pseudo-label calibration instead of traditional single hard labels, it can more finely characterize the degree of membership of samples to each category and improve the recognition stability of samples near the decision boundary. In addition, the pseudo-labels are subjected to equiangular tight frame (ETF) geometric constraints, which force the feature distribution of different categories to approach the ideal equiangular tight frame structure. This better preserves the topological boundary of EEG signals in cross-subject scenarios, and balances the accuracy and robustness of motor imagery EEG classification. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating a semi-supervised cross-subject motor imagery EEG classification method provided in one embodiment of this application.
[0020] Figure 2 This is a schematic diagram illustrating the principle architecture of a semi-supervised cross-subject motor imagery EEG classification method provided in one embodiment of this application.
[0021] Figure 3 This is a schematic diagram of the structure of a feature extraction module based on augmented covariance matrix provided in an embodiment of this application.
[0022] Figure 4 This is a schematic diagram of the structure of an ETF-guided hybrid semi-supervised module provided in an embodiment of this application.
[0023] Figure 5 This is a schematic diagram of the structure of a semantic consistency comparison learning module provided in an embodiment of this application.
[0024] Figure 6 This is a schematic diagram of the structure of an ETF geometric constraint module provided in an embodiment of this application.
[0025] Figure 7 This is a schematic diagram of the structure of a domain adaptive module provided in an embodiment of this application. Detailed Implementation
[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0027] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0028] In one exemplary embodiment, such as Figure 1 and Figure 2 As shown, this application provides a semi-supervised cross-subject motor imagery EEG classification method, including steps 101-103.
[0029] Step 101: Obtain the motor imagery EEG dataset; the motor imagery EEG dataset includes EEG signal data from multiple subjects.
[0030] Step 102: The target learning model is trained using the motor imagery EEG classification dataset to obtain the motor imagery EEG classification model. During the training process, the target learning model introduces hybrid semi-supervised learning guided by isoangular tight frame and pseudo-label calibration, generates a pseudo-label sample set, and applies isoangular tight frame geometric constraints to the feature matrix formed by the pseudo-label samples of each motor imagery category in the pseudo-label sample set.
[0031] Step 103: Perform motor imagery EEG classification using the motor imagery EEG classification model.
[0032] This application uses pseudo-label calibration instead of traditional single hard labels, which can more finely characterize the degree of membership of samples to each category and improve the recognition stability of samples near the decision boundary. In addition, the pseudo-label is subjected to isoangular tight frame geometric constraints, which force the feature distribution of different categories to approach the ideal isoangular tight frame structure, thereby better preserving the topological boundary of EEG signals in cross-subject scenarios and balancing the accuracy and robustness of motor imagery EEG classification.
[0033] In an exemplary embodiment, this application first preprocesses the collected EEG data of motor imagery from multiple subjects, and then constructs robust spatiotemporal features through a tangent space feature extraction module based on augmented covariance matrix.
[0034] Specifically, raw EEG signal data and motor imagery labels (categories) of multiple subjects were imported from the storage medium, and the data were organized and labeled according to subject number and trial order.
[0035] To simulate a scenario of scarce annotations in real-world scenarios, the EEG signal data (EEG signals) of each participant in the motor imagery EEG dataset is divided into a source domain and a target domain. A predetermined proportion of samples in the source domain are labeled, while all samples in the target domain are unlabeled. The predetermined proportion is 10%, meaning that only 10% of the samples in the source domain are labeled, and the remaining samples and all samples in the target domain are unlabeled. This is used to simulate a semi-supervised cross-subject learning task with scarce annotations in real-world scenarios.
[0036] Subsequently, the original EEG signal is subjected to preprocessing operations such as bandpass filtering, power frequency interference suppression (notch filtering), and artifact removal through the signal preprocessing module to improve signal quality and form a motor imagery EEG dataset.
[0037] In one exemplary embodiment, to address the scarcity of annotations, a hybrid semi-supervised module guided by an isoangular tight frame is employed. This module initializes category prototypes using the isoangular tight frame and fuses classification head predictions with prototype similarity to generate calibrated pseudo-labels. Furthermore, a semantic consistency contrastive learning module is introduced, which enhances feature discriminative power by using sign flipping enhancement and semantic thresholding to filter reliable positive sample pairs. Simultaneously, an isoangular tight frame geometric constraint module imposes orthogonality constraints on the feature matrices of the pseudo-label samples, forcing each category's features to approximate the maximum angle separation structure; the domain adaptation module reduces cross-subject distribution bias by minimizing the target domain prediction entropy.
[0038] The target learning model includes a feature extraction module, a hybrid semi-supervised module, a semantic consistency contrastive learning module, and an isoangular tight frame geometric constraint module.
[0039] The feature extraction module is used to extract spatiotemporal feature vectors from the input EEG signal data based on the augmented covariance matrix. Specifically, the EEG signals in the motor imagery EEG dataset are input into the feature extraction module. This module first incorporates the temporal information of the EEG signals into the covariance matrix through time-delay embedding. Then, the temporal information incorporated into the covariance matrix is sequentially subjected to diagonal loading regularization to ensure the positive definiteness of the matrix, projected to the Riemann tangent space, and quantized. Finally, normalization is adaptively determined based on the embedding parameters, resulting in a high-dimensional spatiotemporal feature vector with good numerical robustness, improving the numerical stability and robustness of features under cross-domain distribution shifts.
[0040] like Figure 3 As shown, the feature extraction module includes an augmented covariance matrix calculation unit, a diagonal loading regularization unit, a tangent space feature calculation unit, and a source adaptive normalization unit.
[0041] The augmented covariance matrix calculation unit is used based on the input EEG signal data. An augmented state vector at time t is constructed by time-delay embedding, and the sample covariance matrix is calculated based on the augmented state vectors of all valid time points from which vectors can be constructed. . Let C be the set of real numbers in rows C and columns T.
[0042] ; in, Let the embedding order be , To delay the step size, Where T is the number of channels, and T represents the number of sampling points for the time series.
[0043] The diagonal loading regularization unit applies diagonal loading regularization to the sample covariance matrix.
[0044] ; in, To load the regularized matrix diagonally, The sample covariance matrix, The regularization coefficient is . It is the identity matrix, used to ensure the positive definiteness of the matrix and preserve the original covariance structure.
[0045] The tangent space feature calculation unit is used to calculate the Riemann mean of all samples in the motor imagery EEG dataset. Using the reference point, calculate the tangent space projection of the diagonally loaded regularized matrix and vectorize it to obtain the tangent space features; ; in, For the matrix logarithm, Let be a symmetric matrix in the tangent space. Vectorize its upper triangular elements (including the diagonal) to obtain the tangent space features. ; The source adaptive normalization unit is used to adjust the embedding order. Delay step size and time delay An adaptive determination is made as to whether to normalize the tangent space features, and a spatiotemporal feature vector is output; the embedding order is the embedding order used to calculate the sample covariance matrix.
[0046] ; in, For spatiotemporal feature vectors, For adaptive coefficients, for Norm. This strategy is based on The parameter adaptation determines whether the features are normalized, ultimately yielding feature representations for downstream classification and contrastive learning.
[0047] The hybrid semi-supervised module is used to initialize a geometric prototype with maximum angular separation characteristics for each motor imagery category in the motor imagery EEG dataset using an isoangular tight frame; apply weak noise enhancement to the spatiotemporal feature vector corresponding to any unlabeled sample i in the motor imagery EEG dataset; calculate the predicted category distribution output by the classification head, as well as the semantic similarity distribution between each weakly enhanced feature and each geometric prototype; fuse the predicted category distribution and the semantic similarity distribution to obtain pseudo-labels for unlabeled samples; and use pseudo-labels with a maximum confidence exceeding a preset threshold as supervision signals for unlabeled samples i after strong noise enhancement (applying consistency constraints to the same unlabeled sample after strong enhancement); the samples corresponding to pseudo-labels with a maximum confidence exceeding the preset threshold constitute a pseudo-label sample set; calculate the consistency loss of the spatiotemporal feature vector corresponding to any unlabeled sample i after weak and strong enhancement, thereby optimizing the target learning model using unlabeled data.
[0048] The classification head is a fully connected layer. Its input is connected to the output of the feature extraction module, and it is responsible for mapping the spatiotemporal feature vector to the class probability distribution.
[0049] The semantic consistency contrastive learning module enhances feature discrimination capabilities by introducing a high-confidence pseudo-label sample set and its category assignment results generated by the hybrid semi-supervised module. Specifically, the semantic consistency contrastive learning module applies multiple different enhancement methods to the spatiotemporal feature vectors corresponding to unlabeled samples in the source domain, obtaining multiple views, calculating the semantic consistency of features after different enhancement methods, and statistically analyzing the proportion of samples with semantic consistency exceeding a preset similarity threshold under different enhancement combinations based on the entire unlabeled dataset in the source domain. It then selects reliable enhancement combinations (including weak-strong noise enhancement and weak-sign-flip weak noise enhancement) sample sets that maintain semantic stability. The reliable enhancement combination sample set and the pseudo-label sample set are then used for contrastive learning. Specifically, the reliable enhancement combination sample set is combined with high-confidence pseudo-label samples of the same category in the pseudo-label sample set (providing a "category attribution" supervision signal for the reliable enhancement combination) to construct positive sample pairs and calculate the contrastive learning loss, forcing similar samples to cluster tightly in the embedding space.
[0050] The isoangular tight frame geometric constraint module is used to determine the feature matrix of samples of each motion imagination category in the pseudo-label sample set, and calculate the isoangular tight frame geometric constraint loss based on the feature matrix of the samples. The isoangular tight frame geometric constraint loss function forces the feature vectors of different categories to maintain an approximately isoangular relationship and norm consistency in space by calculating the difference between the product of the feature matrix and its transpose and the identity matrix, thereby approximating the ideal ETF geometry.
[0051] In an exemplary embodiment, the hybrid semi-supervised module initializes the category prototypes of the spatiotemporal feature vectors output by the feature extraction module using an ETF structure, providing a good structured geometric precursor for the semi-supervised learning process. Figure 4 As shown, the hybrid semi-supervised module includes a source domain labeled sample cross-entropy loss unit, an isoangular tight frame prototype initialization and update unit, a weakly enhanced semantic similarity unit, a pseudo-label calibration unit, a strongly enhanced consistency constraint unit, and an unlabeled loss unit.
[0052] The source domain labeled sample cross-entropy loss unit is used to calculate the cross-entropy loss, i.e., supervised loss, for labeled samples in the source domain. This is used to constrain the model to correctly classify labeled samples and ensure that the model has basic discriminative ability in the source domain.
[0053] ; Among them, This is the spatiotemporal feature representation output from the second step. For sample batch size, For feature encoder, Extract labeled data The spatiotemporal characteristics, For classification header, This is the cross-entropy loss function.
[0054] The isoangular tight frame prototype initialization and update unit is used to initialize a geometric prototype with maximum angular separation characteristics for the category prototype of each motion imagination category using an isoangular tight frame. The initial prototype matrix formed by the geometric prototypes satisfies the following formula: ; in, To initialize the prototype matrix, for transpose, Let d represent the number of categories in the motion imagery category, and d represent the dimension of the feature vector. It is the identity matrix. It is a vector of all ones. for transpose, Let k be the geometric prototype of the motion imagination category k, i.e., the initial value.
[0055] The category prototype is dynamically updated using the embedding mean of labeled samples and high-confidence unlabeled samples: , ; in, Let be the embedding mean of the labeled samples and the high-confidence unlabeled samples corresponding to the motion imagination category k. For the updated motion imagination category k, It includes labeled samples of class k and unlabeled samples with high confidence. The geometric prototype is the feature representation of sample i after being mapped into the embedding space by the projection head. During training, the geometric prototype... Dynamically adjusted based on updates, Let i be the spatiotemporal features of sample i.
[0056] The weakly enhanced semantic similarity unit is used to apply weak enhancement (noise weak enhancement) to the spatiotemporal feature vector corresponding to any unlabeled sample i in the motor imagery EEG dataset, and calculate the cosine similarity distribution between the feature vector after applying weak enhancement and each geometric prototype as the semantic similarity distribution. ; in, The cosine similarity between the features of sample i after applying weak enhancement and the label type k is calculated. , This indicates the features of sample i after applying weak enhancement. This indicates weak noise enhancement. For cosine similarity, For temperature parameters, This indicates the mapping header.
[0057] The pseudo-label calibration unit is used to fuse the predicted category distribution with the semantic similarity distribution according to the following formula to obtain pseudo-labels for unlabeled samples: ; in, The pseudo-label for unlabeled sample i. This is the predicted class probability of the unlabeled sample i output by the classification head. Let be the semantic similarity distribution of unlabeled sample i. To multiply element by element, the denominator is the normalization factor; The strong enhancement consistency constraint unit is used to take the pseudo-label whose maximum confidence (i.e. the maximum value in the probability distribution output by the classification head of the weakly enhanced sample) exceeds a preset threshold as the supervision signal of the corresponding unlabeled sample i after strong enhancement, and to apply consistency constraints to the strongly enhanced sample. The unlabeled loss unit is used to calculate the consistency loss of the spatiotemporal feature vector corresponding to any unlabeled sample i after weak and strong enhancement according to the following formula: ; in, The consistency loss is represented by σ, which is a preset threshold. For indicator functions, Represents the cross-entropy loss function. This represents the predicted probability distribution output by the target learning model. This indicates the ratio of unlabeled samples to labeled samples.
[0058] In an exemplary embodiment, the semantic consistency contrastive learning module proposes a semantic consistency verification strategy before introducing new enhancement methods. This strategy is used to determine whether different enhancement methods can maintain the semantic stability of samples, thereby deciding whether they can be used to construct positive sample pairs for contrastive learning.
[0059] like Figure 5 As shown, the semantic consistency comparison learning module includes multiple strong and weak enhancement units, semantic consistency calculation units, semantic consistency filtering units, and positive sample comparison learning units; The various strong and weak enhancement units are used to apply weak noise enhancement to the spatiotemporal feature vectors corresponding to the unlabeled samples in the source domain. Noise intensity enhancement and weak enhancement of sign flipping This yields multiple enhanced views for subsequent semantic consistency evaluation; The semantic consistency calculation unit is used to calculate unlabeled samples. The cosine similarity between the feature representations enhanced by each of the two enhancement methods is used as the semantic consistency of the feature representations enhanced by the corresponding two enhancement methods. ; in, Let represent the cosine similarity between the feature representations of sample i after enhancement by method a and method b. and They represent different enhancement methods. The feature representation in the embedding space.
[0060] The semantic consistency screening unit is used to calculate the proportion of samples whose semantic consistency exceeds a preset similarity threshold under different enhancement combinations using the following formula: ; in, This indicates a preset similarity threshold. Let represent the cosine similarity between the feature representations of sample i after enhancement by method a and method b. This represents the number of unlabeled samples in the source domain. This represents the proportion of samples whose semantic consistency exceeds a preset similarity threshold under the enhancement combination of enhancement method a and enhancement method b. If the enhancement combination is considered to maintain semantic consistency, sample i is stored in the reliable enhancement combination sample set; otherwise, it is discarded. Through this strategy, the noise weak enhancement-noise strong enhancement combination and the noise weak enhancement-sign flip weak enhancement combination are selected to construct positive sample pairs. The positive sample contrastive learning unit is used to construct a contrastive learning loss based on the selected reliable enhancement combinations and high-confidence pseudo-label samples. The contrastive learning loss is expressed as: ; in, Indicates the contrast learning loss. The sum of the samples in the reliable enhanced combined sample set and the pseudo-label sample set, The set of reliable enhanced combined samples and the set of pseudo-labeled samples. Desirable and This represents the set of positive samples under noise enhancement and flip enhancement, and also includes high-confidence pseudo-label samples of the same class. T represents the temperature parameter. and Let represent the enhanced feature representations of sample i and sample p, respectively. Finally, the contrastive learning loss based on noise enhancement is obtained. Contrastive learning loss based on flipping enhancement .
[0061] In one exemplary embodiment, the isoangular tight frame geometric constraint module is used to guide feature vectors of different categories to form a distribution with good separability by imposing explicit constraints on the geometric structure of sample features.
[0062] like Figure 6 As shown, the equiangular tight frame geometric constraint module includes feature matrix units and ETF geometric constraint units.
[0063] Characteristic matrix unit, used to make Let represent the feature matrix of all pseudo-labeled samples in class k. They represent the first, second, and third classes of the k-th class, respectively. Features of a pseudo-labeled sample denoted as the number of all pseudo-labeled samples in class k.
[0064] The geometric constraint loss of the equal-angle tight frame is expressed as: ; in, For the geometric constraint loss of the equiangular tight frame, This is the feature matrix of samples from each motion imagery category in the pseudo-label sample set. for transpose, For the batch size of labeled samples, It is the Euclidean norm.
[0065] The isoangular tight frame geometric constraint loss forces feature vectors to maintain isoangular relationships and norm consistency, improving pseudo-label stability and cross-subject classification performance.
[0066] In an exemplary embodiment, to reduce the distribution offset between the source domain and the target domain, the target learning model further includes a domain adaptation module for calculating the entropy loss of the predicted probability distribution of the target domain samples, thereby encouraging the target learning model to produce more deterministic prediction results.
[0067] The domain adaptation module facilitates the shift of the decision boundary towards high-density regions by constraining the classifier's prediction uncertainty on target domain samples. For example... Figure 7 As shown, the domain adaptation module includes an entropy loss unit, which calculates the entropy loss based on the predicted probability distribution of the target domain samples. .
[0068] ; in, Indicates the target domain. The sample belongs to the first Predicted probability of class For the number of categories, This represents the number of target domain samples used in training. Minimizing this loss encourages the model to produce more deterministic predictions in the target domain, thus enabling adaptive adjustment across domain distributions.
[0069] In an exemplary embodiment, during the training of the target learning model, the supervised loss of the source domain samples, the consistency loss, the contrastive learning loss, the isoangular tight frame geometric constraint loss, and the entropy loss are weighted and summed to form a total loss, and the target learning model is trained end-to-end using the total loss.
[0070] After the target learning model is trained, the saving and output module stores and outputs information such as the classification results of motor imagery across subjects, the main evaluation indicators, the feature samples of each subject and pseudo-labels, for subsequent analysis or actual BCI system deployment.
[0071] In an exemplary embodiment, comparative experiments were conducted on three publicly available motion imagery datasets—BCI2a, BCI2b, and BCI2015—using the motion imagery features and recognition results obtained in steps 101-103. Commonly used machine learning methods and typical transfer learning methods for semi-supervised motion imagery across subject domains were selected as baselines, including the Meaneacher model, Virtual Adversarial Training (VAT), Fixed-threshold pseudo-label model (Fixmatch), Adaptive-threshold pseudo-label model (Freematch), and DeepSource Semi-Supervised Transfer Learning (DS3TL). In the subject-independent scenario, all samples from one subject were used as the test set each time, while the remaining subject samples were used to train the model. This process was repeated sequentially, and the average accuracy and standard deviation of each method on the three datasets were statistically analyzed. Detailed results are shown in Table 1.
[0072] Table 1. Mean accuracy and standard deviation (%) of the comparative experiment in independent scenarios for the participants.
[0073] As shown in Table 1, the average accuracy of the method proposed in this application is significantly higher than that of the comparative methods on the three datasets. For example, the average accuracy of the method proposed in this application is 55.19% ± 0.65% on the BCI2a dataset, 62.41% ± 0.62% on the BCI2b dataset, and 72.99% ± 0.80% on the BCI2015 dataset. All of these are better than baseline models such as Meaneacher, VAT, Fixmatch, Freematch, and DS3TL, indicating that the method proposed in this application has better discrimination ability and overall performance in semi-supervised cross-subject task of motor imagery.
[0074] Compared with existing technologies, the present application achieves the following beneficial technical effects: Higher classification accuracy and stability: By outputting "calibrated soft labels" that predict similarity between the fused classification head and the prototype, instead of the traditional single "hard label", it can more finely characterize the degree of membership of samples to each category, significantly improve the recognition stability of samples near the decision boundary, and outperform existing mainstream methods in average accuracy on multiple public datasets.
[0075] Effective suppression of negative transfer: The dual pseudo-label calibration mechanism of "ETF prototype initialization + confidence threshold screening" provides a geometric prior for the maximum angle separation of categories and effectively eliminates low-confidence noise labels, fundamentally reducing the accumulation and amplification effect of erroneous pseudo-labels in iterative training and reducing the risk of negative transfer across subject scenarios.
[0076] Superior inter-class separability: The innovative introduction of category ETF geometric constraint loss, while focusing on intra-class aggregation, imposes explicit orthogonality constraints on the feature matrix, forcing the feature distribution of different categories to approach the ideal isoangular tight frame structure, thereby better preserving the topological boundary of EEG signals in cross-subject scenarios, and balancing the accuracy and robustness of motor imagery classification.
[0077] Based on the same inventive concept, this application also provides a semi-supervised cross-subject motor imagery EEG classification device for implementing the aforementioned semi-supervised cross-subject motor imagery EEG classification method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more embodiments of the semi-supervised cross-subject motor imagery EEG classification device provided below can be found in the above-described limitations of the semi-supervised cross-subject motor imagery EEG classification method, and will not be repeated here.
[0078] In one exemplary embodiment, a semi-supervised cross-subject motor imagery EEG classification device is provided, the semi-supervised cross-subject motor imagery EEG classification device applying the aforementioned semi-supervised cross-subject motor imagery EEG classification method, the semi-supervised cross-subject motor imagery EEG classification device comprising: The dataset acquisition module is used to acquire a motor imagery EEG dataset; the motor imagery EEG dataset includes EEG signal data from multiple subjects; The model training module is used to train the target learning model using the motor imagery EEG classification dataset to obtain the motor imagery EEG classification model. During the training process, the target learning model introduces hybrid semi-supervised learning guided by isoangular tight frame and pseudo-label calibration, generates a pseudo-label sample set, and applies isoangular tight frame geometric constraints to the feature matrix formed by the pseudo-label samples of each motor imagery category in the pseudo-label sample set. The model application module is used to perform motor imagery EEG classification using the motor imagery EEG classification model.
[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0080] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A semi-supervised method for classifying cross-subject motor imagery EEG, characterized in that, include: Acquire a motor imagery EEG dataset; the motor imagery EEG dataset includes EEG signal data from multiple subjects; The target learning model is trained using the motor imagery EEG classification dataset to obtain the motor imagery EEG classification model. During the training process, the target learning model introduces hybrid semi-supervised learning guided by isoangular tight frame and pseudo-label calibration, generates a pseudo-label sample set, and applies isoangular tight frame geometric constraints to the feature matrix formed by the pseudo-label samples of each motor imagery category in the pseudo-label sample set. The motor imagery EEG classification model described above was used for motor imagery EEG classification.
2. The semi-supervised cross-subject motor imagery EEG classification method according to claim 1, characterized in that, In the motor imagery EEG dataset, the EEG signal data of each subject is divided into a source domain and a target domain. A set proportion of samples in the source domain are labeled, while all samples in the target domain are unlabeled. The set ratio is 10%.
3. The semi-supervised cross-subject motor imagery EEG classification method according to claim 2, characterized in that, The target learning model includes a feature extraction module, a hybrid semi-supervised module, a semantic consistency contrastive learning module, and an isoangular tight frame geometric constraint module. The feature extraction module is used to extract spatiotemporal feature vectors from the input EEG signal data based on the augmented covariance matrix; The hybrid semi-supervised module is used to initialize a geometric prototype with maximum angular separation characteristics for each motor imagery category in the motor imagery EEG dataset using an isoangular tight frame; apply weak enhancement to the spatiotemporal feature vector corresponding to any unlabeled sample i in the motor imagery EEG dataset; calculate the predicted category distribution output by the classifier head, as well as the semantic similarity distribution between each weakly enhanced feature and each geometric prototype; fuse the predicted category distribution and the semantic similarity distribution to obtain pseudo-labels for unlabeled samples; and use pseudo-labels with a maximum confidence exceeding a preset threshold as supervision signals for strongly enhanced unlabeled samples i; the samples corresponding to pseudo-labels with a maximum confidence exceeding the preset threshold constitute a pseudo-label sample set; and calculate the consistency loss of the spatiotemporal feature vector corresponding to any unlabeled sample i after weak and strong enhancement. The semantic consistency contrastive learning module is used to apply various enhancement methods to the spatiotemporal feature vectors corresponding to unlabeled samples in the source domain, calculate the semantic consistency of the features after different enhancement methods, and count the proportion of samples whose semantic consistency exceeds a preset similarity threshold under different enhancement combinations. A reliable enhancement combination sample set is selected, and the reliable enhancement combination sample set and the pseudo-label sample set are used for contrastive learning, and the contrastive learning loss is calculated. The isoangular tight frame geometric constraint module is used to determine the feature matrix of samples of each motion imagination category in the pseudo-label sample set, and calculate the isoangular tight frame geometric constraint loss based on the feature matrix of the samples.
4. The semi-supervised cross-subject motor imagery EEG classification method according to claim 2, characterized in that, The feature extraction module includes an augmented covariance matrix calculation unit, a diagonal loading regularization unit, a tangent space feature calculation unit, and a source adaptive normalization unit. The augmented covariance matrix calculation unit is used to construct an augmented state vector at time t based on the input EEG signal data through time delay embedding, and to calculate the sample covariance matrix based on the augmented state vectors at all times. The diagonal loading regularization unit applies diagonal loading regularization to the sample covariance matrix; The tangent space feature calculation unit is used to calculate the tangent space projection of the matrix after diagonal loading regularization with the Riemann mean of all samples in the motor imagery EEG dataset as the reference point, and then vectorize it to obtain the tangent space features. The source adaptive normalization unit is used to adaptively determine whether to normalize the tangent space features based on the embedding order and the delay step size of the time delay, and outputs a spatiotemporal feature vector; the embedding order is the embedding order used to calculate the sample covariance matrix.
5. The semi-supervised cross-subject motor imagery EEG classification method according to claim 3, characterized in that, The hybrid semi-supervised module includes a source domain labeled sample cross-entropy loss unit, an isoangular tight frame prototype initialization and update unit, a weakly enhanced semantic similarity unit, a pseudo-label calibration unit, a strongly enhanced consistency constraint unit, and an unlabeled loss unit. The source domain labeled sample cross-entropy loss unit is used to calculate the cross-entropy loss for labeled samples in the source domain. The isoangular tight frame prototype initialization and update unit is used to initialize a geometric prototype with maximum angular separation characteristics for the category prototype of each motion imagination category using an isoangular tight frame. The initial prototype matrix formed by the geometric prototypes satisfies the following formula: ； in, To initialize the prototype matrix, for transpose, The number of categories for the "motion imagination" category. It is the identity matrix. It is a vector of all ones. for transpose, The geometric prototype for the category k of motion imagination; The weakly enhanced semantic similarity unit is used to apply weak enhancement to the spatiotemporal feature vector corresponding to any unlabeled sample i in the motor imagery EEG dataset, and calculate the cosine similarity distribution between the feature vector after weak enhancement and each geometric prototype as the semantic similarity distribution. The pseudo-label calibration unit is used to fuse the predicted category distribution with the semantic similarity distribution according to the following formula to obtain pseudo-labels for unlabeled samples: ； in, The pseudo-label for unlabeled sample i. This is the predicted class probability of the unlabeled sample i output by the classification head. Let be the semantic similarity distribution of unlabeled sample i; The strongly enhanced consistency constraint unit is used to take the pseudo-label with the maximum confidence exceeding the preset threshold as the strongly enhanced supervision signal of the corresponding unlabeled sample i. The unlabeled loss unit is used to calculate the consistency loss of the spatiotemporal feature vector corresponding to any unlabeled sample i after weak and strong enhancement according to the following formula: ； in, The consistency loss is represented by σ, which is a preset threshold. For indicator functions, Represents the predicted probability distribution. Represents cross-entropy loss, For the batch size of labeled samples, This indicates the ratio of unlabeled samples to labeled samples.
6. The semi-supervised cross-subject motor imagery EEG classification method according to claim 3, characterized in that, The semantic consistency comparison learning module includes multiple strong and weak enhancement units, semantic consistency calculation units, semantic consistency filtering units, and positive sample comparison learning units. The various strong and weak enhancement units are used to apply weak noise enhancement, strong noise enhancement, and weak sign flipping enhancement to the spatiotemporal feature vectors corresponding to unlabeled samples in the source domain, respectively. The semantic consistency calculation unit is used to calculate the cosine similarity of the feature representations after enhancement by each of the two enhancement methods, as the semantic consistency of the feature representations after enhancement by the corresponding two enhancement methods. The semantic consistency screening unit is used to calculate the proportion of samples whose semantic consistency exceeds a preset similarity threshold under different enhancement combinations using the following formula: ； in, This indicates a preset similarity threshold. Let represent the cosine similarity between the feature representations of sample i after enhancement by method a and method b. This represents the number of unlabeled samples in the source domain. This represents the proportion of samples whose semantic consistency exceeds a preset similarity threshold under the enhancement combination of enhancement method a and enhancement method b. When sample i is stored in the reliable enhanced combined sample set; The contrastive learning loss of the positive sample contrastive learning unit is expressed as: ； in, Indicates the contrast learning loss. The sum of the samples in the reliable enhanced combined sample set and the pseudo-label sample set, The set of reliable enhanced combined samples and the set of pseudo-labeled samples, where T represents the temperature parameter. and Let i and p represent the enhanced feature representations of sample i and sample p, respectively.
7. The semi-supervised cross-subject motor imagery EEG classification method according to claim 3, characterized in that, The geometric constraint loss of the equal-angle tight frame is expressed as: ； in, For the geometric constraint loss of the equiangular tight frame, This is the feature matrix of samples from each motion imagery category in the pseudo-label sample set. for transpose, It is the identity matrix. This indicates the batch size of the labeled samples.
8. The semi-supervised cross-subject motor imagery EEG classification method according to claim 7, characterized in that, The target learning model also includes a domain adaptation module, which is used to calculate the entropy loss for the predicted probability distribution of the target domain samples.
9. The semi-supervised cross-subject motor imagery EEG classification method according to claim 8, characterized in that, During the training of the target learning model, the supervised loss of the source domain samples, the consistency loss, the contrastive learning loss, the isoangular tight frame geometric constraint loss, and the entropy loss are weighted and summed to form the total loss.
10. A semi-supervised EEG classification device for cross-subject motor imagery, characterized in that, The semi-supervised cross-subject motor imagery EEG classification device applies the semi-supervised cross-subject motor imagery EEG classification method according to any one of claims 1-8, and the semi-supervised cross-subject motor imagery EEG classification device comprises: The dataset acquisition module is used to acquire a motor imagery EEG dataset; the motor imagery EEG dataset includes EEG signal data from multiple subjects; The model training module is used to train the target learning model using the motor imagery EEG classification dataset to obtain the motor imagery EEG classification model. During the training process, the target learning model introduces hybrid semi-supervised learning guided by isoangular tight frame and pseudo-label calibration, generates a pseudo-label sample set, and applies isoangular tight frame geometric constraints to the feature matrix formed by the pseudo-label samples of each motor imagery category in the pseudo-label sample set. The model application module is used to perform motor imagery EEG classification using the motor imagery EEG classification model.