Dual-view clustering pseudo-label injection method and system for cross-subject eeg decoding
Patent Information
- Application Number
- CN202610982367.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-02
- Publication Date
- 2026-10-30
AI Technical Summary
这种分布偏移使得在特定样本集上训练的高精度深度学习模型，在迁移至新的、未见过的目标用户时，往往会出现“断崖式”的性能下降
[0038]本发明围绕目标域语义校正构建了样本、原型双视角聚类伪标签机制，突破了现有方法依赖单一路径生成伪标签的局限。通过同时引入整体语义约束与局部判别信息，所生成的伪标签不再仅依赖单一样本预测或单一中心匹配结果，而是在不同信息来源之间形成互补校验。由此，目标域监督信号的准确性与一致性得到提升，错误伪标签在迭代训练中的累积效应得到有效抑制，从而增强了模型对跨被试类别分布偏移的校正能力。
Smart Images
Figure CN123012691A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of artificial intelligence and EEG signal decoding technology, and specifically refers to a cross-subject EEG decoding method and system with dual-view clustering pseudo-label injection. Background Technology
[0002] EEG decoding (such as emotion decoding, motor imagery decoding, and verbal imagery decoding) is a cornerstone topic in EEG computing and advanced human-computer interaction, aiming to build intelligent systems capable of accurately perceiving, quantifying, and understanding human brainwave states. Among numerous biometric decoding technologies, electroencephalography (EEG) has become the most effective means of capturing instantaneous emotional responses of the human central nervous system due to its non-invasiveness, high temporal resolution, and unalterability. With advancements in hardware integration technology, EEG-based decoding technology is rapidly moving from the laboratory to practical applications, including but not limited to mental state monitoring, neurotrophic rehabilitation, active safety control for fatigued driving, and immersive interactive entertainment.
[0003] However, EEG signals are inherently highly nonlinear and non-stationary bioelectrophysiological signals, exhibiting extremely high complexity in terms of time, frequency, and spatial dimensions. In practical large-scale deployments, this field faces a severe bottleneck due to "individual variability." Differences in physiological structure (such as scalp thickness and cortical folding), psychological traits, cognitive differences, and acquisition environments (such as electrode contact impedance and environmental noise) among different subjects lead to drastic shifts in the statistical distribution of their EEG signals, even under the same evoked conditions. This distribution shift often results in a "cliff-like" performance drop when high-precision deep learning models trained on specific sample sets are transferred to new, unseen target users. Summary of the Invention
[0004] To address the technical problems existing in the prior art, this invention provides a cross-subject EEG decoding method and system with dual-view clustering pseudo-label injection, the technical solution of which is as follows: On the one hand, a cross-subject EEG decoding method based on dual-view clustering pseudo-label injection is provided, which includes: S1. Domain Subcluster Construction and Feature Encoding: Obtain EEG differential entropy (DE) samples including multiple source and target domains; wherein, the source domain DE samples have preset task-related auxiliary information labels, and the target domain DE samples are unlabeled data; based on the task-related auxiliary information labels, the source domain DE samples under the same EEG category are divided into multiple source domain task semantic subclusters; cluster the target domain DE samples to obtain multiple target domain clusters; use a deep feature encoder to perform nonlinear feature mapping on the source domain DE samples and target domain DE samples to obtain source domain feature representations and target domain feature representations, respectively; S2. Prototype Calculation: Based on the multiple source domain task semantic subclusters of the source domain DE samples, calculate the source domain prototype corresponding to each source domain task semantic subcluster; based on the clustering results of the target domain clusters, calculate the target domain clustering prototype corresponding to each target domain cluster. S3. Nearest neighbor allocation based on Sinkhorn equilibrium constraint: Introduce a global optimization matching mechanism based on optimal transmission theory, use the nearest neighbor allocation algorithm based on Sinkhorn equilibrium constraint, and combine global equilibrium constraint to correct the local nearest neighbor allocation, and calculate the soft matching matrix between the source domain prototype and the target domain clustering prototype. S4. Dual-Perspective Clustering Pseudo-Label Injection and Collaborative Decision-Making: A dual-perspective clustering pseudo-label injection and collaborative decision-making mechanism is constructed. Based on the soft matching matrix, the category classification of target domain samples is jointly determined from both the prototype perspective and the sample perspective. The prototype perspective focuses on the overall semantic position of the target domain in cross-domain mapping, emphasizing the consistency and stability of category semantics. The sample perspective focuses on the local associations between samples within the target domain, emphasizing the preservation and aggregation of fine-grained discriminative evidence. The semantic certainty of each perspective is evaluated through a confidence gating mechanism, and the perspective with higher confidence is adaptively selected as the final pseudo-label for the target domain sample cluster. S5. Joint Model Optimization: Based on the final pseudo-label, construct and jointly optimize a joint loss function including multiple complementary terms, and perform end-to-end training on the deep feature encoder, domain discriminator and classifier; S6. EEG Decoding: Input the EEG data to be decoded into the trained deep feature encoder to extract feature representations, and input the feature representations into the classifier to output the decoded EEG category.
[0005] Optionally, S1 specifically includes: Suppose that the source domain DE samples are generated from K evoked videos, and each source domain DE sample corresponds to a pre-defined task-related auxiliary information label, containing a total of L EEG categories. First, the source domain DE samples are divided into K source domain task semantic subclusters according to their task-related auxiliary information labels. The k-th source domain task semantic subcluster... Represented as: Task-related auxiliary information
[0006] in, Indicates the first Task-related auxiliary information labels corresponding to each source domain DE sample; On the unlabeled target domain side, the DBSCAN clustering method was used to detect the manifold distribution structure of the target EEG DE data and to divide the target domain DE samples into... J Clusters:
[0007] in, Let represent the j-th cluster in the target domain, ε represent the neighborhood radius, and MinPts represent the minimum number of neighborhood samples required to form the core point. Indicates the target domain. j A cluster; Subsequently, a deep feature encoder was used to perform nonlinear feature mapping on the source and target domain EEG DE data to obtain the source domain feature representations. With target domain feature representation :
[0008]
[0009] in, This refers to the deep feature encoder, which is used to map raw EEG DE data to a high-dimensional feature space. and These represent the number of DE samples in the source domain and the number of DE samples in the target domain, respectively.
[0010] Optionally, S2 specifically includes: Based on the multiple source domain task semantic subclusters of the source domain DE samples, calculate the source domain prototype corresponding to the k-th source domain task semantic subcluster:
[0011] in, Indicates the first The number of source domain samples corresponding to each task-related auxiliary information Indicates the first One source domain prototype; Based on the DBSCAN clustering results of the target domain, calculate the first... The target domain clustering prototype corresponding to each target domain cluster:
[0012] in, This represents the number of samples in the j-th cluster of the target domain. Indicates the first A target domain clustering prototype.
[0013] Optionally, S3 specifically includes: First, calculate the target domain object. With the source domain prototype The cosine or Euclidean distance between them, where the distance is from the prototype's perspective. A=J, from the perspective of the sample , Construct the basic cost matrix :
[0014] Introducing a term with entropy regularization The Sinkhorn algorithm solves for a soft-matching matrix with global equilibrium constraints through an iterative scaling process. S :
[0015] in, For smoothing parameters, and It is a scaling vector; By alternating normalization, S satisfies the global equilibrium constraint:
[0016] in The dimension is A column vector of all 1s. This represents a row vector of dimension K consisting entirely of 1s.
[0017] Optionally, S4 specifically includes: From a prototype perspective, the focus is on the global topological properties of cluster centers, and the target domain clustering prototype is calculated. With the source domain prototype soft matching matrix between Here are some suggestions for pseudo-labels at the macro level:
[0018] Based on the soft allocation matrix Target clustering Belongs to the category of EEG The class probability is represented as the sum of the weights assigned to all source domain task semantic subclusters under that class:
[0019] in, This indicates that it belongs to the category of EEG. The set of source domain-induced semantic cluster indexes; The certainty of a prediction is measured by its confidence level.
[0020] From a sample perspective, pseudo-label suggestions are generated by analyzing the micro-matching relationships between individual samples in the target domain and prototypes in the source domain, targeting the feature representation of the target domain. Calculate its relationship with the source domain prototype. The soft allocation matrix between:
[0021] Based on the soft allocation matrix, the target domain is... The samples belong to the EEG category. The probability is obtained by aggregating the weights assigned to the sample for all source domain prototypes belonging to that category:
[0022] Cluster In-sample predicted probabilities are aggregated by mean to achieve hierarchical alignment with the prototype perspective and enhance robustness.
[0023] Then, high-confidence samples were selected using the average confidence level as a threshold:
[0024] Indicates by The set of high-confidence samples Cluster The maximum class prediction probability of sample i. Cluster The maximum class probability with average confidence. Then, a majority vote is performed on the predicted categories to obtain cluster-level pseudo-labels, which are then broadcast to all samples within the cluster.
[0025] in, This indicates that the category with the highest predicted probability for sample i is selected. The pseudo-label represents the sample in the j-th cluster of the target domain; This computational path, which maps individual samples to cluster-level consensus, can effectively utilize local consistency evidence within the target domain, providing micro-level discriminative support for subsequent dual-perspective verification. After obtaining the class probability distributions from both the prototype and sample perspectives, a confidence gating mechanism is used to evaluate the semantic certainty of each perspective, adaptively filtering out more reliable supervision signals from the dual perspectives, including: For the j-th cluster in the target domain, the highest class probability from each of the two perspectives is used as the confidence measure for its pseudo-label:
[0026] in, This reflects the extent to which the prototype perspective grasps the semantic belonging of the cluster from the perspective of global topology, while This reflects the strength of the judgment based on local consistency within the cluster from the perspective of the sample; Based on the principle of selecting the best option, the perspective with higher confidence is chosen as the final pseudo-label for this cluster:
[0027] Select cluster-level pseudo-labels Broadcast to cluster All samples within the target domain are subject to pseudo-supervisory constraints that ensure class consistency within the manifold space.
[0028] Optionally, the joint loss function is defined as:
[0029] in, The source domain supervised classification loss is represented by the standard cross-entropy loss, and a basic EEG discrimination boundary is established using labeled source domain DE data:
[0030] Indicates domain adversarial alignment loss: Introducing a domain discriminator By reducing the overall offset between the feature distributions of the source and target domains through adversarial training, the deep feature encoder is prompted to extract domain-invariant features.
[0031] The target domain pseudo-label loss is represented by pseudo-labels filtered using confidence gating. Strongly supervised guidance for samples in the target domain:
[0032] Through weighting coefficients and By adjusting the weights of each component, the model can effectively utilize the local manifold structure and global topological information of the target domain while maintaining the transfer of knowledge from the source domain.
[0033] Optionally, the depth feature encoder It consists of a 2-layer fully connected network, which is responsible for the initial dimensionality reduction and feature extraction of the differential entropy (DE) of EEG; The classifier It consists of a 3-layer fully connected network, which enhances the robustness of classification through dimensionality compression and recovery in the middle layer; The domain discriminator It consists of two fully connected network layers, connected to the deep feature encoder through a gradient inversion layer (GRL) to receive features processed by GRL, and uses the fully connected network to determine whether the sample comes from the source domain or the target domain.
[0034] On the other hand, a cross-subject EEG decoding system with dual-view clustering pseudo-label injection is provided, the system comprising: The domain sub-cluster construction and feature encoding module is used to acquire EEG differential entropy (DE) samples including multiple source domains and target domains. The source domain DE samples have preset task-related auxiliary information labels, while the target domain DE samples are unlabeled data. Based on the task-related auxiliary information labels, the source domain DE samples under the same EEG category are divided into multiple source domain task semantic sub-clusters. The target domain DE samples are clustered to obtain multiple target domain clusters. A deep feature encoder is used to perform nonlinear feature mapping on the source domain DE samples and the target domain DE samples to obtain source domain feature representations and target domain feature representations, respectively. The prototype calculation module is used to calculate the source domain prototype corresponding to each source domain task semantic sub-cluster based on multiple source domain task semantic sub-clusters of the source domain DE samples; and to calculate the target domain clustering prototype corresponding to each target domain cluster based on the clustering results of the target domain clusters. The nearest neighbor allocation module based on Sinkhorn equilibrium constraints is used to introduce a global optimization matching mechanism based on optimal transmission theory. It uses the nearest neighbor allocation algorithm based on Sinkhorn equilibrium constraints and combines global equilibrium constraints to correct the local nearest neighbor allocation and calculate the soft matching matrix between the source domain prototype and the target domain clustering prototype. A dual-perspective clustering pseudo-label injection and collaborative decision-making module is used to construct a dual-perspective clustering pseudo-label injection and collaborative decision-making mechanism. Based on the soft matching matrix, it jointly discriminates the category attribution of target domain samples from both prototype and sample perspectives. The prototype perspective focuses on the overall semantic position of the target domain in cross-domain mapping, emphasizing the consistency and stability of category semantics. The sample perspective focuses on the local associations between samples within the target domain, emphasizing the preservation and aggregation of fine-grained discriminative evidence. A confidence gating mechanism is used to evaluate the semantic certainty of each perspective, and the perspective with higher confidence is adaptively selected as the final pseudo-label for the target domain sample cluster. The model joint optimization module is used to construct and jointly optimize a joint loss function including multiple complementary terms based on the final pseudo-label, and to perform end-to-end training on the deep feature encoder, domain discriminator and classifier; The EEG decoding module is used to input the EEG data to be decoded into the trained deep feature encoder to extract feature representations, and input the feature representations into the classifier to output the decoded EEG category.
[0035] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described cross-subject EEG decoding method for dual-view clustering pseudo-label injection.
[0036] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement the above-described cross-subject EEG decoding method for dual-view clustering pseudo-label injection.
[0037] The beneficial effects of the technical solution provided by this invention include at least the following: (1) Improved the reliability of target domain pseudo-label generation and enhanced the ability to correct cross-subject category offset.
[0038] This invention constructs a dual-perspective clustering pseudo-label mechanism based on samples and prototypes for target domain semantic correction, overcoming the limitation of existing methods that rely on a single path to generate pseudo-labels. By simultaneously introducing global semantic constraints and local discriminative information, the generated pseudo-labels no longer depend solely on single sample predictions or single center matching results, but instead form complementary verifications between different information sources. As a result, the accuracy and consistency of the target domain supervision signal are improved, the cumulative effect of erroneous pseudo-labels in iterative training is effectively suppressed, and thus the model's ability to correct for cross-subject category distribution shifts is enhanced.
[0039] (2) Improved the decoding stability and generalization performance of the model in complex cross-subject scenarios.
[0040] Because this invention simultaneously improves the reliability of pseudo-label generation, intra-class structure modeling ability, and global consistency of cross-domain matching, the model's target domain semantic correction process in cross-subject EEG decoding is more stable. The results show that: category discrimination boundaries are clearer, semantic drift is mitigated, and decoding robustness under complex category distributions is improved. Experimental results demonstrate that this method achieves stable improvements on multiple public datasets such as SEED, SEED-IV, and SEED-V, indicating that this invention can effectively enhance the model's cross-individual generalization ability under unlabeled target domain conditions.
[0041] (3) It lowers the application threshold in unlabeled target domain scenarios and enhances the practical application value of the method.
[0042] This invention achieves category semantic correction without manual annotation in the target domain and improves the stability of the unsupervised transfer process through dual-view clustering pseudo-label generation and structured matching mechanisms. This allows the model to obtain relatively reliable decoding results when facing new subjects without the need to construct additional high-cost labeled data. For practical brain-computer interface applications, this effect helps reduce the data dependency and calibration burden when deploying the system to new users, enhancing the usability and promotional value of the method in real-world scenarios. Attached Figure Description
[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0044] Figure 1 This is a flowchart of a cross-subject EEG decoding method for dual-view clustering pseudo-label injection provided by an embodiment of the present invention; Figure 2 This is a general block diagram of a cross-subject EEG decoding method with dual-view clustering pseudo-label injection provided in an embodiment of the present invention; Figure 3 This is a block diagram of a cross-subject EEG decoding system with dual-view clustering pseudo-label injection provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0045] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.
[0046] This invention proposes a dual-view clustering pseudo-label (DVC-PL) mechanism for target domain semantic correction. This mechanism uses two complementary semantic observation methods to jointly determine the category attribution of target domain EEG differential entropy (DE) samples. The prototype perspective focuses on the overall semantic position of the target domain in cross-domain mapping, emphasizing the consistency and stability of category semantics; the sample perspective focuses on the local associations between samples within the target domain, emphasizing the preservation and aggregation of fine-grained discriminative evidence. Specifically, the prototype perspective models the correspondence between the overall representation unit of the target domain and its semantic space in the source domain. This perspective focuses on the central semantic position corresponding to the target domain sample set, thus providing a relatively stable category reference even when cross-domain distribution shifts exist. In contrast, the sample perspective does not directly rely on the overall central representation, but starts from individual samples in the target domain, focusing on the local correspondence between samples and source domain semantic references. This perspective preserves more fine-grained discriminative information, reflecting subtle semantic differences between different samples within the target domain, thereby avoiding information loss caused by excessive smoothing of the overall representation.
[0047] Furthermore, considering that there are usually multiple response patterns induced by different stimuli within the same EEG category, this embodiment of the invention further introduces intraclass structural information to support the effective implementation of the dual-perspective mechanism in EEG scenarios.
[0048] In addition, to ensure the stability of pseudo-labels from both perspectives, embodiments of the present invention further introduce a balanced matching and cluster-level selection strategy.
[0049] Therefore, the embodiments of the present invention aim to solve the target domain category shift problem. With a dual-view clustering pseudo-label mechanism as the core, a more reliable pseudo-supervision is constructed at the sub-cluster level, thereby achieving more stable cross-subject EEG decoding.
[0050] This invention provides a cross-subject EEG decoding method using dual-view clustering pseudo-label injection, which can be implemented by an electronic device, such as a terminal or a server. Figure 1 The flowchart of this method is shown below. Figure 2 The diagram shown is an overall block diagram of this method (taking emotion decoding as an example). The processing flow may include the following steps: S1. Domain Subcluster Construction and Feature Encoding: Obtain EEG Differential Entropy (DE) samples including multiple source and target domains; wherein, the source domain DE samples have preset task-related auxiliary information labels, and the target domain DE samples are unlabeled data; based on the task-related auxiliary information labels, the source domain DE samples under the same EEG category are divided into multiple source domain task semantic subclusters; the target domain DE samples are clustered to obtain multiple target domain clusters; a deep feature encoder is used to perform nonlinear feature mapping on the source domain DE samples and the target domain DE samples to obtain source domain feature representations and target domain feature representations, respectively; When constructing the semantic alignment benchmark of the feature space, this embodiment of the invention combines preset task-related auxiliary information tags to further subdivide the source domain EEG differential entropy (DE) data under the same EEG category into multiple source domain task semantic subclusters.
[0051] The task-related auxiliary information tags may include the following: 1. Differences within the same emotion category caused by different triggering videos, triggering images, triggering audio, triggering text, virtual scenarios, or event stimuli.
[0052] For example, the same emotion of "happiness" can be triggered by different video clips, different pictures, different music, different voices, or different event scenarios.
[0053] Although these materials correspond to the same emotion category label, their specific semantic content, emotional intensity, arousal level, and individual feelings are different. Therefore, corresponding task-related auxiliary information labels are preset, and the corresponding EEG DE samples are divided into multiple sub-clusters within the category.
[0054] 2. Differences within the same active task category caused by early, middle, and late stages of the experiment, cumulative successive trials, repeated training, or fatigue accumulation.
[0055] For example, in the case of "left-hand motor imagery", participants showed good concentration and fast reaction speed in the early stages of the experiment; in the later stages, participants may experience fatigue, decreased attention, or a decline in the quality of motor imagery. Therefore, corresponding task-related auxiliary information labels were preset, and the corresponding EEG DE samples were divided into multiple sub-clusters within the category.
[0056] 3. Within the same active task category, differences arise from variations in participants' attentional state, fatigue level, alertness, tension, task engagement, psychological load, or performance stability.
[0057] For example, even with the same "motor imagery" or "verbal imagery," the subjects' attention, fatigue level, tension level, drowsiness level, and task engagement level may differ in different trials. Therefore, corresponding task-related auxiliary information labels are preset, and the corresponding EEG DE samples are divided into multiple sub-clusters within the category.
[0058] 4. Differences within the same task category caused by variations in reaction time, reaction speed, task accuracy, task completion status, key response results, subjective confirmation results, or behavioral feedback results.
[0059] For example, even for the same type of task, some people react quickly in one trial while others react slowly; some complete the task correctly while others complete it incorrectly; some have high confidence while others have low confidence. Therefore, corresponding task-related auxiliary information labels are preset, and the corresponding EEG DE samples are divided into multiple sub-clusters within the category.
[0060] 5. Differences within the same EEG category caused by variations in trial number, trial order, task start and end phases, task duration, task intervals, or experimental procedure location.
[0061] For example, the EEG states may differ in the first few trials, the middle trials, the last few trials, or the beginning, stable execution, and end stages of a task. Therefore, corresponding task-related auxiliary information labels are preset, and the corresponding EEG DE samples are divided into multiple sub-clusters within the category.
[0062] 6. Differences within the same active task category caused by variations in clarity of action imagery, completeness of verbal imagery, task execution persistence, execution intensity, execution stability, or execution quality.
[0063] For example, in the same "right-hand motor imagery" scenario, some participants' imagery was clearer, while others' was unstable; similarly, in the same "verbal imagery" scenario, some participants' internal speech was more complete, while others' was less consistent. Therefore, corresponding task-related auxiliary information labels were preset, and the corresponding EEG DE samples were divided into multiple sub-clusters within the category.
[0064] Optionally, S1 specifically includes: Suppose that the source domain DE samples are generated from K evoked videos, and each source domain DE sample corresponds to a pre-defined task-related auxiliary information label, containing a total of L EEG categories. First, the source domain DE samples are divided into K source domain task semantic subclusters according to their task-related auxiliary information labels. The k-th source domain task semantic subcluster... Represented as:
[0065] in, Indicates the first Task-related auxiliary information labels corresponding to each source domain DE sample; On the unlabeled target domain side, the DBSCAN clustering method was used to detect the manifold distribution structure of the target EEG DE data (since DBSCAN does not require pre-setting the number of clusters and can automatically discover local structures in the target domain based on sample density, it is suitable for characterizing the manifold distribution of target domain EEG data), and the target domain DE samples were divided into... J Clusters:
[0066] in, Let represent the j-th cluster in the target domain, ε represent the neighborhood radius, and MinPts represent the minimum number of neighborhood samples required to form the core point. Indicates the target domain. j A cluster; Subsequently, a deep feature encoder was used to perform nonlinear feature mapping on the source and target domain EEG DE data to obtain the source domain feature representations. With target domain feature representation :
[0067]
[0068] in, This refers to the deep feature encoder, which is used to map raw EEG DE data to a high-dimensional feature space. and These represent the number of DE samples in the source domain and the number of DE samples in the target domain, respectively.
[0069] S2. Prototype Calculation: Based on the multiple source domain task semantic subclusters of the source domain DE samples, calculate the source domain prototype corresponding to each source domain task semantic subcluster; based on the clustering results of the target domain clusters, calculate the target domain clustering prototype corresponding to each target domain cluster. Optionally, S2 specifically includes: Based on the multiple source domain task semantic subclusters of the source domain DE samples, calculate the source domain prototype corresponding to the k-th source domain task semantic subcluster:
[0070] in, Indicates the first The number of source domain samples corresponding to each task-related auxiliary information Indicates the first One source domain prototype; Based on the DBSCAN clustering results of the target domain, calculate the first... The target domain clustering prototype corresponding to each target domain cluster:
[0071] in, This represents the number of samples in the j-th cluster of the target domain. Indicates the first A target domain clustering prototype.
[0072] Through the above process, the source domain no longer relies solely on coarse-grained EEG categories as semantic references, but further constructs a fine-grained semantic prototype formed by different task-related auxiliary information; the target domain obtains its own local manifold structure through DBSCAN clustering. Together, they provide a more detailed topological foundation for subsequent cross-domain semantic matching. This partitioning method fully considers the differences in activation patterns of EEG signals under different task-related auxiliary information, helping to alleviate the problem of traditional category-level alignment methods ignoring intra-class multimodal distribution.
[0073] S3. Nearest neighbor allocation based on Sinkhorn equilibrium constraint: Introduce a global optimization matching mechanism based on optimal transmission theory, use the nearest neighbor allocation algorithm based on Sinkhorn equilibrium constraint, and combine global equilibrium constraint to correct the local nearest neighbor allocation, and calculate the soft matching matrix between the source domain prototype and the target domain clustering prototype. Optionally, S3 specifically includes: First, calculate the target domain object. With the source domain prototype The cosine or Euclidean distance between them, where the distance is from the prototype's perspective. A=J, from the perspective of the sample , Construct the basic cost matrix :
[0074] Introducing a term with entropy regularization The Sinkhorn algorithm (to overcome nonlinear bias caused by individual differences in cross-subject scenarios) solves for a soft-matching matrix with global equilibrium constraints through an iterative scaling process. S :
[0075] in, For smoothing parameters, and It is a scaling vector; By alternating normalization, S satisfies the global equilibrium constraint:
[0076] in The dimension is A column vector of all 1s. This represents a row vector of dimension K consisting entirely of 1s.
[0077] The key to this step lies in using global balance constraints to correct the traditional local nearest neighbor allocation logic. This forces the model to consider the completeness of class distribution globally, ensuring that the target domain prototype can accurately lock onto the logically corresponding source domain prototype. This balanced allocation mechanism effectively prevents class collapse caused by the clustering of target domain predictions towards a few strong feature points.
[0078] S4. Dual-Perspective Clustering Pseudo-Label Injection and Collaborative Decision-Making: A dual-perspective clustering pseudo-label injection and collaborative decision-making mechanism is constructed. Based on the soft matching matrix, the category classification of target domain samples is jointly determined from both the prototype perspective and the sample perspective. The prototype perspective focuses on the overall semantic position of the target domain in cross-domain mapping, emphasizing the consistency and stability of category semantics. The sample perspective focuses on the local associations between samples within the target domain, emphasizing the preservation and aggregation of fine-grained discriminative evidence. The semantic certainty of each perspective is evaluated through a confidence gating mechanism, and the perspective with higher confidence is adaptively selected as the final pseudo-label for the target domain sample cluster. Optionally, S4 specifically includes: From a prototype perspective, the focus is on the global topological properties of cluster centers, and the target domain clustering prototype is calculated. With the source domain prototype soft matching matrix between Here are some suggestions for pseudo-labels at the macro level:
[0079] Based on the soft allocation matrix Target clustering Belongs to the category of EEG The class probability is represented as the sum of the weights assigned to all source domain task semantic subclusters under that class:
[0080] in, This indicates that it belongs to the category of EEG. The set of source domain-induced semantic cluster indexes; The certainty of a prediction is measured by its confidence level.
[0081] From a sample perspective, pseudo-label suggestions are generated by analyzing the micro-matching relationships between individual samples in the target domain and prototypes in the source domain, targeting the feature representation of the target domain. Calculate its relationship with the source domain prototype. The soft allocation matrix between:
[0082] Based on the soft allocation matrix, the target domain is... The samples belong to the EEG category. The probability is obtained by aggregating the weights assigned to the sample for all source domain prototypes belonging to that category:
[0083] Cluster In-sample predicted probabilities are aggregated by mean to achieve hierarchical alignment with the prototype perspective and enhance robustness.
[0084] Then, high-confidence samples were selected using the average confidence level as a threshold:
[0085] Indicates by The set of high-confidence samples Cluster The maximum class prediction probability of sample i. Cluster The maximum class probability with average confidence. Then, a majority vote is performed on the predicted categories to obtain cluster-level pseudo-labels, which are then broadcast to all samples within the cluster.
[0086] in, This indicates that the category with the highest predicted probability for sample i is selected. The pseudo-label represents the sample in the j-th cluster of the target domain; This computational path, which maps individual samples to cluster-level consensus, can effectively utilize local consistency evidence within the target domain, providing micro-level discriminative support for subsequent dual-perspective verification. After obtaining the class probability distributions from both the prototype and sample perspectives, a confidence gating mechanism is used to evaluate the semantic certainty of each perspective, adaptively filtering out more reliable supervision signals from the dual perspectives, including: For the j-th cluster in the target domain, the highest class probability from each of the two perspectives is used as the confidence measure for its pseudo-label:
[0087] in, This reflects the extent to which the prototype perspective grasps the semantic belonging of the cluster from the perspective of global topology, while This reflects the strength of the judgment based on local consistency within the cluster from the perspective of the sample; Based on the principle of selecting the best option, the perspective with higher confidence is chosen as the final pseudo-label for this cluster:
[0088] Select cluster-level pseudo-labels Broadcast to cluster All samples within the target domain are subject to pseudo-supervisory constraints that ensure class consistency within the manifold space.
[0089] This mechanism suppresses semantic drift in self-supervised learning through multi-dimensional logical verification. Its advantage is that when the manifold structure of the target domain cluster is clear and the local consistency is strong, the model can keenly capture micro-evidence from the sample perspective; while when there is ambiguity in the local discrimination, the model tends to rely on the overall semantic alignment result from the prototype perspective.
[0090] S5. Joint Model Optimization: Based on the final pseudo-label, construct and jointly optimize a joint loss function including multiple complementary terms, and perform end-to-end training on the deep feature encoder, domain discriminator and classifier; Optionally, the joint loss function is defined as:
[0091] in, The source domain supervised classification loss is represented by the standard cross-entropy loss, and a basic EEG discrimination boundary is established using labeled source domain DE data:
[0092] Indicates domain adversarial alignment loss: Introducing a domain discriminator By reducing the overall offset between the feature distributions of the source and target domains through adversarial training, the deep feature encoder is prompted to extract domain-invariant features.
[0093] The target domain pseudo-label loss is represented by pseudo-labels filtered using confidence gating. Strongly supervised guidance for samples in the target domain:
[0094] Through weighting coefficients and By adjusting the weights of each component, the model can effectively utilize the local manifold structure and global topological information of the target domain while maintaining the transfer of knowledge from the source domain.
[0095] This multi-objective joint optimization strategy not only improves the model's robustness to differences across individual participants, but also effectively suppresses the risk of semantic drift during the pseudo-label generation process through a dual-perspective verification mechanism.
[0096] Optionally, the depth feature encoder It consists of a 2-layer fully connected network, which is responsible for the initial dimensionality reduction and feature extraction of the differential entropy (DE) of EEG; The classifier It consists of a 3-layer fully connected network, which enhances the robustness of classification through dimensionality compression and recovery in the middle layer; The domain discriminator It consists of two fully connected network layers, connected to the deep feature encoder through a gradient inversion layer (GRL) to receive features processed by GRL, and uses the fully connected network to determine whether the sample comes from the source domain or the target domain.
[0097] S6. EEG Decoding: Input the EEG data to be decoded into the trained deep feature encoder to extract feature representations, and input the feature representations into the classifier to output the decoded EEG category.
[0098] To more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be further described in detail below: Experimental data: SEED-III: This dataset uses 15 film clips as evoked stimuli, covering three basic emotional states: positive, neutral, and negative. It is often used to validate algorithm performance on standard three-class EEG decoding tasks, with the evoked stimuli serving as task-related auxiliary information.
[0099] SEED-IV: This dataset further increases the task difficulty by selecting 24 evoked stimuli covering four emotions: positive, neutral, negative, and fear. This dataset requires the model to distinguish more subtle differences in physiological electrical signals, particularly in capturing the strong emotional stimulus of "fear," treating the evoked stimuli as task-related auxiliary information for this dataset.
[0100] SEED-V is one of the most challenging datasets currently available. It selects 15 evoked stimuli, encompassing five emotions: positive, neutral, negative, fear, and disgust. The evoked stimuli are used as task-related auxiliary information in this dataset. As the number of categories increases, the boundaries in the feature space become extremely blurred. This invention's embodiment effectively solves the semantic confusion problem in the five-class classification task by constructing fine-grained subclusters.
[0101] Experimental details: Feature engineering: All experiments used DE (Differential Entropy) features. Feature extraction was performed across five frequency bands (delta, theta, alpha, beta, gamma) to ensure the integrity of the signal in the frequency domain.
[0102] Experimental Procedure: A rigorous leave-one-out-of-subjects method was employed. Within each experimental period, all test samples from one subject were sequentially selected as the target domain (unlabeled), while the historical data of the remaining subjects were used as the source domain (labeled). This setup simulates the new user onboarding scenario encountered in real-world applications of brain-computer interfaces.
[0103] Computational equipment and metrics: The experiment was performed under the constraint of a joint loss function, with end-to-end optimization. Decoding accuracy was used as the core metric, and the standard deviation was recorded to measure the stability of the model across different individuals.
[0104] Experimental results: By comparing the Dual-View Clustering Pseudo-Label (DVC-PL) model proposed in this invention with the best-performing algorithms in the field of Brain-Computer Interface (BCI) over the past few years, the results show that this invention has achieved breakthroughs in all dimensions.
[0105]
[0106] like Figure 3 As shown, this embodiment of the invention also provides a cross-subject EEG decoding system with dual-view clustering pseudo-label injection, the system comprising: The domain sub-cluster construction and feature encoding module 310 is used to acquire EEG differential entropy (DE) samples including multiple source domains and target domains; wherein, the source domain DE samples have preset task-related auxiliary information labels, and the target domain DE samples are unlabeled data; based on the task-related auxiliary information labels, the source domain DE samples under the same EEG category are divided into multiple source domain task semantic sub-clusters; the target domain DE samples are clustered to obtain multiple target domain clusters; and a deep feature encoder is used to perform nonlinear feature mapping on the source domain DE samples and the target domain DE samples to obtain source domain feature representations and target domain feature representations, respectively. The prototype calculation module 320 is used to calculate the source domain prototype corresponding to each source domain task semantic sub-cluster based on multiple source domain task semantic sub-clusters of the source domain DE samples; and to calculate the target domain clustering prototype corresponding to each target domain cluster based on the clustering results of the target domain clusters. The nearest neighbor allocation module 330 based on Sinkhorn equilibrium constraints is used to introduce a global optimization matching mechanism based on optimal transmission theory. It uses the nearest neighbor allocation algorithm based on Sinkhorn equilibrium constraints and combines global equilibrium constraints to correct the local nearest neighbor allocation and calculate the soft matching matrix between the source domain prototype and the target domain clustering prototype. The dual-view clustering pseudo-label injection and collaborative decision-making module 340 is used to construct a dual-view clustering pseudo-label injection and collaborative decision-making mechanism. Based on the soft matching matrix, it jointly discriminates the category attribution of target domain samples from the prototype perspective and the sample perspective. The prototype perspective focuses on the overall semantic position of the target domain in cross-domain mapping, emphasizing the consistency and stability of category semantics. The sample perspective focuses on the local associations between samples within the target domain, emphasizing the preservation and aggregation of fine-grained discriminative evidence. The semantic certainty of each perspective is evaluated through a confidence gating mechanism, and the perspective with higher confidence is adaptively selected as the final pseudo-label for the target domain sample cluster. The model joint optimization module 350 is used to construct and jointly optimize a joint loss function including multiple complementary terms based on the final pseudo label, and to perform end-to-end training on the deep feature encoder, domain discriminator and classifier; The EEG decoding module 360 is used to input the EEG DE data to be decoded into the trained deep feature encoder to extract feature representations, and input the feature representations into the classifier to output the decoded EEG category.
[0107] The cross-subject EEG decoding system with dual-view clustering pseudo-label injection provided in this embodiment of the invention has a functional structure that corresponds to the cross-subject EEG decoding method with dual-view clustering pseudo-label injection provided in this embodiment of the invention, and will not be described again here.
[0108] Figure 4This is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present invention. The electronic device 400 may vary considerably due to different configurations or performance. It may include one or more central processing units (CPUs) 401 and one or more memories 402. The memory 402 stores at least one instruction, which is loaded and executed by the processor 401 to implement the steps of the above-described dual-view clustering pseudo-label injection cross-subject EEG decoding method.
[0109] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to complete the aforementioned cross-subject EEG decoding method for dual-view clustering pseudo-label injection. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.
[0110] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A cross-subject EEG decoding method using dual-view clustering pseudo-label injection, characterized in that, The method includes: S1. Domain Subcluster Construction and Feature Encoding: Obtain EEG differential entropy (DE) samples including multiple source and target domains; wherein, the source domain DE samples have preset task-related auxiliary information labels, and the target domain DE samples are unlabeled data; based on the task-related auxiliary information labels, the source domain DE samples under the same EEG category are divided into multiple source domain task semantic subclusters; cluster the target domain DE samples to obtain multiple target domain clusters; use a deep feature encoder to perform nonlinear feature mapping on the source domain DE samples and target domain DE samples to obtain source domain feature representations and target domain feature representations, respectively; S2. Prototype Calculation: Based on the multiple source domain task semantic subclusters of the source domain DE samples, calculate the source domain prototype corresponding to each source domain task semantic subcluster; based on the clustering results of the target domain clusters, calculate the target domain clustering prototype corresponding to each target domain cluster. S3. Nearest neighbor allocation based on Sinkhorn equilibrium constraint: Introduce a global optimization matching mechanism based on optimal transmission theory, use the nearest neighbor allocation algorithm based on Sinkhorn equilibrium constraint, and combine global equilibrium constraint to correct the local nearest neighbor allocation, and calculate the soft matching matrix between the source domain prototype and the target domain clustering prototype. S4. Dual-Perspective Clustering Pseudo-Label Injection and Collaborative Decision-Making: A dual-perspective clustering pseudo-label injection and collaborative decision-making mechanism is constructed. Based on the soft matching matrix, the category classification of target domain samples is jointly determined from both the prototype perspective and the sample perspective. The prototype perspective focuses on the overall semantic position of the target domain in cross-domain mapping, emphasizing the consistency and stability of category semantics. The sample perspective focuses on the local associations between samples within the target domain, emphasizing the preservation and aggregation of fine-grained discriminative evidence. The semantic certainty of each perspective is evaluated through a confidence gating mechanism, and the perspective with higher confidence is adaptively selected as the final pseudo-label for the target domain sample cluster. S5. Joint Model Optimization: Based on the final pseudo-label, construct and jointly optimize a joint loss function including multiple complementary terms, and perform end-to-end training on the deep feature encoder, domain discriminator and classifier; S6. EEG Decoding: Input the EEG data to be decoded into the trained deep feature encoder to extract feature representations, and input the feature representations into the classifier to output the decoded EEG category.
2. The method according to claim 1, characterized in that, S1 specifically includes: Suppose that the source domain DE samples are generated from K evoked videos, and each source domain DE sample corresponds to a task-related auxiliary information label, containing a total of L EEG categories. First, the source domain DE samples are divided into K source domain task semantic subclusters according to their task-related auxiliary information labels. The k-th source domain task semantic subcluster... Represented as: in, Indicates the first Task-related auxiliary information labels corresponding to each source domain DE sample; On the unlabeled target domain side, the DBSCAN clustering method was used to detect the manifold distribution structure of the target EEG DE data and to divide the target domain DE samples into... J Clusters: in, Let represent the j-th cluster in the target domain, ε represent the neighborhood radius, and MinPts represent the minimum number of neighborhood samples required to form the core point. Indicates the target domain. j A cluster; Subsequently, a deep feature encoder was used to perform nonlinear feature mapping on the source and target domain EEG DE data to obtain the source domain feature representations. With target domain feature representation : in, This refers to the deep feature encoder, which is used to map raw EEG DE data to a high-dimensional feature space. and These represent the number of DE samples in the source domain and the number of DE samples in the target domain, respectively.
3. The method according to claim 2, characterized in that, S2 specifically includes: Based on the multiple source domain task semantic subclusters of the source domain DE samples, calculate the source domain prototype corresponding to the k-th source domain task semantic subcluster: in, Indicates the first The number of source domain samples corresponding to each task-related auxiliary information Indicates the first One source domain prototype; Based on the DBSCAN clustering results of the target domain, calculate the first... The target domain clustering prototype corresponding to each target domain cluster: in, This represents the number of samples in the j-th cluster of the target domain. Indicates the first A target domain clustering prototype.
4. The method according to claim 3, characterized in that, S3 specifically includes: First, calculate the target domain object. With the source domain prototype The cosine or Euclidean distance between them, where the distance is from the prototype's perspective. A=J, from the perspective of the sample , Construct the basic cost matrix : Introducing a term with entropy regularization The Sinkhorn algorithm solves for a soft-matching matrix with global equilibrium constraints through an iterative scaling process. S : in, For smoothing parameters, and It is a scaling vector; By alternating normalization, S satisfies the global equilibrium constraint: in The dimension is A column vector of all 1s. This represents a row vector of dimension K consisting entirely of 1s.
5. The method according to claim 4, characterized in that, S4 specifically includes: From a prototype perspective, the focus is on the global topological properties of cluster centers, and the target domain clustering prototype is calculated. With the source domain prototype soft matching matrix between Here are some suggestions for pseudo-labels at the macro level: Based on the soft allocation matrix Target clustering Belongs to the category of EEG The class probability is represented as the sum of the weights assigned to all source domain task semantic subclusters under that class: in, This indicates that it belongs to the category of EEG. The set of source domain-induced semantic cluster indexes; The certainty of a prediction is measured by its confidence level. From a sample perspective, pseudo-label suggestions are generated by analyzing the micro-matching relationships between individual samples in the target domain and prototypes in the source domain, targeting the feature representation of the target domain. Calculate its relationship with the source domain prototype. The soft allocation matrix between: Based on the soft allocation matrix, the target domain is... The samples belong to the EEG category. The probability is obtained by aggregating the weights assigned to the sample for all source domain prototypes belonging to that category: Cluster In-sample predicted probabilities are aggregated by mean to achieve hierarchical alignment with the prototype perspective and enhance robustness. Then, high-confidence samples were selected using the average confidence level as a threshold: Indicates by The set of high-confidence samples Cluster The maximum class prediction probability of sample i. Cluster The maximum class probability with average confidence. Then, a majority vote is performed on the predicted categories to obtain cluster-level pseudo-labels, which are then broadcast to all samples within the cluster. in, This indicates that the category with the highest predicted probability for sample i is selected. The pseudo-label represents the sample in the j-th cluster of the target domain; This computational path, which maps individual samples to cluster-level consensus, can effectively utilize local consistency evidence within the target domain, providing micro-level discriminative support for subsequent dual-perspective verification. After obtaining the class probability distributions from both the prototype and sample perspectives, a confidence gating mechanism is used to evaluate the semantic certainty of each perspective, adaptively filtering out more reliable supervision signals from the dual perspectives, including: For the j-th cluster in the target domain, the highest class probability from each of the two perspectives is used as the confidence measure for its pseudo-label: in, This reflects the extent to which the prototype perspective grasps the semantic belonging of the cluster from the perspective of global topology, while This reflects the strength of the judgment based on local consistency within the cluster from the perspective of the sample; Based on the principle of selecting the best option, the perspective with higher confidence is chosen as the final pseudo-label for this cluster: Selected cluster-level pseudo-labels Broadcast to cluster All samples within the manifold space are subject to pseudo-supervisory constraints that ensure class consistency of the target domain within the manifold space.
6. The method according to claim 5, characterized in that, The joint loss function is defined as follows: in, The source domain supervised classification loss is represented by the standard cross-entropy loss, and a basic EEG discrimination boundary is established using labeled source domain DE data: Indicates domain adversarial alignment loss: Introducing a domain discriminator By reducing the overall offset between the feature distributions of the source and target domains through adversarial training, the deep feature encoder is prompted to extract domain-invariant features. The target domain pseudo-label loss is represented by pseudo-labels filtered using confidence gating. Strongly supervised guidance for samples in the target domain: Through weighting coefficients and By adjusting the weights of each component, the model can effectively utilize the local manifold structure and global topological information of the target domain while maintaining the transfer of knowledge from the source domain.
7. The method according to claim 1, characterized in that, The depth feature encoder It consists of a 2-layer fully connected network, which is responsible for the initial dimensionality reduction and feature extraction of the differential entropy (DE) of EEG; The classifier It consists of a 3-layer fully connected network, which enhances the robustness of classification through dimensionality compression and recovery in the middle layer; The domain discriminator It consists of two fully connected network layers, connected to the deep feature encoder through a gradient inversion layer (GRL) to receive features processed by GRL, and uses the fully connected network to determine whether the sample comes from the source domain or the target domain.
8. A cross-subject EEG decoding system with dual-view clustering pseudo-label injection, characterized in that, The system includes: The domain sub-cluster construction and feature encoding module is used to acquire EEG differential entropy (DE) samples including multiple source domains and target domains. The source domain DE samples have preset task-related auxiliary information labels, while the target domain DE samples are unlabeled data. Based on the task-related auxiliary information labels, the source domain DE samples under the same EEG category are divided into multiple source domain task semantic sub-clusters. The target domain DE samples are clustered to obtain multiple target domain clusters. A deep feature encoder is used to perform nonlinear feature mapping on the source domain DE samples and the target domain DE samples to obtain source domain feature representations and target domain feature representations, respectively. The prototype calculation module is used to calculate the source domain prototype corresponding to each source domain task semantic sub-cluster based on multiple source domain task semantic sub-clusters of the source domain DE samples; and to calculate the target domain clustering prototype corresponding to each target domain cluster based on the clustering results of the target domain clusters. The nearest neighbor allocation module based on Sinkhorn equilibrium constraints is used to introduce a global optimization matching mechanism based on optimal transmission theory. It uses the nearest neighbor allocation algorithm based on Sinkhorn equilibrium constraints and combines global equilibrium constraints to correct the local nearest neighbor allocation and calculate the soft matching matrix between the source domain prototype and the target domain clustering prototype. A dual-perspective clustering pseudo-label injection and collaborative decision-making module is used to construct a dual-perspective clustering pseudo-label injection and collaborative decision-making mechanism. Based on the soft matching matrix, it jointly discriminates the category attribution of target domain samples from both prototype and sample perspectives. The prototype perspective focuses on the overall semantic position of the target domain in cross-domain mapping, emphasizing the consistency and stability of category semantics. The sample perspective focuses on the local associations between samples within the target domain, emphasizing the preservation and aggregation of fine-grained discriminative evidence. A confidence gating mechanism is used to evaluate the semantic certainty of each perspective, and the perspective with higher confidence is adaptively selected as the final pseudo-label for the target domain sample cluster. The model joint optimization module is used to construct and jointly optimize a joint loss function including multiple complementary terms based on the final pseudo-label, and to perform end-to-end training on the deep feature encoder, domain discriminator and classifier; The EEG decoding module is used to input the EEG data to be decoded into the trained deep feature encoder to extract feature representations, and input the feature representations into the classifier to output the decoded EEG category.
9. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the cross-subject EEG decoding method for dual-view clustering pseudo-label injection as described in any one of claims 1-7.
10. A computer-readable storage medium storing at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the cross-subject EEG decoding method for dual-view clustering pseudo-label injection as described in any one of claims 1-7.