Electroencephalogram signal domain generalization classification method based on two-stage multi-scale neural network

Abstract

The invention relates to an electroencephalogram signal domain generalization classification method based on a two-stage multi-scale neural network. The method comprises the following steps: collecting motor imagery electroencephalogram signals of multiple subjects and preprocessing the motor imagery electroencephalogram signals; constructing a two-stage multi-scale neural network (TSMSNet), and capturing complementary spatio-temporal features through a parallel branch structure of a differential time sequence convolution kernel; a dynamic weighted fusion module (WFC) driven by a verification set is adopted to realize adaptive feature fusion, and contribution of each branch is balanced based on Softmax normalized weight distribution; a joint supervision optimization module (ABL) is introduced to provide an auxiliary supervision signal through feature splicing classification, and system integration of multi-scale feature extraction, dynamic weighted fusion and joint supervision optimization is realized for the first time, so that the cross-subject classification performance is improved to a new height; the key problem of a traditional end-to-end method in the aspects of gradient conflict and branch contribution quantification is solved through a two-stage training strategy, and a reliable technical scheme is provided for clinical application of a zero-calibration brain-computer interface system.

Description

Technical Field

[0001] This invention relates to a generalized classification method for EEG signal domains based on a two-stage multi-scale neural network, belonging to the field of brain-computer interface (BCI) technology, and is applicable to zero-calibration classification tasks across subject motor imagery EEG signals (MI-EEG). Background Technology

[0002] Brain-computer interface (BCI) technology enables human-computer interaction by analyzing the brain's neural electrical activity, among which decoding motor imagery (MI) EEG signals is a core challenge. Existing methods mainly use multi-scale convolutional neural networks to extract spatiotemporal features and improve cross-subject adaptability through parallel branching structures. However, these methods have three key bottlenecks: (1) fixed-scale convolutional kernels are difficult to simultaneously handle the feature extraction of transient events (such as motor command triggering) and continuous rhythms (such as μ / β wave modulation); (2) end-to-end joint training leads to gradient conflicts between branches, resulting in redundant feature representation; (3) static linear fusion strategies cannot dynamically adjust feature weights according to subject-specific characteristics, with an average accuracy of less than 60% on datasets such as BCI Competition IV 2a. More seriously, the non-stationary nature of EEG signals leads to significant differences in distribution among subjects (KL divergence > 2.5), and the performance degradation of traditional models on new subjects reaches 15-20%.

[0003] In recent years, domain generalization techniques have provided new ideas for cross-subject decoding, such as maximum mean difference (MMD) alignment and meta-learning strategies. These methods learn invariant feature representations through source domain data, achieving approximately 58.5% zero-calibration accuracy on the PhysioNet dataset. However, existing solutions still have two major drawbacks: (1) the feature alignment process is prone to losing key discriminative information of the task, such as the spatiotemporal pattern of event-related desynchronization (ERD); (2) most methods rely on complex adversarial training, and the computational complexity increases exponentially when the number of sampling points exceeds 1000. Although dynamic weighted fusion mechanisms (such as the WFC module) partially solve these problems through validation set-driven Softmax weight allocation, the robustness of the model still needs to be improved when dealing with heterogeneous data collected from multiple devices. Summary of the Invention

[0004] To overcome the technical bottlenecks of existing cross-subject motor imagery EEG signal classification methods in areas such as multi-scale feature fusion, domain generalization optimization, and dynamic weight allocation, this invention provides an EEG signal domain generalization classification method based on a two-stage multi-scale neural network. It innovatively adopts a staged training strategy, captures complementary spatiotemporal features through differentiated temporal convolution kernels, and combines a validation set-driven dynamic weighted fusion mechanism and a joint supervised optimization framework. This significantly improves the model's feature representation ability and classification robustness for unknown subject EEG signals without requiring any target domain data calibration.

[0005] The generalization classification method for EEG signal domain based on a two-stage multi-scale neural network includes the following steps:

[0006] Step 1: Use EEG acquisition equipment to collect EEG signals of motor imagery from multiple subjects;

[0007] Step 2: Preprocess the acquired EEG signals, including filtering, noise reduction, and segmentation;

[0008] Step 3: Construct a two-stage multi-scale neural network (TSMSNet), including:

[0009] Multi-scale classification network module: It uses a parallel branching structure to capture complementary spatiotemporal features and make independent decisions;

[0010] Dynamic weighted fusion module: Trains the model in one stage and generates adaptive generalization weights based on the accuracy of the validation set;

[0011] Joint supervised optimization module: It integrates branch features for decision-making and introduces auxiliary classification loss to guide features to reach the global optimal solution;

[0012] Step 4: Train the two-stage multi-scale neural network model in stages;

[0013] Step 5: Perform zero-calibration classification prediction on the EEG signals of subjects who are not seen as targets.

[0014] Step three specifically includes:

[0015] S3.1: The multi-scale classification network module uses differentiated temporal convolutional kernels. Specifically, it constructs four parallel branches, each configured with a depthwise separable convolutional kernel of (1, FS / 16), (1, FS / 8), (1, FS / 4), and (1, FS / 2), respectively, where FS is the sampling frequency of the EEG acquisition device. Each branch independently optimizes the cross-entropy loss.

[0016]

[0017] Where k represents the branch index. Let y represent the cross-entropy loss of the k-th branch, where N represents the total number of training samples.i This represents the true label of the i-th sample. This represents the predicted probability distribution of the k-th branch for the i-th sample. Each convolutional layer is followed by a batch normalization and Dropout (p=0.3) layer.

[0018] S3.2: The dynamic weighted fusion module adopts a validation set-driven Softmax weight allocation, specifically implemented as follows:

[0019] Validation accuracy α of each branch k The fusion weights are generated after adjusting the temperature parameter τ:

[0020]

[0021] Where k represents the branch index, K represents the total number of branches, and w k Let α represent the dynamic fusion weight of the k-th branch. k Let α represent the classification accuracy of the k-th branch on the validation set. j This represents the classification accuracy of the j-th branch on the validation set, and τ represents the temperature parameter (which controls the smoothness of the weight distribution).

[0022] The final decision is a weighted sum:

[0023]

[0024] Where k represents the branch index. w represents the predicted probability distribution after final weighted fusion. k This represents the dynamic fusion weight of the k-th branch. Let represent the predicted probability distribution of the k-th branch.

[0025] Corresponding classification loss:

[0026]

[0027] in The loss represents the main classification loss after fusion, where N represents the total number of training samples, and y represents the total number of training samples. i This represents the true label of the i-th sample. This represents the predicted probability distribution after final weighted fusion.

[0028] S3.3: The joint supervision optimization module generates auxiliary supervision signals by fusing branch features and making decisions. The specific implementation method is as follows:

[0029] The multi-branch feature hk is concatenated and then the loss is calculated using an auxiliary classifier.

[0030]

[0031] in Let y represent the auxiliary classification loss, N represent the total number of training samples, and y represent the total number of training samples. i This represents the true label of the i-th sample. This represents the prediction result of the auxiliary classifier after concatenating all branch features.

[0032] The total loss function is a weighted sum:

[0033]

[0034] in This represents the total loss function of the model, where λ∈(0,1) are the balance coefficients. This represents the main classification loss after fusion. This represents the auxiliary classification loss.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] The two-stage multi-scale neural network (TSMSNet) model proposed in this invention has achieved a major breakthrough in the task of classifying EEG signals of motor imagery across subjects by innovatively integrating multi-scale spatiotemporal feature extraction and dynamic weighted decision fusion mechanism.

[0037] Experimental results show that, on the three benchmark datasets BCI Competition IV 2a, 2b and PhysioNet, the average classification accuracy is improved by 7.54%, 1.14% and 2.14% respectively compared with the existing best methods, and the highest Kappa coefficient reaches 0.5077.

[0038] The innovative architecture of this invention achieves multi-scale feature capture through differentiated temporal convolutional kernels (15-125 sampling points). Small-scale convolutional kernels focus on transient event detection, while large-scale convolutional kernels capture the continuous modulation patterns of μ / β rhythms. Simultaneously, a validation-set-driven Softmax weight allocation mechanism is employed, dynamically adjusting the fusion weights based on the performance of each branch on the validation set, effectively solving the information dilution problem caused by traditional fixed fusion strategies. The joint supervised optimization module provides auxiliary supervision signals through feature concatenation classification, guiding the features of each branch to converge towards the global optimum, significantly enhancing the model's domain generalization ability. Regarding model interpretability, such as… Figure 2 As shown, SHAP analysis confirmed the complementary nature of the multi-branched structure, such as the symmetrical activation pattern of the C3 / C4 electrodes in the left and right hand imagery tasks, which conforms to the neural mechanism.

[0039] This invention is the first to achieve a system integration of multi-scale feature extraction, dynamic weighted fusion, and joint supervised optimization. It not only improves cross-subject classification performance to a new level, but also solves the key problems of gradient conflict and branch contribution quantification in traditional end-to-end methods through a two-stage training strategy, providing a reliable technical solution for the clinical application of zero-calibration brain-computer interface systems. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a diagram of the TSMSNet model framework for the EEG signal domain generalization classification method based on a two-stage multi-scale neural network, as described in this invention.

[0042] Figure 2 This is a diagram illustrating the interpretability analysis of the BCICompetition IV 2b dataset for the EEG signal domain generalization classification method based on a two-stage multi-scale neural network, as described in this invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Reference Figure 1 The diagram shows the TSMSNet model framework for a two-stage multi-scale neural network-based generalized classification method for the EEG signal domain. This method includes the following steps:

[0045] Step 1: Collect EEG signals of motor imagery from multiple participants using an EEG acquisition device (using the BCICompetition IV 2a dataset as an example);

[0046] Step 2: Preprocess the acquired EEG signals, including filtering, noise reduction, and segmentation;

[0047] 1) 0.5-100Hz bandpass filter;

[0048] 2) 50Hz notch filter;

[0049] 3) Extract the imagined movement range of -0.5s to 4s after the segmented prompt appears.

[0050] Step 3: Construct a two-stage multi-scale neural network (TSMSNet), including:

[0051] Multi-scale classification network module: It uses a parallel branching structure to capture complementary spatiotemporal features and make independent decisions;

[0052] Dynamic weighted fusion module: Trains the model in one stage and generates adaptive generalization weights based on the accuracy of the validation set;

[0053] Joint supervised optimization module: It integrates branch features for decision-making and introduces auxiliary classification loss to guide features to reach the global optimal solution;

[0054] Step three specifically includes:

[0055] S3.1: The multi-scale classification network module uses differentiated temporal convolutional kernels. Specifically, it constructs four parallel branches, each configured with a depthwise separable convolutional kernel of (1, FS / 16), (1, FS / 8), (1, FS / 4), and (1, FS / 2), respectively, where FS is the sampling frequency of the EEG acquisition device. Each branch independently optimizes the cross-entropy loss.

[0056]

[0057] Where k represents the branch index. Let y represent the cross-entropy loss of the k-th branch, where N represents the total number of training samples. i This represents the true label of the i-th sample. This represents the predicted probability distribution of the k-th branch for the i-th sample. Each convolutional layer is followed by a batch normalization and Dropout (p=0.3) layer.

[0058] S3.2: The dynamic weighted fusion module adopts a validation set-driven Softmax weight allocation, specifically implemented as follows:

[0059] Validation accuracy α of each branch k The fusion weights are generated after adjusting the temperature parameter τ:

[0060]

[0061] Where k represents the branch index, K represents the total number of branches, and w k Let α represent the dynamic fusion weight of the k-th branch. k Let α represent the classification accuracy of the k-th branch on the validation set. j This represents the classification accuracy of the j-th branch on the validation set, and τ represents the temperature parameter (which controls the smoothness of the weight distribution).

[0062] The final decision is a weighted sum:

[0063]

[0064] Where k represents the branch index. w represents the predicted probability distribution after final weighted fusion. k This represents the dynamic fusion weight of the k-th branch. Let represent the predicted probability distribution of the k-th branch.

[0065] Corresponding classification loss:

[0066]

[0067] in The loss represents the main classification loss after fusion, where N represents the total number of training samples, and y represents the total number of training samples. i This represents the true label of the i-th sample. This represents the predicted probability distribution after final weighted fusion.

[0068] S3.3: The joint supervision optimization module generates auxiliary supervision signals by fusing branch features and making decisions. The specific implementation method is as follows:

[0069] The multi-branch feature hk is concatenated and then the loss is calculated using an auxiliary classifier.

[0070]

[0071] in Let y represent the auxiliary classification loss, N represent the total number of training samples, and y represent the total number of training samples. i This represents the true label of the i-th sample. This represents the prediction result of the auxiliary classifier after concatenating all branch features.

[0072] The total loss function is a weighted sum:

[0073]

[0074] in This represents the total loss function of the model, where λ∈(0,1) are the balance coefficients. This represents the main classification loss after fusion. This represents the auxiliary classification loss.

[0075] Step 4: Train the two-stage multi-scale neural network model in stages;

[0076] Using the Adam optimizer (lr = 1e-4), with weighted cross-entropy loss:

[0077]

[0078] in This represents the total loss function of the model, where λ∈(0,1) are the balance coefficients. This represents the main classification loss after fusion. This represents the auxiliary classification loss.

[0079] Train for 200 epochs (batch_size=64) on an NVIDIA RTX 4060ti.

[0080] Step 5: Perform zero-calibration classification prediction and cross-subject verification on the EEG signals of unseen target subjects.

[0081] Leave-one-out cross-validation was employed. As shown in Table 1, TSMSNet achieved an average accuracy of 61.18% on subjects A01-A09, significantly outperforming the comparative methods.

[0082] Table 1

[0083] Subjects EEGNet DCN TIDNet Conformer FBCNet TSMSNet A01 51.00 48.91 48.20 51.17 52.71 66.56 A02 44.10 39.52 42.56 41.30 37.37 49.57 A03 59.25 62.33 58.42 63.52 65.43 78.10 A04 48.61 48.03 44.03 47.77 38.87 53.78 A05 55.73 56.79 53.99 53.23 40.67 51.02 A06 47.16 47.74 48.46 49.68 36.35 49.57 A07 60.11 60.77 59.68 65.75 45.90 68.32 A08 57.55 54.21 50.63 60.63 63.98 70.70 A09 49.02 47.51 52.89 49.70 57.12 62.96 mean 52.50 51.76 50.98 53.64 48.71 61.18 Standard deviation 5.46 6.91 5.55 7.58 10.8 9.93

[0084] This invention is the first to achieve a system integration of multi-scale feature extraction, dynamic weighted fusion, and joint supervised optimization. It not only improves cross-subject classification performance to a new level, but also solves the key problems of gradient conflict and branch contribution quantification in traditional end-to-end methods through a two-stage training strategy, providing a reliable technical solution for the clinical application of zero-calibration brain-computer interface systems.

[0085] This example demonstrates TSMSNet's groundbreaking performance in classifying EEG signals across subjects' motor imagery. Through a two-stage, multi-scale learning architecture, this model successfully addresses key challenges of traditional methods in feature fusion and cross-subject generalization.

[0086] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A generalized classification method for EEG signal domain based on a two-stage multi-scale neural network, characterized by: Includes the following steps: Step 1: Use EEG acquisition equipment to collect EEG signals of motor imagery from multiple subjects; Step 2: Preprocess the acquired EEG signals, including filtering, noise reduction, and segmentation; Step 3: Construct a two-stage multi-scale neural network, including: Multi-scale classification network module: It uses a parallel branching structure to capture complementary spatiotemporal features and make independent decisions; Dynamic weighted fusion module: Trains the model in one stage and generates adaptive generalization weights based on the accuracy of the validation set; Joint supervised optimization module: It integrates branch features for decision-making and introduces auxiliary classification loss to guide features to reach the global optimal solution; Step 4: Train the two-stage multi-scale neural network model in stages; Step 5: Perform zero-calibration classification prediction on the EEG signals of subjects who are not seen as targets.

2. The EEG signal domain generalization classification method based on a two-stage multi-scale neural network according to claim 1, characterized in that: Step three specifically includes: S3.1: The multi-scale classification network module uses differentiated temporal convolutional kernels, specifically implemented as follows: four parallel branches are constructed, each configured with a depthwise separable convolutional kernel of (1, FS / 16), (1, FS / 8), (1, FS / 4), and (1, FS / 2), respectively, where FS is the sampling frequency of the EEG acquisition device. Each branch independently optimizes the cross-entropy loss. Where k represents the branch index. Let y represent the cross-entropy loss of the k-th branch, where N represents the total number of training samples. i This represents the true label of the i-th sample. This represents the predicted probability distribution of the k-th branch for the i-th sample. Each convolutional layer is followed by a batch normalization and Dropout (p=0.3) layer. S3.2: The dynamic weighted fusion module adopts a validation set-driven Softmax weight allocation, specifically implemented as follows: Validation accuracy α of each branch k The fusion weights are generated after adjusting the temperature parameter τ: Where k represents the branch index, K represents the total number of branches, and w k Let α represent the dynamic fusion weight of the k-th branch. k Let α represent the classification accuracy of the k-th branch on the validation set. j τ represents the classification accuracy of the j-th branch on the validation set, and τ represents the temperature parameter. The final decision is a weighted sum: Where k represents the branch index. w represents the predicted probability distribution after final weighted fusion. k This represents the dynamic fusion weight of the k-th branch. This represents the predicted probability distribution of the k-th branch; Corresponding classification loss: in The loss represents the main classification loss after fusion, where N represents the total number of training samples, and y represents the total number of training samples. i This represents the true label of the i-th sample. This represents the predicted probability distribution after final weighted fusion; S3.3: The joint supervision optimization module generates auxiliary supervision signals by fusing branch features and making decisions. The specific implementation method is as follows: The multi-branch feature hk is concatenated and then the loss is calculated using an auxiliary classifier. in Let y represent the auxiliary classification loss, N represent the total number of training samples, and y represent the total number of training samples. i This represents the true label of the i-th sample. This represents the prediction result of the auxiliary classifier after concatenating all branch features; The total loss function is a weighted sum: in This represents the total loss function of the model, where λ∈(0,1) are the balance coefficients. This represents the main classification loss after fusion. This represents the auxiliary classification loss.

Images